[jira] [Assigned] (IMPALA-8476) Replace Sentry admin check workaround with proper Sentry API
[ https://issues.apache.org/jira/browse/IMPALA-8476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fang-Yu Rao reassigned IMPALA-8476: --- Assignee: Fang-Yu Rao > Replace Sentry admin check workaround with proper Sentry API > > > Key: IMPALA-8476 > URL: https://issues.apache.org/jira/browse/IMPALA-8476 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Fredy Wijaya >Assignee: Fang-Yu Rao >Priority: Minor > Labels: newbie > > Impala uses a workaround to detect if a user is a Sentry admin by calling > list privileges API before Sentry didn't provide an API to tell if user is a > Sentry admin: > https://github.com/apache/impala/blob/d820952d86d34ba887c55a09e58b735cbef866c2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L378-L393 > https://issues.apache.org/jira/browse/SENTRY-2440 supports a new API. We > should update the Impala to use the new proper Sentry API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8476) Replace Sentry admin check workaround with proper Sentry API
Fredy Wijaya created IMPALA-8476: Summary: Replace Sentry admin check workaround with proper Sentry API Key: IMPALA-8476 URL: https://issues.apache.org/jira/browse/IMPALA-8476 Project: IMPALA Issue Type: Improvement Components: Catalog Reporter: Fredy Wijaya Impala uses a workaround to detect if a user is a Sentry admin by calling list privileges API before Sentry didn't provide an API to tell if user is a Sentry admin: https://github.com/apache/impala/blob/d820952d86d34ba887c55a09e58b735cbef866c2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L378-L393 https://issues.apache.org/jira/browse/SENTRY-2440 supports a new API. We should update the Impala to use the new proper Sentry API. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8476) Replace Sentry admin check workaround with proper Sentry API
Fredy Wijaya created IMPALA-8476: Summary: Replace Sentry admin check workaround with proper Sentry API Key: IMPALA-8476 URL: https://issues.apache.org/jira/browse/IMPALA-8476 Project: IMPALA Issue Type: Improvement Components: Catalog Reporter: Fredy Wijaya Impala uses a workaround to detect if a user is a Sentry admin by calling list privileges API before Sentry didn't provide an API to tell if user is a Sentry admin: https://github.com/apache/impala/blob/d820952d86d34ba887c55a09e58b735cbef866c2/fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java#L378-L393 https://issues.apache.org/jira/browse/SENTRY-2440 supports a new API. We should update the Impala to use the new proper Sentry API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8469) Admit memory not set in backend descriptor for coordinator-only nodes
[
https://issues.apache.org/jira/browse/IMPALA-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8469.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Admit memory not set in backend descriptor for coordinator-only nodes
> -
>
> Key: IMPALA-8469
> URL: https://issues.apache.org/jira/browse/IMPALA-8469
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Ian Buss
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: admission-control, resource-management
> Fix For: Impala 3.3.0
>
>
> When configuring admission control with dedicated coordinator daemons,
> queries in pools with memory limits fail with the admission rejections like
> the following:
> {noformat}
> Rejected query from pool root.default: request memory needed 3.00 GB per node
> is greater than memory available for admission 0 of coord1.example.com:22000.
> Use the MEM_LIMIT query option to indicate how much memory is required per
> node.{noformat}
> Tracing this in the code leads us to line 576 of {{admission-controller.cc}}
> and therefore to suspect that the local {{TBackendDescriptor}}
> ({{local_backend_descriptor_}}) for the coordinator node in {{scheduler.cc}}
> never has {{admit_mem_limit}} set, and thus ends up with the default value of
> 0.
> The issue goes away if NO_SPECIALIZATION is used instead of COORDINATOR_ONLY.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8469) Admit memory not set in backend descriptor for coordinator-only nodes
[
https://issues.apache.org/jira/browse/IMPALA-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8469.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Admit memory not set in backend descriptor for coordinator-only nodes
> -
>
> Key: IMPALA-8469
> URL: https://issues.apache.org/jira/browse/IMPALA-8469
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Ian Buss
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: admission-control, resource-management
> Fix For: Impala 3.3.0
>
>
> When configuring admission control with dedicated coordinator daemons,
> queries in pools with memory limits fail with the admission rejections like
> the following:
> {noformat}
> Rejected query from pool root.default: request memory needed 3.00 GB per node
> is greater than memory available for admission 0 of coord1.example.com:22000.
> Use the MEM_LIMIT query option to indicate how much memory is required per
> node.{noformat}
> Tracing this in the code leads us to line 576 of {{admission-controller.cc}}
> and therefore to suspect that the local {{TBackendDescriptor}}
> ({{local_backend_descriptor_}}) for the coordinator node in {{scheduler.cc}}
> never has {{admit_mem_limit}} set, and thus ends up with the default value of
> 0.
> The issue goes away if NO_SPECIALIZATION is used instead of COORDINATOR_ONLY.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-8469) Admit memory not set in backend descriptor for coordinator-only nodes
[
https://issues.apache.org/jira/browse/IMPALA-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830782#comment-16830782
]
ASF subversion and git services commented on IMPALA-8469:
-
Commit d820952d86d34ba887c55a09e58b735cbef866c2 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d820952 ]
IMPALA-8469: admit_mem_limit for dedicated coordinator
Refactored to avoid the code duplication that resulted in this bug:
* admit_mem_limit is calculated once in ExecEnv
* The local backend descriptor is always constructed with
a static helper: Scheduler::BuildLocalBackendDescriptor()
I chose to factor it in this way, in part, to avoid invasive
changes to scheduler-test, which currently doesn't depend on
ExecEnv or ImpalaServer.
Testing:
Added basic test that reproduces the bug.
Change-Id: Iaceb21b753b9b021bedc4187c0d44aaa6a626521
Reviewed-on: http://gerrit.cloudera.org:8080/13180
Reviewed-by: Tim Armstrong
Tested-by: Impala Public Jenkins
> Admit memory not set in backend descriptor for coordinator-only nodes
> -
>
> Key: IMPALA-8469
> URL: https://issues.apache.org/jira/browse/IMPALA-8469
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Ian Buss
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: admission-control, resource-management
>
> When configuring admission control with dedicated coordinator daemons,
> queries in pools with memory limits fail with the admission rejections like
> the following:
> {noformat}
> Rejected query from pool root.default: request memory needed 3.00 GB per node
> is greater than memory available for admission 0 of coord1.example.com:22000.
> Use the MEM_LIMIT query option to indicate how much memory is required per
> node.{noformat}
> Tracing this in the code leads us to line 576 of {{admission-controller.cc}}
> and therefore to suspect that the local {{TBackendDescriptor}}
> ({{local_backend_descriptor_}}) for the coordinator node in {{scheduler.cc}}
> never has {{admit_mem_limit}} set, and thus ends up with the default value of
> 0.
> The issue goes away if NO_SPECIALIZATION is used instead of COORDINATOR_ONLY.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8280) Implement SHOW GRANT USER
[
https://issues.apache.org/jira/browse/IMPALA-8280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8280.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Implement SHOW GRANT USER
>
>
> Key: IMPALA-8280
> URL: https://issues.apache.org/jira/browse/IMPALA-8280
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog, Frontend
>Reporter: Fredy Wijaya
>Assignee: Austin Nobis
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> Syntax:
> {noformat}
> SHOW GRANT USER [ON ]
> {noformat}
> The command is to show list of privileges for a given user with an optional
> ON clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8281) Implement SHOW GRANT GROUP
[
https://issues.apache.org/jira/browse/IMPALA-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8281.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Implement SHOW GRANT GROUP
> --
>
> Key: IMPALA-8281
> URL: https://issues.apache.org/jira/browse/IMPALA-8281
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog, Frontend
>Reporter: Fredy Wijaya
>Assignee: Austin Nobis
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> Syntax:
> {noformat}
> SHOW GRANT GROUP [ON ]
> {noformat}
> The command is to show list of privileges for a given group with an optional
> ON clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8281) Implement SHOW GRANT GROUP
[
https://issues.apache.org/jira/browse/IMPALA-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8281.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Implement SHOW GRANT GROUP
> --
>
> Key: IMPALA-8281
> URL: https://issues.apache.org/jira/browse/IMPALA-8281
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog, Frontend
>Reporter: Fredy Wijaya
>Assignee: Austin Nobis
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> Syntax:
> {noformat}
> SHOW GRANT GROUP [ON ]
> {noformat}
> The command is to show list of privileges for a given group with an optional
> ON clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8280) Implement SHOW GRANT USER
[
https://issues.apache.org/jira/browse/IMPALA-8280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya resolved IMPALA-8280.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Implement SHOW GRANT USER
>
>
> Key: IMPALA-8280
> URL: https://issues.apache.org/jira/browse/IMPALA-8280
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog, Frontend
>Reporter: Fredy Wijaya
>Assignee: Austin Nobis
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> Syntax:
> {noformat}
> SHOW GRANT USER [ON ]
> {noformat}
> The command is to show list of privileges for a given user with an optional
> ON clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-7973) Add support for fine-grained updates at partition level
[ https://issues.apache.org/jira/browse/IMPALA-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830765#comment-16830765 ] ASF subversion and git services commented on IMPALA-7973: - Commit 3ad5b3fba202bf6809986a86f5e041da656f0a88 in impala's branch refs/heads/master from Anurag Mantripragada [ https://gitbox.apache.org/repos/asf?p=impala.git;h=3ad5b3f ] IMPALA-7973: Add support for fine grained events processing for partition level HMS events. This patch adds support for fine grained updates for add/drop/alter partition events. Currently, partition events invalidate the table. This can be expensive for large tables. Here, we refresh affected partitions in case of add/drop/alter partition events. HMS processes add/drop partitions in a transaction, which means there may be multiple partitions affected in a single add/drop event. We try to refresh all these partitions in a loop. If any of the partition refresh fails, we throw MetastoreNotificationNeedsInvalidateException to mandate a manual invalidate for event processing to continue. Testing: Modified pre-existing tests for partition events to instead test if partitions are added/dropped/altered when event processing is enabled. Change-Id: I213401329f3965dd81055197792ccf8a05368af5 Reviewed-on: http://gerrit.cloudera.org:8080/13111 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Add support for fine-grained updates at partition level > --- > > Key: IMPALA-7973 > URL: https://issues.apache.org/jira/browse/IMPALA-7973 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Anurag Mantripragada >Priority: Major > > When data is inserted into a partition or a new partition is created in a > large table, we should not be invalidating the whole table. Instead it should > be possible to refresh/add/drop certain partitions on the table directly > based on the event information. This would help with the performance of > subsequent access to the table by avoiding reloading the large table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8281) Implement SHOW GRANT GROUP
[
https://issues.apache.org/jira/browse/IMPALA-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830767#comment-16830767
]
ASF subversion and git services commented on IMPALA-8281:
-
Commit d533cb1d43879cfa3892726bcae6316dea212754 in impala's branch
refs/heads/master from Austin Nobis
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d533cb1 ]
IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Add support for SHOW GRANT statements for Apache Ranger. This patch also
adds the RangerImpaladAuthorizationManager as the show grant statement
is called from impalad. The new supported syntax is:
SHOW GRANT USER/GROUP ON server;
SHOW GRANT USER/GROUP ON database ;
SHOW GRANT USER/GROUP ON uri ;
SHOW GRANT USER/GROUP ON table .;
SHOW GRANT USER/GROUP ON column ..;
The following syntax is valid SQL, but is not supported currently by the
Apache Ranger integration with Impala:
SHOW GRANT USER/GROUP
Testing:
- Ran all FE unit tests
- Ran authorization E2E tests
- Updated test_ranger to use show grant statement for verification of
granted privileges
Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4
Reviewed-on: http://gerrit.cloudera.org:8080/13074
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Implement SHOW GRANT GROUP
> --
>
> Key: IMPALA-8281
> URL: https://issues.apache.org/jira/browse/IMPALA-8281
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog, Frontend
>Reporter: Fredy Wijaya
>Assignee: Austin Nobis
>Priority: Critical
>
> Syntax:
> {noformat}
> SHOW GRANT GROUP [ON ]
> {noformat}
> The command is to show list of privileges for a given group with an optional
> ON clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8280) Implement SHOW GRANT USER
[
https://issues.apache.org/jira/browse/IMPALA-8280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830766#comment-16830766
]
ASF subversion and git services commented on IMPALA-8280:
-
Commit d533cb1d43879cfa3892726bcae6316dea212754 in impala's branch
refs/heads/master from Austin Nobis
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d533cb1 ]
IMPALA-8280, IMPALA-8281: Add support for show grant user/group with Ranger
Add support for SHOW GRANT statements for Apache Ranger. This patch also
adds the RangerImpaladAuthorizationManager as the show grant statement
is called from impalad. The new supported syntax is:
SHOW GRANT USER/GROUP ON server;
SHOW GRANT USER/GROUP ON database ;
SHOW GRANT USER/GROUP ON uri ;
SHOW GRANT USER/GROUP ON table .;
SHOW GRANT USER/GROUP ON column ..;
The following syntax is valid SQL, but is not supported currently by the
Apache Ranger integration with Impala:
SHOW GRANT USER/GROUP
Testing:
- Ran all FE unit tests
- Ran authorization E2E tests
- Updated test_ranger to use show grant statement for verification of
granted privileges
Change-Id: Ic46fb9fc36c9e11ec78d5840d22eb0668150c2a4
Reviewed-on: http://gerrit.cloudera.org:8080/13074
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Implement SHOW GRANT USER
>
>
> Key: IMPALA-8280
> URL: https://issues.apache.org/jira/browse/IMPALA-8280
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog, Frontend
>Reporter: Fredy Wijaya
>Assignee: Austin Nobis
>Priority: Critical
>
> Syntax:
> {noformat}
> SHOW GRANT USER [ON ]
> {noformat}
> The command is to show list of privileges for a given user with an optional
> ON clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8455) GET_TABLE failed with InvalidStorageDescriptorException
[ https://issues.apache.org/jira/browse/IMPALA-8455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830768#comment-16830768 ] ASF subversion and git services commented on IMPALA-8455: - Commit c1b0a073938c144e9bf33901bd4df6dcda0f09ec in impala's branch refs/heads/master from Bharath Vissapragada [ https://gitbox.apache.org/repos/asf?p=impala.git;h=c1b0a07 ] IMPALA-8455: Gracefully handle failed loads in GET_TABLES req Some HS2 clients (like Hue) rely on the GET_TABLES request to populate table meta information. In local catalog mode, for any table loading failures, this call throws a cryptic exception that the callers do not expect. The original ImpaladCatalog implementation gracefully handled this by silently ignoring such tables. This patch replicates that behavior in LocalCatalog mode. Testing: Added a unit test that consistently fails without the patch. Change-Id: I89cc2c585aa83a9aea5343f0d0383ba72bafd618 Reviewed-on: http://gerrit.cloudera.org:8080/13186 Reviewed-by: Bharath Vissapragada Tested-by: Impala Public Jenkins > GET_TABLE failed with InvalidStorageDescriptorException > --- > > Key: IMPALA-8455 > URL: https://issues.apache.org/jira/browse/IMPALA-8455 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 3.2.0 >Reporter: Xiaomin Zhang >Priority: Critical > > When ImpalaD is enabled local catalog mode, GET_TABLE call returns error when > some table is using unsupported serde. > To reproduce: > 1) Enable local catalog mode: > --use_local_catalog=true > --catalog_topic_mode=minimal > --invalidate_tables_timeout_s=60 > 2) Create an CSV table: > create table sample(id int) > row format serde 'org.apache.hadoop.hive.serde2.OpenCSVSerde' > stored as textfile; > 3) In HUE, it shows below error while loading the tables: > *LocalCatalogException: Could not load table default.sample from metastore > CAUSED BY: TException: > TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, > error_msgs:[TableLoadingException: Failed to load metadata for table: > default.sample CAUSED BY: InvalidStorageDescriptorException: Impala does not > support tables of this type. REASON: SerDe library > 'org.apache.hadoop.hive.serde2.OpenCSVSerde' is not supported.]), > lookup_status:OK)* > 4) "SHOW TABLES" is not affected and runs fine. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8475) buildall.sh fails with undefined CMAKE_BUILD_TYPE_LIST on Centos
Joe McDonnell created IMPALA-8475:
-
Summary: buildall.sh fails with undefined CMAKE_BUILD_TYPE_LIST on
Centos
Key: IMPALA-8475
URL: https://issues.apache.org/jira/browse/IMPALA-8475
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
Assignee: Joe McDonnell
A recent change added a bash array to keep track of which build types had been
specified to buildall.sh. On Ubuntu, everything seems to work. On Centos, this
is producing an error:
{noformat}
15:23:15
/data/jenkins/workspace/impala-asf-master-core/repos/Impala/buildall.sh: line
316: CMAKE_BUILD_TYPE_LIST[0]: unbound variable{noformat}
Given that this is on a debug build, that should mean that the
CMAKE_BUILD_TYPE_LIST is empty. The current theory is that Ubuntu and Centos
treat [[ -v VARNAME ]] differently, so in the following code, we are entering
the block even with an empty list:
{code:java}
if [[ -v CMAKE_BUILD_TYPE_LIST ]]; then
if [[ ${#CMAKE_BUILD_TYPE_LIST[@]} -gt 1 ]]; then
echo "ERROR: more than one CMake build type defined:
${CMAKE_BUILD_TYPE_LIST[@]}"
exit 1
fi
CMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE_LIST[0]}
fi
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8475) buildall.sh fails with undefined CMAKE_BUILD_TYPE_LIST on Centos
Joe McDonnell created IMPALA-8475:
-
Summary: buildall.sh fails with undefined CMAKE_BUILD_TYPE_LIST on
Centos
Key: IMPALA-8475
URL: https://issues.apache.org/jira/browse/IMPALA-8475
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
Assignee: Joe McDonnell
A recent change added a bash array to keep track of which build types had been
specified to buildall.sh. On Ubuntu, everything seems to work. On Centos, this
is producing an error:
{noformat}
15:23:15
/data/jenkins/workspace/impala-asf-master-core/repos/Impala/buildall.sh: line
316: CMAKE_BUILD_TYPE_LIST[0]: unbound variable{noformat}
Given that this is on a debug build, that should mean that the
CMAKE_BUILD_TYPE_LIST is empty. The current theory is that Ubuntu and Centos
treat [[ -v VARNAME ]] differently, so in the following code, we are entering
the block even with an empty list:
{code:java}
if [[ -v CMAKE_BUILD_TYPE_LIST ]]; then
if [[ ${#CMAKE_BUILD_TYPE_LIST[@]} -gt 1 ]]; then
echo "ERROR: more than one CMake build type defined:
${CMAKE_BUILD_TYPE_LIST[@]}"
exit 1
fi
CMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE_LIST[0]}
fi
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
[ https://issues.apache.org/jira/browse/IMPALA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Garg updated IMPALA-8473: Priority: Critical (was: Major) > Refactor lineage publication mechanism to allow for different consumers > --- > > Key: IMPALA-8473 > URL: https://issues.apache.org/jira/browse/IMPALA-8473 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend >Reporter: radford nguyen >Priority: Critical > > Impetus for this change is to allow lineage to be consumed by Atlas via Kafka. > Approach should be similar to that of choosing authorization provider, where > the publication strategy can be chosen at runtime via configuration flag(s). > Scope of this ticket is to move lineage publication to the fe and add > appropriate hooks that a user can implement -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
[ https://issues.apache.org/jira/browse/IMPALA-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] radford nguyen updated IMPALA-8473: --- Description: Impetus for this change is to allow lineage to be consumed by Atlas via Kafka. Approach should be similar to that of choosing authorization provider, where the publication strategy can be chosen at runtime via configuration flag(s). Scope of this ticket is to move lineage publication to the fe and add appropriate hooks that a user can implement was: Impetus for this change is to allow lineage to be consumed by Altus via Kafka. Approach should be similar to that of choosing authorization provider, where the publication strategy can be chosen at runtime via configuration flag(s). Scope of this ticket is to move lineage publication to the fe and add appropriate hooks that a user can implement > Refactor lineage publication mechanism to allow for different consumers > --- > > Key: IMPALA-8473 > URL: https://issues.apache.org/jira/browse/IMPALA-8473 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend >Reporter: radford nguyen >Priority: Major > > Impetus for this change is to allow lineage to be consumed by Atlas via Kafka. > Approach should be similar to that of choosing authorization provider, where > the publication strategy can be chosen at runtime via configuration flag(s). > Scope of this ticket is to move lineage publication to the fe and add > appropriate hooks that a user can implement -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8419) Fetch metastore configuration values to detect misconfigured setups
[ https://issues.apache.org/jira/browse/IMPALA-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830734#comment-16830734 ] ASF subversion and git services commented on IMPALA-8419: - Commit 66fda62213472ab1a5e07e8719c73de4d4cd7ab4 in impala's branch refs/heads/master from Bharath Krishna [ https://gitbox.apache.org/repos/asf?p=impala.git;h=66fda62 ] IMPALA-8419 : Validate event processing related configurations Using the Metastore API to get the configuration values, verify that the configurations needed for event processing are set correctly. Also check that the parameters required for event processing is not filtered out by the Hive config METASTORE_PARAMETER_EXCLUDE_PATTERNS. This validation is done while creating the event processor and throws CatalogException if the configuration is incorrect. Testing - Added unit tests Change-Id: I94c2783e36287a65122003aa55d8075a806bc606 Reviewed-on: http://gerrit.cloudera.org:8080/13019 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Fetch metastore configuration values to detect misconfigured setups > --- > > Key: IMPALA-8419 > URL: https://issues.apache.org/jira/browse/IMPALA-8419 > Project: IMPALA > Issue Type: Sub-task >Reporter: Bharathkrishna Guruvayoor Murali >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > > HMS provides a API to get the server side configuration value. This can be > very useful to detect misconfigured setups. Currently, we rely on user to set > up the configurations using safety valves. It would be good if we can verify > that the required configurations for this feature are correctly set in HMS > before starting CatalogService so that we don't error out later in the > runtime due to misconfigurations. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8468) buildall.sh should warn that asan/ubsan/... are exclusive
[ https://issues.apache.org/jira/browse/IMPALA-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830733#comment-16830733 ] ASF subversion and git services commented on IMPALA-8468: - Commit 7d80aedc3e0ee634e2a229fba8388f01097189de in impala's branch refs/heads/master from Csaba Ringhofer [ https://gitbox.apache.org/repos/asf?p=impala.git;h=7d80aed ] IMPALA-8468: buildall.sh should warn that asan/ubsan/... are exclusive Before the fix "buidall.sh -asan -ubsan -tsan -tidy" ran without giving any warning, but actually only tsan did have effect. Added a check for this. Change-Id: Iecaf1e1987a92dd911b30cf11b6e762320bdc4e5 Reviewed-on: http://gerrit.cloudera.org:8080/13177 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > buildall.sh should warn that asan/ubsan/... are exclusive > - > > Key: IMPALA-8468 > URL: https://issues.apache.org/jira/browse/IMPALA-8468 > Project: IMPALA > Issue Type: Bug > Components: Backend, Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Csaba Ringhofer >Priority: Minor > > "buidall.sh -asan -ubsan -tsan -tidy" runs without giving any warning, but > actually only tsan will have effect. See > https://github.com/apache/impala/blob/931a8f0ba7f45d5b1608e62aff397b517b943e95/buildall.sh#L308 > for the logic behind this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-5543) Hide "advanced" query options from default list.
[ https://issues.apache.org/jira/browse/IMPALA-5543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Kaszab updated IMPALA-5543: - Fix Version/s: Impala 2.11.0 > Hide "advanced" query options from default list. > > > Key: IMPALA-5543 > URL: https://issues.apache.org/jira/browse/IMPALA-5543 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.10.0 >Reporter: Tim Armstrong >Assignee: Gabor Kaszab >Priority: Major > Labels: usability > Fix For: Impala 2.11.0 > > > After IMPALA-2181 is fixed, we should re-evaluate the existing query options > and hide the ones that are not useful for more users. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6423) HDFS scanners do not get cancelled in a timely manner
[
https://issues.apache.org/jira/browse/IMPALA-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Kaszab updated IMPALA-6423:
-
Fix Version/s: Impala 2.12.0
> HDFS scanners do not get cancelled in a timely manner
> -
>
> Key: IMPALA-6423
> URL: https://issues.apache.org/jira/browse/IMPALA-6423
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.11.0
>Reporter: Quanlong Huang
>Assignee: Gabor Kaszab
>Priority: Critical
> Fix For: Impala 2.12.0
>
> Attachments: hdfs_scan_node_cancellation.log
>
>
> The non-MT scanner (MT_DOP=0) does not check RuntimeState::is_cancelled() for
> cancellation. Instead, it checks ScannerContext::cancelled() which only
> returns HdfsScanNode:: Done(). Thus, the scanner can't notice the
> cancellation in time. The FragmentInstance of it will still try to send out
> data until it reaches the 2m time-out.
> This can be reproduced by
> {code:java}
> bin/impala-shell.sh -q "set debug_action=1:PREPARE:FAIL; select * from
> functional_parquet.alltypes"
> {code}
> Then tail -f the log. You can find logs like
> {code:java}
> I0119 05:39:25.453212 14051 impala-server.cc:1165] ReportExecStatus():
> Received report for unknown query ID (probably closed or cancelled):
> 714ff9fe107a7c25:a2344b18
> I0119 05:39:25.456718 14054 impala-server.cc:1165] ReportExecStatus():
> Received report for unknown query ID (probably closed or cancelled):
> 714ff9fe107a7c25:a2344b18
> I0119 05:39:30.408521 13825 impala-server.cc:1165] ReportExecStatus():
> Received report for unknown query ID (probably closed or cancelled):
> 714ff9fe107a7c25:a2344b18
> I0119 05:39:30.408646 14022 query-state.cc:401] Cancel:
> query_id=714ff9fe107a7c25:a2344b18
> I0119 05:39:35.409497 13825 impala-server.cc:1165] ReportExecStatus():
> Received report for unknown query ID (probably closed or cancelled):
> 714ff9fe107a7c25:a2344b18
> I0119 05:39:35.409601 14022 query-state.cc:401] Cancel:
> query_id=714ff9fe107a7c25:a2344b18
> ..
> I0119 05:41:25.429842 13825 impala-server.cc:1165] ReportExecStatus():
> Received report for unknown query ID (probably closed or cancelled):
> 714ff9fe107a7c25:a2344b18
> I0119 05:41:25.429960 14022 query-state.cc:401] Cancel:
> query_id=714ff9fe107a7c25:a2344b18
> {code}
> and finally the time-out logs
> {code:java}
> I0119 05:41:25.445632 13811 data-stream-mgr.cc:130] Datastream sender
> timed-out waiting for recvr for fragment instance:
> 714ff9fe107a7c25:a2344b18 (time-out was: 2m). Increase
> --datastream_sender_timeout_ms if you see this message frequently.
> I0119 05:41:25.445688 13811 data-stream-mgr.cc:191] DataStreamMgr::AddData():
> Sender timed out waiting for receiver fragment instance:
> 714ff9fe107a7c25:a2344b18, dest node: 1
> E0119 05:41:25.445838 14015 data-stream-sender.cc:278] channel send to
> iadhadoop-c39:22000 failed: Sender timed out waiting for receiver fragment
> instance: 714ff9fe107a7c25:a2344b18, dest node: 1
> I0119 05:41:25.446475 13825 impala-server.cc:1165] ReportExecStatus():
> Received report for unknown query ID (probably closed or cancelled):
> 714ff9fe107a7c25:a2344b18
> I0119 05:41:25.446524 14015 query-state.cc:401] Cancel:
> query_id=714ff9fe107a7c25:a2344b18
> I0119 05:41:25.447132 14015 query-state.cc:388] Instance completed.
> instance_id=714ff9fe107a7c25:a2344b180002 #in-flight=0
> status=DATASTREAM_SENDER_TIMEOUT: Sender timed out waiting for receiver
> fragment instance: 714ff9fe107a7c25:a2344b18, dest node: 1
> I0119 05:41:25.447149 14015 query-state.cc:401] Cancel:
> query_id=714ff9fe107a7c25:a2344b18
> I0119 05:41:25.447201 14015 query-exec-mgr.cc:149] ReleaseQueryState():
> query_id=714ff9fe107a7c25:a2344b18 refcnt=1
> I0119 05:41:25.448110 13843 data-stream-mgr.cc:130] Datastream sender
> timed-out waiting for recvr for fragment instance:
> 714ff9fe107a7c25:a2344b18 (time-out was: 2m). Increase
> --datastream_sender_timeout_ms if you see this message frequently.
> I0119 05:41:25.448130 13843 data-stream-mgr.cc:191] DataStreamMgr::AddData():
> Sender timed out waiting for receiver fragment instance:
> 714ff9fe107a7c25:a2344b18, dest node: 1
> I0119 05:41:25.454107 13852 data-stream-mgr.cc:130] Datastream sender
> timed-out waiting for recvr for fragment instance:
> 714ff9fe107a7c25:a2344b18 (time-out was: 2m). Increase
> --datastream_sender_timeout_ms if you see this message frequently.
> I0119 05:41:25.454149 13852 data-stream-mgr.cc:191] DataStreamMgr::AddData():
> Sender timed out waiting for receiver fragment instance:
> 714ff9fe107a7c25:a2344b18, dest node:
[jira] [Updated] (IMPALA-8448) Impala Doc: Doc the feature for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-8448: Description: https://gerrit.cloudera.org/#/c/13199/ > Impala Doc: Doc the feature for alter_database events > - > > Key: IMPALA-8448 > URL: https://issues.apache.org/jira/browse/IMPALA-8448 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_33 > > https://gerrit.cloudera.org/#/c/13199/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8448) Impala Doc: Doc the feature for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8448 started by Alex Rodoni. --- > Impala Doc: Doc the feature for alter_database events > - > > Key: IMPALA-8448 > URL: https://issues.apache.org/jira/browse/IMPALA-8448 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_33 > > https://gerrit.cloudera.org/#/c/13199/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8474) Impala Doc: 3.3 Release Notes
Alex Rodoni created IMPALA-8474: --- Summary: Impala Doc: 3.3 Release Notes Key: IMPALA-8474 URL: https://issues.apache.org/jira/browse/IMPALA-8474 Project: IMPALA Issue Type: Task Components: Docs Reporter: Alex Rodoni Assignee: Alex Rodoni -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-8474) Impala Doc: 3.3 Release Notes
Alex Rodoni created IMPALA-8474: --- Summary: Impala Doc: 3.3 Release Notes Key: IMPALA-8474 URL: https://issues.apache.org/jira/browse/IMPALA-8474 Project: IMPALA Issue Type: Task Components: Docs Reporter: Alex Rodoni Assignee: Alex Rodoni -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8458) Can't set numNull/maxSize/avgSize column stats with local catalog without also setting NDV
[
https://issues.apache.org/jira/browse/IMPALA-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-8458:
-
Assignee: Todd Lipcon (was: Tim Armstrong)
> Can't set numNull/maxSize/avgSize column stats with local catalog without
> also setting NDV
> --
>
> Key: IMPALA-8458
> URL: https://issues.apache.org/jira/browse/IMPALA-8458
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Todd Lipcon
>Priority: Critical
>
> Repro:
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] default> create table test_stats2(s
> string);
> +-+
> | summary |
> +-+
> | Table has been created. |
> +-+
> Fetched 1 row(s) in 0.36s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('avgSize'='1234');
> +-+
> | summary |
> +-+
> | Updated 0 partition(s) and 1 column(s). |
> +-+
> Fetched 1 row(s) in 0.14s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('maxSize'='1234');
> +-+
> | summary |
> +-+
> | Updated 0 partition(s) and 1 column(s). |
> +-+
> Fetched 1 row(s) in 0.10s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> invalidate metadata
> test_stats2;
> Fetched 0 row(s) in 0.03s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> Query: show column stats test_stats2
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.07s
> {noformat}
> I expected that the updates would take effect. Weirdly it doesn't happen for
> NDV and NULLS:
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('numDVs'='1234','numNulls'='12345');
> Query: alter table test_stats2 set column stats
> s('numDVs'='1234','numNulls'='12345')
> +-+
> | summary |
> +-+
> | Updated 0 partition(s) and 1 column(s). |
> +-+
> Fetched 1 row(s) in 0.12s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> Query: show column stats test_stats2
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | 1234 | 12345 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
>
[jira] [Assigned] (IMPALA-8459) Cannot delete impala/kudu table if backing kudu table dropped with local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-8459:
-
Assignee: Todd Lipcon (was: Tim Armstrong)
> Cannot delete impala/kudu table if backing kudu table dropped with local
> catalog
>
>
> Key: IMPALA-8459
> URL: https://issues.apache.org/jira/browse/IMPALA-8459
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Todd Lipcon
>Priority: Critical
> Labels: kudu
>
> test_delete_external_kudu_table and test_delete_managed_kudu_table fail with
> local catalog, e.g. with:
> {noformat}
> E HiveServer2Error: LocalCatalogException: Error opening Kudu table
> 'testimpalakuduintegration_1715_p3r46w.ogslbjblgv', Kudu error: the table
> does not exist: table_name: "testimpalakuduintegration_1715_p3r46w.ogslbjblgv"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8464) JdbcTest.testMetaDataGetColumnComments() fails with local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-8464:
-
Assignee: Todd Lipcon (was: Tim Armstrong)
> JdbcTest.testMetaDataGetColumnComments() fails with local catalog
> -
>
> Key: IMPALA-8464
> URL: https://issues.apache.org/jira/browse/IMPALA-8464
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0
>Reporter: Tim Armstrong
>Assignee: Todd Lipcon
>Priority: Major
>
> {noformat}
> Error Message
> Incorrect table comment expected:<[]> but was:<[table comment]>
> Stacktrace
> org.junit.ComparisonFailure: Incorrect table comment expected:<[]> but
> was:<[table comment]>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.impala.service.JdbcTest.testMetaDataGetColumnComments(JdbcTest.java:484)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> Standard Output
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils: Resolved authority: localhost:21050
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Using JDBC Driver Name:
> org.apache.hive.jdbc.HiveDriver
> 19/04/26 18:35:36 INFO testutil.ImpalaJdbcClient: Connecting to:
> jdbc:hive2://localhost:21050/default;auth=noSasl
> 19/04/26 18:35:36 INFO jdbc.Utils: Supplied authorities: localhost:21050
> 19/04/26 18:35:36 INFO jdbc.Utils:
[jira] [Updated] (IMPALA-3292) Kudu scanner should not fail if KeepAlive request fails
[ https://issues.apache.org/jira/browse/IMPALA-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated IMPALA-3292: Fix Version/s: Impala 2.7.0 > Kudu scanner should not fail if KeepAlive request fails > --- > > Key: IMPALA-3292 > URL: https://issues.apache.org/jira/browse/IMPALA-3292 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Kudu_Impala >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: Impala 2.7.0 > > > The KeepKuduScannerAlive() function can fail with ServiceUnavailable if it is > running against a very heavily-loaded tablet server. This is OK -- it should > ignore the warning and just try again after a short time, rather than failing > the whole query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8458) Can't set numNull/maxSize/avgSize column stats with local catalog without also setting NDV
[
https://issues.apache.org/jira/browse/IMPALA-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-8458:
--
Summary: Can't set numNull/maxSize/avgSize column stats with local catalog
without also setting NDV (was: Can't set maxSize and avgSize column stats with
local catalog)
> Can't set numNull/maxSize/avgSize column stats with local catalog without
> also setting NDV
> --
>
> Key: IMPALA-8458
> URL: https://issues.apache.org/jira/browse/IMPALA-8458
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>
> Repro:
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] default> create table test_stats2(s
> string);
> +-+
> | summary |
> +-+
> | Table has been created. |
> +-+
> Fetched 1 row(s) in 0.36s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('avgSize'='1234');
> +-+
> | summary |
> +-+
> | Updated 0 partition(s) and 1 column(s). |
> +-+
> Fetched 1 row(s) in 0.14s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('maxSize'='1234');
> +-+
> | summary |
> +-+
> | Updated 0 partition(s) and 1 column(s). |
> +-+
> Fetched 1 row(s) in 0.10s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> invalidate metadata
> test_stats2;
> Fetched 0 row(s) in 0.03s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> Query: show column stats test_stats2
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | -1 | -1 | -1 | -1 |
> +++--++--+--+
> Fetched 1 row(s) in 0.07s
> {noformat}
> I expected that the updates would take effect. Weirdly it doesn't happen for
> NDV and NULLS:
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('numDVs'='1234','numNulls'='12345');
> Query: alter table test_stats2 set column stats
> s('numDVs'='1234','numNulls'='12345')
> +-+
> | summary |
> +-+
> | Updated 0 partition(s) and 1 column(s). |
> +-+
> Fetched 1 row(s) in 0.12s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> Query: show column stats test_stats2
> +++--++--+--+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +++--++--+--+
> | s | STRING | 1234 | 12345 | -1 |
[jira] [Resolved] (IMPALA-8327) TestRPCTimeout::test_reportexecstatus_retry() times out on exhaustive build
[
https://issues.apache.org/jira/browse/IMPALA-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Tauber-Marshall resolved IMPALA-8327.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
IMPALA-2990 is fixed now
> TestRPCTimeout::test_reportexecstatus_retry() times out on exhaustive build
> ---
>
> Key: IMPALA-8327
> URL: https://issues.apache.org/jira/browse/IMPALA-8327
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Joe McDonnell
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
>
> An exhaustive build timed out while executing
> TestRPCTimeout::test_reportexecstatus_retry():
> {noformat}
> 20:22:45
> custom_cluster/test_rpc_timeout.py::TestRPCTimeout::test_reportexecstatus_jitter[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 09:50:18
> custom_cluster/test_rpc_timeout.py::TestRPCTimeout::test_reportexecstatus_retry[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 09:50:18
> 09:50:18 Tests TIMED OUT! {noformat}
> The impalad log shows a few messages like this:
> {noformat}
> I0319 20:22:56.376278 13826 impala-service-pool.cc:130] ReportExecStatus
> request on impala.ControlService from 127.0.0.1:46358 dropped due to
> backpressure. The service queue contains 0 items out of a maximum of
> 2147483647; memory consumption is 16.17 KB. Contents of service
> queue:{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8327) TestRPCTimeout::test_reportexecstatus_retry() times out on exhaustive build
[
https://issues.apache.org/jira/browse/IMPALA-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Tauber-Marshall resolved IMPALA-8327.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
IMPALA-2990 is fixed now
> TestRPCTimeout::test_reportexecstatus_retry() times out on exhaustive build
> ---
>
> Key: IMPALA-8327
> URL: https://issues.apache.org/jira/browse/IMPALA-8327
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Joe McDonnell
>Assignee: Thomas Tauber-Marshall
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
>
> An exhaustive build timed out while executing
> TestRPCTimeout::test_reportexecstatus_retry():
> {noformat}
> 20:22:45
> custom_cluster/test_rpc_timeout.py::TestRPCTimeout::test_reportexecstatus_jitter[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 09:50:18
> custom_cluster/test_rpc_timeout.py::TestRPCTimeout::test_reportexecstatus_retry[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> 09:50:18
> 09:50:18 Tests TIMED OUT! {noformat}
> The impalad log shows a few messages like this:
> {noformat}
> I0319 20:22:56.376278 13826 impala-service-pool.cc:130] ReportExecStatus
> request on impala.ControlService from 127.0.0.1:46358 dropped due to
> backpressure. The service queue contains 0 items out of a maximum of
> 2147483647; memory consumption is 16.17 KB. Contents of service
> queue:{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7511) translate() string function only works with ascii characters.
[
https://issues.apache.org/jira/browse/IMPALA-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-7511:
---
Fix Version/s: Impala 3.2.0
> translate() string function only works with ascii characters.
> -
>
> Key: IMPALA-7511
> URL: https://issues.apache.org/jira/browse/IMPALA-7511
> Project: IMPALA
> Issue Type: Task
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
> Fix For: Impala 3.2.0
>
>
> The query
> SELECT translate('Gánémílózú', 'áéíóú', 'aeiou') ;
> returns "Gaenaomalaza" instead of the desired "Ganemilozu".
> It seems that translate treats strings as 8 bit characters. So it only works
> with ascii.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6029) Crash in parquet::FileMetaData::~FileMetaData
[
https://issues.apache.org/jira/browse/IMPALA-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-6029:
--
Fix Version/s: Not Applicable
> Crash in parquet::FileMetaData::~FileMetaData
> -
>
> Key: IMPALA-6029
> URL: https://issues.apache.org/jira/browse/IMPALA-6029
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.11.0
>Reporter: Mostafa Mokhtar
>Assignee: Joe McDonnell
>Priority: Critical
> Fix For: Not Applicable
>
>
> Crash occurred while running concurrent TPCDS queries against KRPC branch
> https://github.com/michaelhkw/incubator-impala/tree/krpc-testing-hung
> stack
> {code}
> #0 0x00392c832625 in raise () from /lib64/libc.so.6
> #1 0x00392c833e05 in abort () from /lib64/libc.so.6
> #2 0x7f3353144a55 in os::abort(bool) () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #3 0x7f33532c4f87 in VMError::report_and_die() () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #4 0x7f335314996f in JVM_handle_linux_signal () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #5
> #6 ?? (warning: (Internal error: pc 0x0 in read in psymtab, but not in
> symtab.)
> ) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p3/include/boost/system/system_error.hpp:61
> warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)
> #7 0x00d8decb in _Destroy
> (this=0x7f31c92d6910, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:93
> #8 __destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:103
> #9 _Destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:126
> #10 _Destroy
> (this=0x7f31c92d6910, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:151
> #11 ~vector (this=0x7f31c92d6910, __in_chrg=) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_vector.h:424
> #12 ~RowGroup (this=0x7f31c92d6910, __in_chrg=) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/generated-sources/gen-cpp/parquet_types.h:1095
> #13 _Destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:93
> #14 __destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:103
> #15 _Destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:126
> #16 _Destroy (this=0x7f31c92d6910,
> __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:151
> #17 ~vector (this=0x7f31c92d6910, __in_chrg=) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_vector.h:424
> #18 parquet::FileMetaData::~FileMetaData (this=0x7f31c92d6910,
> __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/generated-sources/gen-cpp/parquet_types.h:1241
> #19 0x00d8e2a4 in impala::HdfsParquetScanner::~HdfsParquetScanner
> (this=0x7f31c92d6700, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/hdfs-parquet-scanner.h:326
> #20 0x00d8e3c9 in impala::HdfsParquetScanner::~HdfsParquetScanner
> (this=0x7f31c92d6700, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/hdfs-parquet-scanner.h:326
> ---Type to continue, or q to quit---
> #21 0x00d4fd48 in checked_delete
> (this=0x43f3c30, filter_ctxs=..., expr_results_pool=0x8, scan_range=0x81f80)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p3/include/boost/core/checked_delete.hpp:34
> #22 ~scoped_ptr (this=0x43f3c30, filter_ctxs=..., expr_results_pool=0x8,
> scan_range=0x81f80)
> at
>
[jira] [Updated] (IMPALA-7122) Data load failure: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try
[
https://issues.apache.org/jira/browse/IMPALA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-7122:
--
Fix Version/s: Not Applicable
> Data load failure: Failed to replace a bad datanode on the existing pipeline
> due to no more good datanodes being available to try
> -
>
> Key: IMPALA-7122
> URL: https://issues.apache.org/jira/browse/IMPALA-7122
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Tim Armstrong
>Assignee: Joe McDonnell
>Priority: Critical
> Labels: flaky
> Fix For: Not Applicable
>
> Attachments: data-load-functional-exhaustive.log, hdfs-logs.tar.gz,
> impalad.ec2-m2-4xlarge-centos-6-4-0570.vpc.cloudera.com.jenkins.log.INFO.20180604-205755.5587,
> load-functional-query.log
>
>
> {noformat}
> 20:58:29 Started Loading functional-query data in background; pid 6813.
> 20:58:29 Loading functional-query data (logging to
> /data/jenkins/workspace/impala-asf-master-core-data-load/repos/Impala/logs/data_loading/load-functional-query.log)...
>
> 20:58:29 Started Loading TPC-H data in background; pid 6814.
> 20:58:29 Loading TPC-H data (logging to
> /data/jenkins/workspace/impala-asf-master-core-data-load/repos/Impala/logs/data_loading/load-tpch.log)...
>
> 20:58:29 Started Loading TPC-DS data in background; pid 6815.
> 20:58:29 Loading TPC-DS data (logging to
> /data/jenkins/workspace/impala-asf-master-core-data-load/repos/Impala/logs/data_loading/load-tpcds.log)...
>
> 21:35:26 FAILED (Took: 36 min 57 sec)
> 21:35:26 'load-data functional-query exhaustive' failed. Tail of log:
> 21:35:26 at
> org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
> 21:35:26 18/06/04 21:20:29 WARN hdfs.DataStreamer: Error Recovery for
> BP-1407206351-127.0.0.1-1528170335185:blk_1073743620_2799 in pipeline
> [DatanodeInfoWithStorage[127.0.0.1:31000,DS-37cfc57c-ab39-443c-80c9-e440cb18b63d,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:31001,DS-2bc41558-4f2c-460f-ae87-5d1a6acbf42f,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:31002,DS-4ba4d3a0-af31-4eaf-b43d-89b408231481,DISK]]:
> datanode
> 0(DatanodeInfoWithStorage[127.0.0.1:31000,DS-37cfc57c-ab39-443c-80c9-e440cb18b63d,DISK])
> is bad.
> 21:35:26 18/06/04 21:21:29 INFO hdfs.DataStreamer: Exception in
> createBlockOutputStream blk_1073743620_2799
> 21:35:26 java.io.IOException: Got error, status=ERROR, status message , ack
> with firstBadLink as 127.0.0.1:31002
> 21:35:26 at
> org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:134)
> 21:35:26 at
> org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:110)
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer.createBlockOutputStream(DataStreamer.java:1778)
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineInternal(DataStreamer.java:1507)
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1481)
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer.processDatanodeOrExternalError(DataStreamer.java:1256)
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:667)
> 21:35:26 18/06/04 21:21:29 WARN hdfs.DataStreamer: Error Recovery for
> BP-1407206351-127.0.0.1-1528170335185:blk_1073743620_2799 in pipeline
> [DatanodeInfoWithStorage[127.0.0.1:31001,DS-2bc41558-4f2c-460f-ae87-5d1a6acbf42f,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:31002,DS-4ba4d3a0-af31-4eaf-b43d-89b408231481,DISK]]:
> datanode
> 1(DatanodeInfoWithStorage[127.0.0.1:31002,DS-4ba4d3a0-af31-4eaf-b43d-89b408231481,DISK])
> is bad.
> 21:35:26 18/06/04 21:21:29 WARN hdfs.DataStreamer: DataStreamer Exception
> 21:35:26 java.io.IOException: Failed to replace a bad datanode on the
> existing pipeline due to no more good datanodes being available to try.
> (Nodes:
> current=[DatanodeInfoWithStorage[127.0.0.1:31001,DS-2bc41558-4f2c-460f-ae87-5d1a6acbf42f,DISK]],
>
> original=[DatanodeInfoWithStorage[127.0.0.1:31001,DS-2bc41558-4f2c-460f-ae87-5d1a6acbf42f,DISK]]).
> The current failed datanode replacement policy is DEFAULT, and a client may
> configure this via
> 'dfs.client.block.write.replace-datanode-on-failure.policy' in its
> configuration.
> 21:35:26 at
> org.apache.hadoop.hdfs.DataStreamer.findNewDatanode(DataStreamer.java:1304)
> 21:35:26 at
>
[jira] [Updated] (IMPALA-5060) Change how Stream::GetBytes returns status
[ https://issues.apache.org/jira/browse/IMPALA-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe McDonnell updated IMPALA-5060: -- Fix Version/s: Not Applicable > Change how Stream::GetBytes returns status > -- > > Key: IMPALA-5060 > URL: https://issues.apache.org/jira/browse/IMPALA-5060 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.9.0 >Reporter: Joe McDonnell >Assignee: Joe McDonnell >Priority: Minor > Fix For: Not Applicable > > > ScannerContext::Stream::GetBytes currently returns a boolean and also takes > in a Status*. It will always fill in the Status* when it returns false, but > it may not fill it in when it returns true. This is error prone, because code > might assume that GetBytes initializes the status. See IMPALA-5055 as an > example bug. > There are two approaches to consider: > 1. Have GetBytes initialize the status to Status::OK() when it returns true. > 2. Change GetBytes to return a Status rather than returning a boolean. > Since GetBytes is used underneath several other functions like ReadBytes, > SkipBytes, and more specialized functions like ReadInt, ReadVLong, ReadText, > etc, every scanner is impacted by these changes. This makes it important to > do performance/functionality testing on any solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6157) THREAD_CREATION_FAILED error should include the hostname
[ https://issues.apache.org/jira/browse/IMPALA-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe McDonnell updated IMPALA-6157: -- Fix Version/s: Not Applicable > THREAD_CREATION_FAILED error should include the hostname > > > Key: IMPALA-6157 > URL: https://issues.apache.org/jira/browse/IMPALA-6157 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.10.0 >Reporter: Joe McDonnell >Assignee: Joe McDonnell >Priority: Minor > Fix For: Not Applicable > > > The THREAD_CREATION_FAILED error currently has the form: > "Failed to create thread $0 in category $1: $2" > It is useful in debugging to know which host hit this error, as this is a > host load issue. The error message should be modified to include this > information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3099) Coordinator::CancelRemoteFragments should use a thread pool
[
https://issues.apache.org/jira/browse/IMPALA-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-3099:
--
Fix Version/s: Not Applicable
> Coordinator::CancelRemoteFragments should use a thread pool
> ---
>
> Key: IMPALA-3099
> URL: https://issues.apache.org/jira/browse/IMPALA-3099
> Project: IMPALA
> Issue Type: Improvement
> Components: Distributed Exec
>Affects Versions: Impala 2.5.0
>Reporter: Henry Robinson
>Assignee: Joe McDonnell
>Priority: Minor
> Labels: ramp-up
> Fix For: Not Applicable
>
>
> {{Coordinator::CancelRemoteFragments()}} makes one RPC for every fragment
> instance, serially. There are no dependencies between fragment instances, so
> we should use {{ExecEnv::rpc_pool()}} to issue the cancellation requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-5185) Parquet scanner should skip pages based on parquet page statistics
[ https://issues.apache.org/jira/browse/IMPALA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker reopened IMPALA-5185: - > Parquet scanner should skip pages based on parquet page statistics > -- > > Key: IMPALA-5185 > URL: https://issues.apache.org/jira/browse/IMPALA-5185 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.9.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: parquet, performance > > IMPALA-2328 added support to skip row groups based on row group statistics. > We also need to support skipping pages based on page statistics. As a > prerequisite the parquet scanners need to be adapted to support synchronous > row skipping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-5185) Parquet scanner should skip pages based on parquet page statistics
[ https://issues.apache.org/jira/browse/IMPALA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker closed IMPALA-5185. --- Resolution: Won't Fix > Parquet scanner should skip pages based on parquet page statistics > -- > > Key: IMPALA-5185 > URL: https://issues.apache.org/jira/browse/IMPALA-5185 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.9.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: parquet, performance > Fix For: Not Applicable > > > IMPALA-2328 added support to skip row groups based on row group statistics. > We also need to support skipping pages based on page statistics. As a > prerequisite the parquet scanners need to be adapted to support synchronous > row skipping. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (IMPALA-5185) Parquet scanner should skip pages based on parquet page statistics
[ https://issues.apache.org/jira/browse/IMPALA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker closed IMPALA-5185. --- Resolution: Won't Fix > Parquet scanner should skip pages based on parquet page statistics > -- > > Key: IMPALA-5185 > URL: https://issues.apache.org/jira/browse/IMPALA-5185 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.9.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: parquet, performance > Fix For: Not Applicable > > > IMPALA-2328 added support to skip row groups based on row group statistics. > We also need to support skipping pages based on page statistics. As a > prerequisite the parquet scanners need to be adapted to support synchronous > row skipping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-5185) Parquet scanner should skip pages based on parquet page statistics
[ https://issues.apache.org/jira/browse/IMPALA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker updated IMPALA-5185: Fix Version/s: Not Applicable > Parquet scanner should skip pages based on parquet page statistics > -- > > Key: IMPALA-5185 > URL: https://issues.apache.org/jira/browse/IMPALA-5185 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.9.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: parquet, performance > Fix For: Not Applicable > > > IMPALA-2328 added support to skip row groups based on row group statistics. > We also need to support skipping pages based on page statistics. As a > prerequisite the parquet scanners need to be adapted to support synchronous > row skipping. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-3643) AuthorizationTest.java fails on dev machine
[
https://issues.apache.org/jira/browse/IMPALA-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Volker updated IMPALA-3643:
Fix Version/s: Not Applicable
> AuthorizationTest.java fails on dev machine
> ---
>
> Key: IMPALA-3643
> URL: https://issues.apache.org/jira/browse/IMPALA-3643
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 2.6.0
>Reporter: Lars Volker
>Assignee: Lars Volker
>Priority: Minor
> Labels: flaky
> Fix For: Not Applicable
>
> Attachments: com.cloudera.impala.analysis.AuthorizationTest.txt
>
>
> [~dtsirogiannis] - I picked you as you look like the author of the relevant
> parts of {AnalysisContext.java} and might have an idea what’s going on here;
> feel free to find another person or assign back to me if you're swamped.
> {{AuthorizationTest.java#TestSelect():498}} fails on my dev machine when
> running this statement:
> {noformat}
> // Select from non-existent db.
> AuthzError("select 1 from nodb.alltypes",
> "User '%s' does not have privileges to execute 'SELECT' on:
> nodb.alltypes");
> {noformat}
> The error is the following:
> {noformat}
> User 'lv' does not have privileges to execute 'SELECT' on: default.nodb
> expected:
> User 'lv' does not have privileges to execute 'SELECT' on: nodb.alltypes
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> com.cloudera.impala.analysis.AuthorizationTest.AuthzError(AuthorizationTest.java:2080)
> at
> com.cloudera.impala.analysis.AuthorizationTest.AuthzError(AuthorizationTest.java:2063)
> at
> com.cloudera.impala.analysis.AuthorizationTest.AuthzError(AuthorizationTest.java:2057)
> at
> com.cloudera.impala.analysis.AuthorizationTest.TestSelect(AuthorizationTest.java:498)
> {noformat}
> I tried to debug this and ended up in {{AnalysisContext.java:430}}.
> {{tablePrivReqs}} contains two entries, {{default.nodb}} and
> {{nodb.alltypes}}. The iteration starts with the former and thus the error is
> raised with the wrong table name.
> Do we expect both these entries in {{tablePrivReqs}}? If so, could this be
> caused by my local JVM iterating elements of HashMaps in a different order?
> My HEAD is at {{* 7167950 - Remove redundant test in
> test_avro_schema_resolution.py (3 days ago) }}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7080) DCHECK hit in DelimitedTextParser
[
https://issues.apache.org/jira/browse/IMPALA-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Volker updated IMPALA-7080:
Fix Version/s: Not Applicable
> DCHECK hit in DelimitedTextParser
> -
>
> Key: IMPALA-7080
> URL: https://issues.apache.org/jira/browse/IMPALA-7080
> Project: IMPALA
> Issue Type: Bug
>Reporter: Thomas Tauber-Marshall
>Assignee: Lars Volker
>Priority: Blocker
> Labels: broken-build
> Fix For: Not Applicable
>
>
> Seen in a 2.x exhaustive run:
> {noformat}
> #0 0x7eff94d2e8e5 in raise () from /lib64/libc.so.6
> #1 0x7eff94d300c5 in abort () from /lib64/libc.so.6
> #2 0x03ffc3a4 in google::DumpStackTraceAndExit() ()
> #3 0x03ff2e1d in google::LogMessage::Fail() ()
> #4 0x03ff46c2 in google::LogMessage::SendToLog() ()
> #5 0x03ff27f7 in google::LogMessage::Flush() ()
> #6 0x03ff5dbe in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x02afcbde in
> impala::DelimitedTextParser::ParseFieldLocations (this=0x17bc5e80,
> max_tuples=1, remaining_len=-60, byte_buffer_ptr=0x7efed9538ab0,
> row_end_locations=0x7efed9538aa0, field_locations=0x152c4000,
> num_tuples=0x7efed9538aac, num_fields=0x7efed9538aa8,
> next_column_start=0x7efed9538ab8) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/delimited-text-parser.cc:205
> #8 0x01d47d61 in impala::HdfsSequenceScanner::ProcessRange
> (this=0x17dd0ac0, row_batch=0xde4b9a0) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/hdfs-sequence-scanner.cc:352
> #9 0x02aebe3e in impala::BaseSequenceScanner::GetNextInternal
> (this=0x17dd0ac0, row_batch=0xde4b9a0) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/base-sequence-scanner.cc:181
> #10 0x01d1e70a in impala::HdfsScanner::ProcessSplit (this=0x17dd0ac0)
> at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/hdfs-scanner.cc:134
> #11 0x01cf5834 in impala::HdfsScanNode::ProcessSplit (this=0xd6b3800,
> filter_ctxs=..., expr_results_pool=0x7efed9539490, scan_range=0xd8d9500,
> scanner_thread_reservation=0x7efed9539408) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/hdfs-scan-node.cc:453
> #12 0x01cf4bd5 in impala::HdfsScanNode::ScannerThread
> (this=0xd6b3800, first_thread=true, scanner_thread_reservation=32768) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/hdfs-scan-node.cc:360
> #13 0x01cf4048 in impala::HdfsScanNodeoperator()(void)
> const (__closure=0x7efed9539bc8) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/exec/hdfs-scan-node.cc:292
> #14 0x01cf60a4 in
> boost::detail::function::void_function_obj_invoker0,
> void>::invoke(boost::detail::function::function_buffer &)
> (function_obj_ptr=...) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
> #15 0x01929fca in boost::function0::operator()
> (this=0x7efed9539bc0) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
> #16 0x01c48275 in impala::Thread::SuperviseThread (name=...,
> category=..., functor=..., parent_thread_info=0x7efed8b38870,
> thread_started=0x7efed8b376d0) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/be/src/util/thread.cc:356
> #17 0x01c5074b in
> boost::_bi::list5 std::char_traits, std::allocator > >,
> boost::_bi::value,
> std::allocator > >, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> >::operator() std::basic_string&, const std::basic_string&,
> boost::function, const impala::ThreadDebugInfo*, impala::Promise int>*), boost::_bi::list0>(boost::_bi::type, void (*&)(const
> std::basic_string, std::allocator > &,
> const std::basic_string, std::allocator >
> &, boost::function, const impala::ThreadDebugInfo *,
> impala::Promise *), boost::_bi::list0 &, int) (this=0xd37a65c0,
> f=@0xd37a65b8, a=...) at
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
> #18 0x01c5066f in boost::_bi::bind_t std::basic_string, std::allocator >&,
> const std::basic_string, std::allocator
> >&, boost::function, const impala::ThreadDebugInfo*,
> impala::Promise*),
> boost::_bi::list5 std::char_traits, std::allocator > >,
> boost::_bi::value,
> std::allocator > >, boost::_bi::value >,
> boost::_bi::value,
> boost::_bi::value*> > >::operator()(void)
> (this=0xd37a65b8) at
>
[jira] [Updated] (IMPALA-7177) Impalad hangs when starting up
[
https://issues.apache.org/jira/browse/IMPALA-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Volker updated IMPALA-7177:
Fix Version/s: Not Applicable
> Impalad hangs when starting up
> --
>
> Key: IMPALA-7177
> URL: https://issues.apache.org/jira/browse/IMPALA-7177
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Tianyi Wang
>Assignee: Lars Volker
>Priority: Critical
> Labels: broken-build
> Fix For: Not Applicable
>
>
> In
> custom_cluster/test_breakpad.py::TestBreakpadExhaustive::test_minidump_cleanup_thread,
> a impalad hangs for 4 minutes when starting up. The tail of its log is:
> {noformat}
> I0613 14:55:55.514082 23644 init.cc:246] OS version: Linux version
> 2.6.32-358.14.1.el6.centos.plus.x86_64 (mockbu...@c6b9.bsys.dev.centos.org)
> (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Tue Jul 16
> 21:33:24 UTC 2013
> Clock: clocksource: 'xen', clockid_t: CLOCK_MONOTONIC_COARSE
> I0613 14:55:55.514091 23644 init.cc:247] Process ID: 23644
> I0613 14:55:55.514098 23644 init.cc:248] Default AES cipher mode for
> spill-to-disk: AES-CTR
> I0613 14:55:58.153287 23644 llvm-codegen.cc:152] CPU class for runtime code
> generation: sandybridge
> I0613 14:55:58.153379 23644 llvm-codegen.cc:154] Detected CPU flags:
> +sse2,+cx16,-tbm,-avx512ifma,-avx512dq,-fma4,-prfchw,-bmi2,-xsavec,-fsgsbase,+popcnt,-aes,-xsaves,-avx512er,-avx512vpopcntdq,-clwb,-avx512f,-clzero,-pku,+mmx,-lwp$
> I0613 14:55:58.153432 23644 llvm-codegen.cc:157] CPU flags enabled for
> runtime code generation:
> +sse2,+cx16,-tbm,-avx512ifma,-avx512dq,-fma4,-prfchw,-bmi2,-xsavec,-fsgsbase,+popcnt,-aes,-xsaves,-avx512er,-avx512vpopcntdq,-clwb,-avx$
> I0613 14:56:00.633289 23644 GlogAppender.java:137] Logging (re)initialized.
> Impala: VLOG, All other: INFO
> I0613 14:56:00.638018 23644 JniFrontend.java:139] Authorization is 'DISABLED'.
> I0613 14:56:00.640646 23644 JniFrontend.java:141] Java Input arguments:
> -agentlib:jdwp=transport=dt_socket,address=30002,server=y,suspend=n
> -Djava.library.path=/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/Impala-Toolchain/cdh_components/hadoop-3.0.0-cdh6.x-417234//lib/native/
> -XX:ErrorFile=$
> Java System properties:
> awt.toolkit:sun.awt.X11.XToolkit
> file.encoding.pkg:sun.io
> java.specification.version:1.8
> sun.cpu.isalist:
> sun.jnu.encoding:UTF-8
> java.class.path:/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/fe/src/test/resources:/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/fe/target/classes:/data/jenkins/workspace/impala-cdh6$
> I0613 14:56:00.691942 23644 HiveConf.java:188] Found configuration file
> file:/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/fe/src/test/resources/hive-site.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8323) Bump impyla to 0.15 once it gets released
[ https://issues.apache.org/jira/browse/IMPALA-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker updated IMPALA-8323: Affects Version/s: Impala 3.3.0 > Bump impyla to 0.15 once it gets released > - > > Key: IMPALA-8323 > URL: https://issues.apache.org/jira/browse/IMPALA-8323 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Fix For: Impala 3.3.0 > > > We should bump to 0.15 once this upstream issue gets resolved: > https://github.com/cloudera/impyla/issues/247 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8323) Bump impyla to 0.15 once it gets released
[ https://issues.apache.org/jira/browse/IMPALA-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker updated IMPALA-8323: Fix Version/s: Impala 3.3.0 > Bump impyla to 0.15 once it gets released > - > > Key: IMPALA-8323 > URL: https://issues.apache.org/jira/browse/IMPALA-8323 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Fix For: Impala 3.3.0 > > > We should bump to 0.15 once this upstream issue gets resolved: > https://github.com/cloudera/impyla/issues/247 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
radford nguyen created IMPALA-8473: -- Summary: Refactor lineage publication mechanism to allow for different consumers Key: IMPALA-8473 URL: https://issues.apache.org/jira/browse/IMPALA-8473 Project: IMPALA Issue Type: Improvement Components: Backend, Frontend Reporter: radford nguyen Impetus for this change is to allow lineage to be consumed by Altus via Kafka. Approach should be similar to that of choosing authorization provider, where the publication strategy can be chosen at runtime via configuration flag(s). Scope of this ticket is to move lineage publication to the fe and add appropriate hooks that a user can implement -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8473) Refactor lineage publication mechanism to allow for different consumers
radford nguyen created IMPALA-8473: -- Summary: Refactor lineage publication mechanism to allow for different consumers Key: IMPALA-8473 URL: https://issues.apache.org/jira/browse/IMPALA-8473 Project: IMPALA Issue Type: Improvement Components: Backend, Frontend Reporter: radford nguyen Impetus for this change is to allow lineage to be consumed by Altus via Kafka. Approach should be similar to that of choosing authorization provider, where the publication strategy can be chosen at runtime via configuration flag(s). Scope of this ticket is to move lineage publication to the fe and add appropriate hooks that a user can implement -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang updated IMPALA-8149: --- Affects Version/s: Impala 3.3.0 > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Affects Versions: Impala 3.3.0 >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang updated IMPALA-8149: --- Fix Version/s: Impala 3.3.0 > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Affects Versions: Impala 3.3.0 >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > Fix For: Impala 3.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830565#comment-16830565 ] Xiaomeng Zhang commented on IMPALA-8149: Hi Tim, what version is master branch? How can I find version number in git branch? > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830544#comment-16830544 ] Tim Armstrong commented on IMPALA-8149: --- [~Xiaomeng Zhang] please set a fix version. > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang resolved IMPALA-8149. Resolution: Fixed > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-2990) Coordinator should timeout and cancel queries with unresponsive / stuck executors
[ https://issues.apache.org/jira/browse/IMPALA-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Tauber-Marshall resolved IMPALA-2990. Resolution: Fixed Fix Version/s: Impala 3.3.0 > Coordinator should timeout and cancel queries with unresponsive / stuck > executors > - > > Key: IMPALA-2990 > URL: https://issues.apache.org/jira/browse/IMPALA-2990 > Project: IMPALA > Issue Type: Bug > Components: Distributed Exec >Affects Versions: Impala 2.3.0 >Reporter: Sailesh Mukil >Assignee: Thomas Tauber-Marshall >Priority: Critical > Labels: hang, observability, supportability > Fix For: Impala 3.3.0 > > > The coordinator currently waits indefinitely if it does not hear back from a > backend. This could cause a query to hang indefinitely in case of a network > error, etc. > We should add logic for determining when a backend is unresponsive and kill > the query. The logic should mostly revolve around Coordinator::Wait() and > Coordinator::UpdateFragmentExecStatus() based on whether it receives periodic > updates from a backed (via FragmentExecState::ReportStatusCb()). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8228) Support for object ownership with Ranger authorization provider
[ https://issues.apache.org/jira/browse/IMPALA-8228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8228: - Issue Type: New Feature (was: Sub-task) Parent: (was: IMPALA-7916) > Support for object ownership with Ranger authorization provider > --- > > Key: IMPALA-8228 > URL: https://issues.apache.org/jira/browse/IMPALA-8228 > Project: IMPALA > Issue Type: New Feature > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Priority: Critical > > This ticket should investigate whether it's feasible to implement object > ownership with Ranger. If it's not feasible, we should update the code to act > accordingly when Impala is enabled with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8228) Support for object ownership with Ranger authorization provider
[ https://issues.apache.org/jira/browse/IMPALA-8228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-8228: - Priority: Major (was: Critical) > Support for object ownership with Ranger authorization provider > --- > > Key: IMPALA-8228 > URL: https://issues.apache.org/jira/browse/IMPALA-8228 > Project: IMPALA > Issue Type: New Feature > Components: Catalog, Frontend >Reporter: Fredy Wijaya >Priority: Major > > This ticket should investigate whether it's feasible to implement object > ownership with Ranger. If it's not feasible, we should update the code to act > accordingly when Impala is enabled with Ranger. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8472) Update Ranger minicluster with the one that supports "refresh" access type
[ https://issues.apache.org/jira/browse/IMPALA-8472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8472 started by Fredy Wijaya. > Update Ranger minicluster with the one that supports "refresh" access type > -- > > Key: IMPALA-8472 > URL: https://issues.apache.org/jira/browse/IMPALA-8472 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > > Update the Ranger minicluster with the one that contains this fix: > https://issues.apache.org/jira/browse/RANGER-2374 and also update the > "refresh" access type workaround. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8472) Update Ranger minicluster with the one that supports "refresh" access type
Fredy Wijaya created IMPALA-8472: Summary: Update Ranger minicluster with the one that supports "refresh" access type Key: IMPALA-8472 URL: https://issues.apache.org/jira/browse/IMPALA-8472 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Update the Ranger minicluster with the one that contains this fix: https://issues.apache.org/jira/browse/RANGER-2374 and also update the "refresh" access type workaround. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8472) Update Ranger minicluster with the one that supports "refresh" access type
Fredy Wijaya created IMPALA-8472: Summary: Update Ranger minicluster with the one that supports "refresh" access type Key: IMPALA-8472 URL: https://issues.apache.org/jira/browse/IMPALA-8472 Project: IMPALA Issue Type: Sub-task Components: Frontend Reporter: Fredy Wijaya Assignee: Fredy Wijaya Update the Ranger minicluster with the one that contains this fix: https://issues.apache.org/jira/browse/RANGER-2374 and also update the "refresh" access type workaround. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8466) TestHdfsCachingDdl.test_caching_ddl fails under docker
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8466.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> TestHdfsCachingDdl.test_caching_ddl fails under docker
> --
>
> Key: IMPALA-8466
> URL: https://issues.apache.org/jira/browse/IMPALA-8466
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Infrastructure
>Reporter: Fang-Yu Rao
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build, docker
> Fix For: Impala 3.3.0
>
>
> After sending my patch set to Jenkins for dry-run tests
> (https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console), unrelated
> failures occurred. Specifically, some tests under
> ubuntu-16.04-dockerised-tests failed
> (https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console).
> Here's the output of the one test that failed.
> {noformat}
> query_test.test_hdfs_caching.TestHdfsCachingDdl.test_caching_ddl[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] (from pytest)
> Failing for the past 4 builds (Since Failed#87 )
> Took 39 sec.
> add description
> Error Message
> ImpalaBeeswaxException: ImpalaBeeswaxException: INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: TableLoadingException: Loading
> file and block metadata for 1 paths for table cachedb.cached_tbl_part: failed
> to load 1 paths. Check the catalog server log for more details.
> Stacktrace
> query_test/test_hdfs_caching.py:207: in test_caching_ddl
> self.run_test_case('QueryTest/hdfs-caching', vector)
> common/impala_test_suite.py:487: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:721: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:180: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:183: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:358: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:352: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:512: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION:
> EMESSAGE: TableLoadingException: Loading file and block metadata for 1
> paths for table cachedb.cached_tbl_part: failed to load 1 paths. Check the
> catalog server log for more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8466) TestHdfsCachingDdl.test_caching_ddl fails under docker
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8466.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> TestHdfsCachingDdl.test_caching_ddl fails under docker
> --
>
> Key: IMPALA-8466
> URL: https://issues.apache.org/jira/browse/IMPALA-8466
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Infrastructure
>Reporter: Fang-Yu Rao
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build, docker
> Fix For: Impala 3.3.0
>
>
> After sending my patch set to Jenkins for dry-run tests
> (https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console), unrelated
> failures occurred. Specifically, some tests under
> ubuntu-16.04-dockerised-tests failed
> (https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console).
> Here's the output of the one test that failed.
> {noformat}
> query_test.test_hdfs_caching.TestHdfsCachingDdl.test_caching_ddl[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] (from pytest)
> Failing for the past 4 builds (Since Failed#87 )
> Took 39 sec.
> add description
> Error Message
> ImpalaBeeswaxException: ImpalaBeeswaxException: INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: TableLoadingException: Loading
> file and block metadata for 1 paths for table cachedb.cached_tbl_part: failed
> to load 1 paths. Check the catalog server log for more details.
> Stacktrace
> query_test/test_hdfs_caching.py:207: in test_caching_ddl
> self.run_test_case('QueryTest/hdfs-caching', vector)
> common/impala_test_suite.py:487: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:721: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:180: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:183: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:358: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:352: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:512: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION:
> EMESSAGE: TableLoadingException: Loading file and block metadata for 1
> paths for table cachedb.cached_tbl_part: failed to load 1 paths. Check the
> catalog server log for more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8463) skip.header.line.count ignored on local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8463.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> skip.header.line.count ignored on local catalog
> ---
>
> Key: IMPALA-8463
> URL: https://issues.apache.org/jira/browse/IMPALA-8463
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> test_text_scanner_with_header() fails when run against local catalog.
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] functional> select c1, c2 from
> table_with_header
> > ;
> Query: select c1, c2 from table_with_header
> Query submitted at: 2019-04-25 22:47:52 (Coordinator:
> http://2c766e5e2187:25000)
> Query progress can be monitored at:
> http://2c766e5e2187:25000/query_plan?query_id=68403d93ac914925:394ee6a1
> +--+--+
> | c1 | c2 |
> +--+--+
> | 5| 6|
> | 1| 2|
> | 3| 4|
> | NULL | NULL |
> +--+--+
> WARNINGS: Error converting column: 0 to INT
> Error converting column: 1 to DOUBLE
> Error parsing row: file:
> hdfs://172.19.0.1:20500/test-warehouse/table_with_header/table_with_header.csv,
> before offset: 22
> {noformat}
> This is suspicious in LocalFsTable.java
> {code}
> @Override
> public int parseSkipHeaderLineCount(StringBuilder error) {
> // TODO Auto-generated method stub
> return 0;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8463) skip.header.line.count ignored on local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8463.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> skip.header.line.count ignored on local catalog
> ---
>
> Key: IMPALA-8463
> URL: https://issues.apache.org/jira/browse/IMPALA-8463
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
> Fix For: Impala 3.3.0
>
>
> test_text_scanner_with_header() fails when run against local catalog.
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] functional> select c1, c2 from
> table_with_header
> > ;
> Query: select c1, c2 from table_with_header
> Query submitted at: 2019-04-25 22:47:52 (Coordinator:
> http://2c766e5e2187:25000)
> Query progress can be monitored at:
> http://2c766e5e2187:25000/query_plan?query_id=68403d93ac914925:394ee6a1
> +--+--+
> | c1 | c2 |
> +--+--+
> | 5| 6|
> | 1| 2|
> | 3| 4|
> | NULL | NULL |
> +--+--+
> WARNINGS: Error converting column: 0 to INT
> Error converting column: 1 to DOUBLE
> Error parsing row: file:
> hdfs://172.19.0.1:20500/test-warehouse/table_with_header/table_with_header.csv,
> before offset: 22
> {noformat}
> This is suspicious in LocalFsTable.java
> {code}
> @Override
> public int parseSkipHeaderLineCount(StringBuilder error) {
> // TODO Auto-generated method stub
> return 0;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8465) hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
[
https://issues.apache.org/jira/browse/IMPALA-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8465.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
>
>
> Key: IMPALA-8465
> URL: https://issues.apache.org/jira/browse/IMPALA-8465
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> Changes to tests.common.ImpalaCluster in [commit
> 2ca7f8e7|https://github.com/apache/impala/commit/2ca7f8e7c0781a1914275b3506cf8a7748c44c85#diff-6fea89ad0e6c440b0373bb136d7510b5]
> introduced a regression in this test.
> {noformat}
> hs2/test_json_endpoints.py:51: in test_waiting_in_flight_queries
> queries_json = self._get_json_queries(http_addr)
> hs2/test_json_endpoints.py:33: in _get_json_queries
> return cluster.impalads[0].service.get_debug_webpage_json("/queries")
> E IndexError: list index out of range
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8465) hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
[
https://issues.apache.org/jira/browse/IMPALA-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8465.
---
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
>
>
> Key: IMPALA-8465
> URL: https://issues.apache.org/jira/browse/IMPALA-8465
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> Changes to tests.common.ImpalaCluster in [commit
> 2ca7f8e7|https://github.com/apache/impala/commit/2ca7f8e7c0781a1914275b3506cf8a7748c44c85#diff-6fea89ad0e6c440b0373bb136d7510b5]
> introduced a regression in this test.
> {noformat}
> hs2/test_json_endpoints.py:51: in test_waiting_in_flight_queries
> queries_json = self._get_json_queries(http_addr)
> hs2/test_json_endpoints.py:33: in _get_json_queries
> return cluster.impalads[0].service.get_debug_webpage_json("/queries")
> E IndexError: list index out of range
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7971) Add support to detect insert events from Impala
[
https://issues.apache.org/jira/browse/IMPALA-7971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830405#comment-16830405
]
ASF subversion and git services commented on IMPALA-7971:
-
Commit 095d609260ba8957c13e61c539892ab37c5da10b in impala's branch
refs/heads/master from Todd Lipcon
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=095d609 ]
IMPALA-7971 (follow-up). Fix compilation error
15a33d1ba was committed at about the same time as 5ced9160bd6, which
renamed one of the methods of the FileDescriptor class, causing a
compilation error. This follow-up mixes the semantic conflict.
Tested tests/custom_cluster/test_event_processing.py manually.
Change-Id: I8071b421f0dcbae2d303bd9f1e1f6f64657c49c0
Reviewed-on: http://gerrit.cloudera.org:8080/13184
Reviewed-by: Todd Lipcon
Tested-by: Todd Lipcon
> Add support to detect insert events from Impala
> ---
>
> Key: IMPALA-7971
> URL: https://issues.apache.org/jira/browse/IMPALA-7971
> Project: IMPALA
> Issue Type: Sub-task
>Reporter: Vihang Karajgaonkar
>Assignee: Anurag Mantripragada
>Priority: Major
>
> When data is inserted into existing tables and partitions, Catalog does not
> issue any metastore API calls. Metastore provides a API called
> {{fire_listener_event}} which can be used to add a {{INSERT_EVENT}} to the
> metastore notification log. This event can be used by other Impala instances
> to invalidate or update the filemetada information when data is inserted or
> overrwriten on a given table or partition.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8465) hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
[
https://issues.apache.org/jira/browse/IMPALA-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830407#comment-16830407
]
ASF subversion and git services commented on IMPALA-8465:
-
Commit c0dc9fddfe5eb7b4f3076f0589dcf02d795be566 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c0dc9fd ]
IMPALA-8465: fix TestJsonEndpoints for remote clusters
ImpalaCluster does not work for remote clusters, and
the patch introduced a dependency on it for a test that previously
didn't depend on it.
This reverts to the old logic for non-dockerized clusters.
Change-Id: I8cf6ecfb72ec18be0b5274e360825fb3f203dd20
Reviewed-on: http://gerrit.cloudera.org:8080/13136
Reviewed-by: Tim Armstrong
Tested-by: Impala Public Jenkins
> hs2.test_json_endpoints.TestJsonEndpoints fails on deployed clusters
>
>
> Key: IMPALA-8465
> URL: https://issues.apache.org/jira/browse/IMPALA-8465
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Major
>
> Changes to tests.common.ImpalaCluster in [commit
> 2ca7f8e7|https://github.com/apache/impala/commit/2ca7f8e7c0781a1914275b3506cf8a7748c44c85#diff-6fea89ad0e6c440b0373bb136d7510b5]
> introduced a regression in this test.
> {noformat}
> hs2/test_json_endpoints.py:51: in test_waiting_in_flight_queries
> queries_json = self._get_json_queries(http_addr)
> hs2/test_json_endpoints.py:33: in _get_json_queries
> return cluster.impalads[0].service.get_debug_webpage_json("/queries")
> E IndexError: list index out of range
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8466) TestHdfsCachingDdl.test_caching_ddl fails under docker
[
https://issues.apache.org/jira/browse/IMPALA-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830410#comment-16830410
]
ASF subversion and git services commented on IMPALA-8466:
-
Commit b273afd94067dfd062f9c1c039a082e4e6cbeb0c in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b273afd ]
IMPALA-8466: disable test_caching_ddl in dockerized cluster
The test creates partitions with file:// URLs pointing at the
host filesystem, which isn't accessible from within the
containers. The only reason the test passed earlier was because
of a bug fixed by the IMPALA-8454 patches which suppressed
the error.
Change-Id: If4e67cf7f5d993c15cb5e1114beb099d0dfdc325
Reviewed-on: http://gerrit.cloudera.org:8080/13181
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> TestHdfsCachingDdl.test_caching_ddl fails under docker
> --
>
> Key: IMPALA-8466
> URL: https://issues.apache.org/jira/browse/IMPALA-8466
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Infrastructure
>Reporter: Fang-Yu Rao
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build, docker
>
> After sending my patch set to Jenkins for dry-run tests
> (https://jenkins.impala.io/job/gerrit-verify-dryrun/4085/console), unrelated
> failures occurred. Specifically, some tests under
> ubuntu-16.04-dockerised-tests failed
> (https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/90/console).
> Here's the output of the one test that failed.
> {noformat}
> query_test.test_hdfs_caching.TestHdfsCachingDdl.test_caching_ddl[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] (from pytest)
> Failing for the past 4 builds (Since Failed#87 )
> Took 39 sec.
> add description
> Error Message
> ImpalaBeeswaxException: ImpalaBeeswaxException: INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: TableLoadingException: Loading
> file and block metadata for 1 paths for table cachedb.cached_tbl_part: failed
> to load 1 paths. Check the catalog server log for more details.
> Stacktrace
> query_test/test_hdfs_caching.py:207: in test_caching_ddl
> self.run_test_case('QueryTest/hdfs-caching', vector)
> common/impala_test_suite.py:487: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:721: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:180: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:183: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:358: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:352: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:512: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION:
> EMESSAGE: TableLoadingException: Loading file and block metadata for 1
> paths for table cachedb.cached_tbl_part: failed to load 1 paths. Check the
> catalog server log for more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7290) Move impala-shell to use HS2
[ https://issues.apache.org/jira/browse/IMPALA-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830409#comment-16830409 ] ASF subversion and git services commented on IMPALA-7290: - Commit 0a9ea803d2ed05135d1538dadf4b09ab6ad58e72 in impala's branch refs/heads/master from Tim Armstrong [ https://gitbox.apache.org/repos/asf?p=impala.git;h=0a9ea80 ] IMPALA-7290: part 1: clean up shell tests This sets up the tests to be extensible to test shell in both beeswax and HS2 modes. Testing: * Add test dimension containing only beeswax in preparation for HS2 dimension. * Factor out hardcoded ports. * Add tests for formatting of all types and NULL values. * Merge date shell test into general type tests. * Added testing for floating point output formatting, which does change as a result of switching to server-side vs client-side formatting. * Use unique_database for tests that create tables. Change-Id: Ibe5ab7f4817e690b7d3be08d71f8f14364b84412 Reviewed-on: http://gerrit.cloudera.org:8080/13083 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Move impala-shell to use HS2 > > > Key: IMPALA-7290 > URL: https://issues.apache.org/jira/browse/IMPALA-7290 > Project: IMPALA > Issue Type: Improvement > Components: Clients >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: maintainability, shell > > Most clients have moved to the HS2 interface. impala-shell is one of the > laggards. We should switch impala-shell to use the newer and more standard > interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8454) Recursively list files within transactional tables
[ https://issues.apache.org/jira/browse/IMPALA-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830411#comment-16830411 ] ASF subversion and git services commented on IMPALA-8454: - Commit b273afd94067dfd062f9c1c039a082e4e6cbeb0c in impala's branch refs/heads/master from Tim Armstrong [ https://gitbox.apache.org/repos/asf?p=impala.git;h=b273afd ] IMPALA-8466: disable test_caching_ddl in dockerized cluster The test creates partitions with file:// URLs pointing at the host filesystem, which isn't accessible from within the containers. The only reason the test passed earlier was because of a bug fixed by the IMPALA-8454 patches which suppressed the error. Change-Id: If4e67cf7f5d993c15cb5e1114beb099d0dfdc325 Reviewed-on: http://gerrit.cloudera.org:8080/13181 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Recursively list files within transactional tables > -- > > Key: IMPALA-8454 > URL: https://issues.apache.org/jira/browse/IMPALA-8454 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Labels: impala-acid > Fix For: Impala 3.3.0 > > > For transactional tables, the data files are not directly within the > partition directories, but instead are stored within subdirectories > corresponding to writeIds, compactions, etc. To support this, we need to be > able to recursively load file lists within partition directories. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8463) skip.header.line.count ignored on local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830408#comment-16830408
]
ASF subversion and git services commented on IMPALA-8463:
-
Commit c6e3ea4846cc56bc2d941a07f2f29821d4b788d5 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c6e3ea4 ]
IMPALA-8463: fix skip.header.line.count on local catalog
This moves the logic to FeFsTable and calls it from
LocalFsTable.
Testing:
Added a unit test that repros the problem. End-to-end tests run against
local catalog will catch this but aren't enabled.
Change-Id: I3957b3661574c356a14197e8829abbed92028a1a
Reviewed-on: http://gerrit.cloudera.org:8080/13131
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> skip.header.line.count ignored on local catalog
> ---
>
> Key: IMPALA-8463
> URL: https://issues.apache.org/jira/browse/IMPALA-8463
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
>
> test_text_scanner_with_header() fails when run against local catalog.
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] functional> select c1, c2 from
> table_with_header
> > ;
> Query: select c1, c2 from table_with_header
> Query submitted at: 2019-04-25 22:47:52 (Coordinator:
> http://2c766e5e2187:25000)
> Query progress can be monitored at:
> http://2c766e5e2187:25000/query_plan?query_id=68403d93ac914925:394ee6a1
> +--+--+
> | c1 | c2 |
> +--+--+
> | 5| 6|
> | 1| 2|
> | 3| 4|
> | NULL | NULL |
> +--+--+
> WARNINGS: Error converting column: 0 to INT
> Error converting column: 1 to DOUBLE
> Error parsing row: file:
> hdfs://172.19.0.1:20500/test-warehouse/table_with_header/table_with_header.csv,
> before offset: 22
> {noformat}
> This is suspicious in LocalFsTable.java
> {code}
> @Override
> public int parseSkipHeaderLineCount(StringBuilder error) {
> // TODO Auto-generated method stub
> return 0;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-2990) Coordinator should timeout and cancel queries with unresponsive / stuck executors
[ https://issues.apache.org/jira/browse/IMPALA-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830404#comment-16830404 ] ASF subversion and git services commented on IMPALA-2990: - Commit a73ef68745da6542657892d256e5b1bd29c68803 in impala's branch refs/heads/master from Thomas Tauber-Marshall [ https://gitbox.apache.org/repos/asf?p=impala.git;h=a73ef68 ] IMPALA-2990: timeout unresponsive queries in coordinator The coordinator currently waits indefinitely if it does not receive a status report from a backend. This could cause a query to hang indefinitely in certain situations, for example if the backend decides to cancel itself as a result of failed status report rpcs. This patch adds a thread to ImpalaServer which periodically iterates over all queries for which that server is the coordinator and cancels any that haven't had a report from a backend in a certain amount of time. This patch adds two flags: --status_report_max_retry_s: the maximum number of seconds a backend will attempt to send status reports before giving up. This is used in place of --status_report_max_retries which is now deprecated. --status_report_cancellation_padding: the coordinator will wait --status_report_max_retry_s * (1 + --status_report_cancellation_padding / 100) before concluding a backend is not responding and cancelling the query. Testing: - Added a functional test that runs a query that is cancelled through the new mechanism. - Passed a full set of exhaustive tests. Ran tests on a 10 node cluster loaded with tpch 500: - Ran the stress test for 1000 queries with the debug actions: 'REPORT_EXEC_STATUS_DELAY:JITTER@1000' Prior to this patch, this setup results in hanging queries. With this patch, no hangs were observed. - Ran perf tests with 4 concurrent streams, 3 iterations per query. Found no change in performance. Change-Id: I196c8c6a5633b1960e2c3a3884777be9b3824987 Reviewed-on: http://gerrit.cloudera.org:8080/12299 Reviewed-by: Thomas Marshall Tested-by: Impala Public Jenkins > Coordinator should timeout and cancel queries with unresponsive / stuck > executors > - > > Key: IMPALA-2990 > URL: https://issues.apache.org/jira/browse/IMPALA-2990 > Project: IMPALA > Issue Type: Bug > Components: Distributed Exec >Affects Versions: Impala 2.3.0 >Reporter: Sailesh Mukil >Assignee: Thomas Tauber-Marshall >Priority: Critical > Labels: hang, observability, supportability > > The coordinator currently waits indefinitely if it does not hear back from a > backend. This could cause a query to hang indefinitely in case of a network > error, etc. > We should add logic for determining when a backend is unresponsive and kill > the query. The logic should mostly revolve around Coordinator::Wait() and > Coordinator::UpdateFragmentExecStatus() based on whether it receives periodic > updates from a backed (via FragmentExecState::ReportStatusCb()). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reopened IMPALA-7368: --- > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong closed IMPALA-7368. - > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong closed IMPALA-7368. - > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-7368. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (IMPALA-7492) Add support for DATE text parser/formatter
[ https://issues.apache.org/jira/browse/IMPALA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong closed IMPALA-7492. - > Add support for DATE text parser/formatter > -- > > Key: IMPALA-7492 > URL: https://issues.apache.org/jira/browse/IMPALA-7492 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-7368. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7492) Add support for DATE text parser/formatter
[ https://issues.apache.org/jira/browse/IMPALA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong closed IMPALA-7492. - > Add support for DATE text parser/formatter > -- > > Key: IMPALA-7492 > URL: https://issues.apache.org/jira/browse/IMPALA-7492 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-7492) Add support for DATE text parser/formatter
[ https://issues.apache.org/jira/browse/IMPALA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reopened IMPALA-7492: --- > Add support for DATE text parser/formatter > -- > > Key: IMPALA-7492 > URL: https://issues.apache.org/jira/browse/IMPALA-7492 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7492) Add support for DATE text parser/formatter
[ https://issues.apache.org/jira/browse/IMPALA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-7492. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add support for DATE text parser/formatter > -- > > Key: IMPALA-7492 > URL: https://issues.apache.org/jira/browse/IMPALA-7492 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7492) Add support for DATE text parser/formatter
[ https://issues.apache.org/jira/browse/IMPALA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-7492. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add support for DATE text parser/formatter > -- > > Key: IMPALA-7492 > URL: https://issues.apache.org/jira/browse/IMPALA-7492 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 3.3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7734) Catalog memz page shows useless memory breakdown
[
https://issues.apache.org/jira/browse/IMPALA-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830386#comment-16830386
]
Naveen Gangam commented on IMPALA-7734:
---
[~ychena] Do you have to take a look at this? Thanks
> Catalog memz page shows useless memory breakdown
>
>
> Key: IMPALA-7734
> URL: https://issues.apache.org/jira/browse/IMPALA-7734
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: newbie, observability, ramp-up, supportability
>
> If you look at catalogd memz, e.g. at localhost:25020/memz, it has a bogus
> breakdown. The catalogd does not use the MemTracker infrastructure since the
> vast majority of it's memory consumption is in the JVM.
> {noformat}
> Breakdown
> : Total=0 Peak=0
> Untracked Memory: Total=0
> {noformat}
> Reported by [~alanj_impala_5a78]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7734) Catalog memz page shows useless memory breakdown
[
https://issues.apache.org/jira/browse/IMPALA-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam reassigned IMPALA-7734:
-
Assignee: (was: Naveen Gangam)
> Catalog memz page shows useless memory breakdown
>
>
> Key: IMPALA-7734
> URL: https://issues.apache.org/jira/browse/IMPALA-7734
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: newbie, observability, ramp-up, supportability
>
> If you look at catalogd memz, e.g. at localhost:25020/memz, it has a bogus
> breakdown. The catalogd does not use the MemTracker infrastructure since the
> vast majority of it's memory consumption is in the JVM.
> {noformat}
> Breakdown
> : Total=0 Peak=0
> Untracked Memory: Total=0
> {noformat}
> Reported by [~alanj_impala_5a78]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-4018) Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ... FORMAT )
[
https://issues.apache.org/jira/browse/IMPALA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830360#comment-16830360
]
Matthias commented on IMPALA-4018:
--
I have a strong opinion about APIs that do not surprise (see [Principle of
least astonishment|http://wiki.c2.com/?PrincipleOfLeastAstonishment]), i.e.:
{quote}The Principle of Least Astonishment states that the result of performing
some operation should be obvious, consistent, and predictable, based upon the
name of the operation and other clues.
{quote}
The defined "SQL" language is the API for the user and the same logic applies
to it. I do understand, and support the desire to align more with the SQL:2016
standard, but this is not where the code started from (see this comment which
is also aligned with what Hive is doing):
{quote}The original intention was to eventually mimic what the SimpleDateFormat
offers
([https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html])
{quote}
I guess that's the reason why we are talking about feature flags in the first
place.
IMHO, putting one format pattern behind a feature flag but not the other would
be very much against the principle of least astonishment. Therefore, we either
change all of them with or without a feature flag. The latter is probably a bad
idea considering the user base and backwards compatibility. Thus, breaking is
not an option and the therefore the only option I see is a feature flag for all.
[~grahn]: What speaks against having ONE feature flag for all format patterns.
The user can select (per session, per server, per ... ) whether he
wants to use a Java-style format pattern or an SQL:2016 format pattern. That
format pattern would then be used for ALL functions, be it `CAST()`,
`TO_TIMESTAMP()`, ... . This would keep the API consistent, but leave it up to
the user to choose. From what I tell this would still fulfill your requirement,
but please correct me if I'm wrong.
Having pluggable implementations (selected by feature flag) would allow other
formatters. What about the one from
[ICU|http://userguide.icu-project.org/formatparse/datetime]?
Some small side notes:
* There are already some inconsistencies, like `TRUNC(TIMESTAMP timestamp,
STRING unit)` would accept `MI` instead of `mm`, but I guess that's because
it's a unit and not a format pattern.
> Add support for SQL:2016 datetime templates/patterns/masks to CAST(... AS ...
> FORMAT )
>
>
> Key: IMPALA-4018
> URL: https://issues.apache.org/jira/browse/IMPALA-4018
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Affects Versions: Impala 2.2.4
>Reporter: Greg Rahn
>Assignee: Gabor Kaszab
>Priority: Critical
> Labels: ansi-sql, compatibility, sql-language
>
> *Summary*
> The format masks/templates for currently are implemented using the [Java
> SimpleDateFormat
> patterns|http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html],
> and although this is what Hive has implemented, it is not what most standard
> SQL systems implement. For example see
> [Vertica|https://my.vertica.com/docs/7.2.x/HTML/Content/Authoring/SQLReferenceManual/Functions/Formatting/TemplatePatternsForDateTimeFormatting.htm],
>
> [Netezza|http://www.ibm.com/support/knowledgecenter/SSULQD_7.2.1/com.ibm.nz.dbu.doc/r_dbuser_ntz_sql_extns_templ_patterns_date_time_conv.html],
>
> [Oracle|https://docs.oracle.com/database/121/SQLRF/sql_elements004.htm#SQLRF00212],
> and
> [PostgreSQL|https://www.postgresql.org/docs/9.5/static/functions-formatting.html#FUNCTIONS-FORMATTING-DATETIME-TABLE].
>
> *Examples of incompatibilities*
> {noformat}
> -- PostgreSQL/Netezza/Vertica/Oracle
> select to_timestamp('May 15, 2015 12:00:00', 'mon dd, hh:mi:ss');
> -- Impala
> select to_timestamp('May 15, 2015 12:00:00', 'MMM dd, HH:mm:ss');
> -- PostgreSQL/Netezza/Vertica/Oracle
> select to_timestamp('2015-02-14 20:19:07','-mm-dd hh24:mi:ss');
> -- Impala
> select to_timestamp('2015-02-14 20:19:07','-MM-dd HH:mm:ss');
> -- Vertica/Oracle
> select to_timestamp('2015-02-14 20:19:07.123456','-mm-dd hh24:mi:ss.ff');
> -- Impala
> select to_timestamp('2015-02-14 20:19:07.123456','-MM-dd
> HH:mm:ss.SS');
> {noformat}
> *Considerations*
> Because this is a change in default behavior for to_timestamp(), if possible,
> having a feature flag to revert to the legacy Java SimpleDateFormat patterns
> should be strongly considered. This would allow users to chose the behavior
> they desire and scope it to a session if need be.
> SQL:2016 defines the following datetime templates
> {noformat}
> ::=
> { }...
> ::=
>
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> |
[jira] [Resolved] (IMPALA-6826) Add support for Ubuntu 18.04
[ https://issues.apache.org/jira/browse/IMPALA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Gaal resolved IMPALA-6826. - Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add support for Ubuntu 18.04 > > > Key: IMPALA-6826 > URL: https://issues.apache.org/jira/browse/IMPALA-6826 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 > Environment: Ubuntu 18.04 >Reporter: Jim Apple >Assignee: Laszlo Gaal >Priority: Major > Fix For: Impala 3.3.0 > > > We support Ubuntu 16.04 (and 14.04, in the 2.x line). > > I'm blocked on Ubuntu 18.04 support in > [https://github.com/cloudera/native-toolchain,] but the toolchain is not > technically a pre-requisite, though I believe it's the easiest way to get a > development environment up and running. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IMPALA-6826) Add support for Ubuntu 18.04
[ https://issues.apache.org/jira/browse/IMPALA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830344#comment-16830344 ] Laszlo Gaal edited comment on IMPALA-6826 at 4/30/19 2:30 PM: -- I have also added the job [https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/], which is a direct clone of the same job for Ubuntu 16.04. The job is operational, but it exposes some failures in BE_TESTS. was (Author: laszlog): I have also added the job [ubuntu-18.04-from-scratch|[https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/]], which is a direct clone of the same job for Ubuntu 16.04. The job is operational, but it exposes some failures in BE_TESTS. > Add support for Ubuntu 18.04 > > > Key: IMPALA-6826 > URL: https://issues.apache.org/jira/browse/IMPALA-6826 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 > Environment: Ubuntu 18.04 >Reporter: Jim Apple >Assignee: Laszlo Gaal >Priority: Major > > We support Ubuntu 16.04 (and 14.04, in the 2.x line). > > I'm blocked on Ubuntu 18.04 support in > [https://github.com/cloudera/native-toolchain,] but the toolchain is not > technically a pre-requisite, though I believe it's the easiest way to get a > development environment up and running. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-6826) Add support for Ubuntu 18.04
[ https://issues.apache.org/jira/browse/IMPALA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Gaal resolved IMPALA-6826. - Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add support for Ubuntu 18.04 > > > Key: IMPALA-6826 > URL: https://issues.apache.org/jira/browse/IMPALA-6826 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 > Environment: Ubuntu 18.04 >Reporter: Jim Apple >Assignee: Laszlo Gaal >Priority: Major > Fix For: Impala 3.3.0 > > > We support Ubuntu 16.04 (and 14.04, in the 2.x line). > > I'm blocked on Ubuntu 18.04 support in > [https://github.com/cloudera/native-toolchain,] but the toolchain is not > technically a pre-requisite, though I believe it's the easiest way to get a > development environment up and running. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6826) Add support for Ubuntu 18.04
[ https://issues.apache.org/jira/browse/IMPALA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830344#comment-16830344 ] Laszlo Gaal commented on IMPALA-6826: - I have also added the job [ubuntu-18.04-from-scratch|[https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/]], which is a direct clone of the same job for Ubuntu 16.04. The job is operational, but it exposes some failures in BE_TESTS. > Add support for Ubuntu 18.04 > > > Key: IMPALA-6826 > URL: https://issues.apache.org/jira/browse/IMPALA-6826 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 > Environment: Ubuntu 18.04 >Reporter: Jim Apple >Assignee: Laszlo Gaal >Priority: Major > > We support Ubuntu 16.04 (and 14.04, in the 2.x line). > > I'm blocked on Ubuntu 18.04 support in > [https://github.com/cloudera/native-toolchain,] but the toolchain is not > technically a pre-requisite, though I believe it's the easiest way to get a > development environment up and running. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8380) The minicluster metastore doesn't start on systems with postgres 10
[
https://issues.apache.org/jira/browse/IMPALA-8380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal resolved IMPALA-8380.
-
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> The minicluster metastore doesn't start on systems with postgres 10
> ---
>
> Key: IMPALA-8380
> URL: https://issues.apache.org/jira/browse/IMPALA-8380
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Lars Volker
>Assignee: Laszlo Gaal
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> We currently hardcode the postgres client version in at least 2 places:
> {noformat}
> $ git grep 9.0-801
> bin/impala-config.sh:129:export IMPALA_POSTGRES_JDBC_DRIVER_VERSION=9.0-801
> fe/pom.xml:425: 9.0-801.jdbc4
> {noformat}
> Apparently version 9 of the JDBC driver doesn't support connecting to
> postgres 10 ([Some info
> here|https://github.com/brettwooldridge/HikariCP/issues/1103#issuecomment-369767801]).
> We should upgrade to v42+.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-8380) The minicluster metastore doesn't start on systems with postgres 10
[
https://issues.apache.org/jira/browse/IMPALA-8380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal resolved IMPALA-8380.
-
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> The minicluster metastore doesn't start on systems with postgres 10
> ---
>
> Key: IMPALA-8380
> URL: https://issues.apache.org/jira/browse/IMPALA-8380
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Lars Volker
>Assignee: Laszlo Gaal
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> We currently hardcode the postgres client version in at least 2 places:
> {noformat}
> $ git grep 9.0-801
> bin/impala-config.sh:129:export IMPALA_POSTGRES_JDBC_DRIVER_VERSION=9.0-801
> fe/pom.xml:425: 9.0-801.jdbc4
> {noformat}
> Apparently version 9 of the JDBC driver doesn't support connecting to
> postgres 10 ([Some info
> here|https://github.com/brettwooldridge/HikariCP/issues/1103#issuecomment-369767801]).
> We should upgrade to v42+.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7875) Make Date a supported type
[ https://issues.apache.org/jira/browse/IMPALA-7875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges closed IMPALA-7875. > Make Date a supported type > -- > > Key: IMPALA-7875 > URL: https://issues.apache.org/jira/browse/IMPALA-7875 > Project: IMPALA > Issue Type: Sub-task >Affects Versions: Impala 3.2.0 >Reporter: Gabor Kaszab >Assignee: Gabor Kaszab >Priority: Major > Fix For: Not Applicable > > > Currently Date is listed as an unsupported type. Making it supported would be > a common step for multiple Sub-tasks under the "Implement Date type" parent > task hence I'm moving this to a separate Jira. > Apparently, once adding Date to the supported types list further adjustments > are required to 1) make Impala build 2) be able to launch cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7492) Add support for DATE text parser/formatter
[ https://issues.apache.org/jira/browse/IMPALA-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges closed IMPALA-7492. > Add support for DATE text parser/formatter > -- > > Key: IMPALA-7492 > URL: https://issues.apache.org/jira/browse/IMPALA-7492 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges closed IMPALA-7368. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges closed IMPALA-7368. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges resolved IMPALA-7368. -- Resolution: Implemented > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - BETWEEN operator > - conditional functions > - infrastructure changes for builtin scalar functions. > - some built-in functions: aggregate functions, analytical functions, math > functions. > - support partitioning. > - text support only. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8471) fs.defaultFS (file:///) is not supported
Lycan created IMPALA-8471:
-
Summary: fs.defaultFS (file:///) is not supported
Key: IMPALA-8471
URL: https://issues.apache.org/jira/browse/IMPALA-8471
Project: IMPALA
Issue Type: Question
Components: Backend
Affects Versions: Impala 3.2.0
Environment: centos7
jdk8
impala3.2
hadoop2.6-cdh5.5
Reporter: Lycan
Attachments: impalad.ERROR, impalad.INFO, start-impalad.sh
I install impala3.2 from building from scratching. I start catatlogd and
statestored successfully.But when I start impalad I get the following error.
{code:java}
I0430 11:51:27.325325 15093 JniFrontend.java:763] Short-circuit reads are not
enabled.
E0430 11:51:27.325462 15093 impala-server.cc:278] Currently configured default
filesystem: ProxyLocalFileSystem. fs.defaultFS (file:///) is not supported.
E0430 11:51:27.325484 15093 impala-server.cc:281] Aborting Impala Server
startup due to improper configuration. Impalad exiting
{code}
Here are my environment variable setting：
{code:java}
export IMPALA_HOME=/var/lib/impala/impala-3.2.0
export IMPALA_CONF_DIR=$IMPALA_HOME/conf
export CLUSTER_DIR=$IMPALA_HOME/testdata/cluster
export HADOOP_HOME=/var/lib/impala/impala-3.2.0/hadoop-2.6.0-cdh5.5.0
export HADOOP_LIB_DIR=$HADOOP_HOME/lib
export HADOOP_INCLUDE_DIR=$HADOOP_HOME/include
export HADOOP_CONF_DIR=/etc/hadoop/conf
export HDFS_CONF_DIR=/etc/hadoop/conf
export YARN_CONF_DIR=/etc/hadoop/conf
export JAVA_HOME=/usr/local/jdk8
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$IMPALA_HOME/shell:$PATH
{code}
Here are my impala config setting:
{code:java}
IMPALA_CATALOG_SERVICE_HOST=172.16.61.208
IMPALA_STATE_STORE_HOST=172.16.61.208
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/lib/impala/impala-3.2.0/logs
export IMPALA_CATALOG_ARGS=${IMPALA_CATALOG_ARGS:- \
-log_dir=${IMPALA_LOG_DIR}
export IMPALA_STATE_STORE_ARGS=${IMPALA_STATE_STORE_ARGS:- \
-log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}}
IMPALAD_SERVICE_ARGS=" \
-log_dir=${IMPALA_LOG_DIR} \
-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
-state_store_port=${IMPALA_STATE_STORE_PORT} \
-use_statestore \
-state_store_host=${IMPALA_STATE_STORE_HOST} \
-be_port=${IMPALA_BACKEND_PORT}"
export ENABLE_CORE_DUMPS=${ENABLE_COREDUMPS:-false}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8471) fs.defaultFS (file:///) is not supported
Lycan created IMPALA-8471:
-
Summary: fs.defaultFS (file:///) is not supported
Key: IMPALA-8471
URL: https://issues.apache.org/jira/browse/IMPALA-8471
Project: IMPALA
Issue Type: Question
Components: Backend
Affects Versions: Impala 3.2.0
Environment: centos7
jdk8
impala3.2
hadoop2.6-cdh5.5
Reporter: Lycan
Attachments: impalad.ERROR, impalad.INFO, start-impalad.sh
I install impala3.2 from building from scratching. I start catatlogd and
statestored successfully.But when I start impalad I get the following error.
{code:java}
I0430 11:51:27.325325 15093 JniFrontend.java:763] Short-circuit reads are not
enabled.
E0430 11:51:27.325462 15093 impala-server.cc:278] Currently configured default
filesystem: ProxyLocalFileSystem. fs.defaultFS (file:///) is not supported.
E0430 11:51:27.325484 15093 impala-server.cc:281] Aborting Impala Server
startup due to improper configuration. Impalad exiting
{code}
Here are my environment variable setting：
{code:java}
export IMPALA_HOME=/var/lib/impala/impala-3.2.0
export IMPALA_CONF_DIR=$IMPALA_HOME/conf
export CLUSTER_DIR=$IMPALA_HOME/testdata/cluster
export HADOOP_HOME=/var/lib/impala/impala-3.2.0/hadoop-2.6.0-cdh5.5.0
export HADOOP_LIB_DIR=$HADOOP_HOME/lib
export HADOOP_INCLUDE_DIR=$HADOOP_HOME/include
export HADOOP_CONF_DIR=/etc/hadoop/conf
export HDFS_CONF_DIR=/etc/hadoop/conf
export YARN_CONF_DIR=/etc/hadoop/conf
export JAVA_HOME=/usr/local/jdk8
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$IMPALA_HOME/shell:$PATH
{code}
Here are my impala config setting:
{code:java}
IMPALA_CATALOG_SERVICE_HOST=172.16.61.208
IMPALA_STATE_STORE_HOST=172.16.61.208
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/lib/impala/impala-3.2.0/logs
export IMPALA_CATALOG_ARGS=${IMPALA_CATALOG_ARGS:- \
-log_dir=${IMPALA_LOG_DIR}
export IMPALA_STATE_STORE_ARGS=${IMPALA_STATE_STORE_ARGS:- \
-log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}}
IMPALAD_SERVICE_ARGS=" \
-log_dir=${IMPALA_LOG_DIR} \
-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
-state_store_port=${IMPALA_STATE_STORE_PORT} \
-use_statestore \
-state_store_host=${IMPALA_STATE_STORE_HOST} \
-be_port=${IMPALA_BACKEND_PORT}"
export ENABLE_CORE_DUMPS=${ENABLE_COREDUMPS:-false}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
