[
https://issues.apache.org/jira/browse/IMPALA-9446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060993#comment-17060993
]
Norbert Luksa commented on IMPALA-9446:
---------------------------------------
Looks like ASF Jira bot failed to copy the commit message, so here it is for
reference:
IMPALA-9446: Fix bug that impala failed to read zstd file on s3
S3 filesystem doesn't have a block concept, so scheduler splits each
file into a smaller scan ranges. Adding ZSTD as a case in the switch
statement to add support for zstd.
Testing done:
In mini-cluster, read external table located on s3 zstd file.
Run end to end test successfully on s3.
Change-Id: I13f8837fda7454ddb4bb47d20a675d6315a3462d
Reviewed-on: http://gerrit.cloudera.org:8080/15391
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Tim Armstrong <[email protected]>
> query_test.test_compressed_formats.TestReadZtsdLibCompressedFile fails on S3
> ----------------------------------------------------------------------------
>
> Key: IMPALA-9446
> URL: https://issues.apache.org/jira/browse/IMPALA-9446
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Assignee: Xiaomeng Zhang
> Priority: Blocker
> Labels: broken-build
>
> On the S3 configuration,
> query_test.test_compressed_formats.TestReadZtsdLibCompressedFile is
> encountering the following error:
>
> {noformat}
> query_test/test_compressed_formats.py:325: in test_query_large_file
> result = self.client.execute("select count(*) from %s" %
> self.COMPRESSED_TABLE_NAME)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:386: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Rejected query from pool default-pool: Error during
> scheduling: Invalid file descriptor compression code: 10{noformat}
> The error comes from here:
>
> [https://github.com/apache/impala/blob/master/be/src/util/flat_buffer.cc#L63]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
