[
https://issues.apache.org/jira/browse/IMPALA-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760860#comment-16760860
]
hakki edited comment on IMPALA-8109 at 2/5/19 2:32 PM:
-------------------------------------------------------
I tried similar query on my table, however it fails. Fsck output:
-
[root@AAAAAAAA ~]# hdfs fsck /user/impala/test/XXXXXXXXXX_198457.log.gz
Connecting to namenode via
http://FFF.FFF.FFF.FFF/user/impala/test/XXXXXXXXXX_198457.log.gz
FSCK started by root (auth:SIMPLE) from /XX.XX.XX.XX for path
/user/impala/test/XXXXXXXXXX_198457.log.gz at Tue Feb 05 17:23:52 EET 2019
.Status: HEALTHY
Total size: 2173247385 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 17 (avg. block size 127838081 B)
Minimally replicated blocks: 17 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 2.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 100
Number of racks: 1
FSCK ended at Tue Feb 05 17:23:52 EET 2019 in 0 milliseconds
The filesystem under path '/user/impala/test/XXXXXXXXXX_198457.log.gz' is
HEALTHY-
was (Author: hakkibc):
I tried similar query on my table, however it fails. Fsck output:
[root@AAAAAAAA ~]# hdfs fsck /user/impala/test/XXXXXXXXXX_198457.log.gz
Connecting to namenode via
http://FFF.FFF.FFF.FFF/user/impala/test/XXXXXXXXXX_198457.log.gz
FSCK started by root (auth:SIMPLE) from /XX.XX.XX.XX for path
/user/impala/test/XXXXXXXXXX_198457.log.gz at Tue Feb 05 17:23:52 EET 2019
.Status: HEALTHY
Total size: 2173247385 B
Total dirs: 0
Total files: 1
Total symlinks: 0
Total blocks (validated): 17 (avg. block size 127838081 B)
Minimally replicated blocks: 17 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 2.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 100
Number of racks: 1
FSCK ended at Tue Feb 05 17:23:52 EET 2019 in 0 milliseconds
The filesystem under path '/user/impala/test/XXXXXXXXXX_198457.log.gz' is
HEALTHY
> Impala cannot read the gzip files bigger than 2 GB
> --------------------------------------------------
>
> Key: IMPALA-8109
> URL: https://issues.apache.org/jira/browse/IMPALA-8109
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.12.0
> Reporter: hakki
> Assignee: Tim Armstrong
> Priority: Major
>
> When querying a partition containing gzip files, the query fails with the
> error below:
> WARNINGS: Disk I/O error: Error seeking to -2147483648 in file:
> hdfs://HADOOP_CLUSTER/user/hive/AAA/BBB/datehour=20180910/XXXXXXX.gz:
> Error(255): Unknown error 255
> Root cause: EOFException: Cannot seek to negative offset
> hdfs://HADOOP_CLUSTER/user/hive/AAA/BBB/datehour=20180910/XXXXXXX.gz file is
> a delimited text file and has a size of bigger than 2 GB (approx: 2.4 GB) The
> uncompressed size is ~13GB
> The impalad version is : 2.12.0-cdh5.15.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
