[
https://issues.apache.org/jira/browse/IMPALA-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyi Wang resolved IMPALA-6394.
---------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.12.0
I haven't seen this for a while. Restarting HDFS seems to work.
> GVO failed in "Waiting for HDFS replication"
> --------------------------------------------
>
> Key: IMPALA-6394
> URL: https://issues.apache.org/jira/browse/IMPALA-6394
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.12.0
> Reporter: Tim Armstrong
> Assignee: Tianyi Wang
> Priority: Critical
> Labels: broken-build, flaky
> Fix For: Impala 2.12.0
>
> Attachments: wait-hdfs-replication.log
>
>
> A GVO failed at this step in data loading:
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/942/console . I
> attached the log tail is:
> {noformat}
> /test-warehouse/zipcode_incomes_seq_gzip/000000_0: Under replicated
> BP-611512709-127.0.0.1-1515787500736:blk_1073751054_10230. Target Replicas is
> 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
> replica(s).
> ..
> /test-warehouse/zipcode_incomes_seq_record_def/000000_0: Under replicated
> BP-611512709-127.0.0.1-1515787500736:blk_1073750916_10092. Target Replicas is
> 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
> replica(s).
> ..
> /test-warehouse/zipcode_incomes_seq_record_snap/000000_0: Under replicated
> BP-611512709-127.0.0.1-1515787500736:blk_1073751714_10890. Target Replicas is
> 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
> replica(s).
> ...
> /test-warehouse/zipcode_incomes_text_gzip/000000_0.gz: Under replicated
> BP-611512709-127.0.0.1-1515787500736:blk_1073748020_7196. Target Replicas is
> 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
> replica(s).
> ..
> /test-warehouse/zipcode_incomes_text_lzo/000000_0.lzo.index: Under
> replicated BP-611512709-127.0.0.1-1515787500736:blk_1073752971_12147. Target
> Replicas is 3 but found 2 live replica(s), 0 decommissioned replica(s), 0
> decommissioning replica(s).
> .Status: HEALTHY
> Total size: 8422045399 B
> Total dirs: 8441
> Total files: 10228
> Total symlinks: 0
> Total blocks (validated): 10249 (avg. block size 821743 B)
> Minimally replicated blocks: 10249 (100.0 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 1523 (14.859986 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 3
> Average block replication: 2.8514001
> Corrupt blocks: 0
> Missing replicas: 1523 (4.9533286 %)
> Number of data-nodes: 3
> Number of racks: 1
> FSCK ended at Fri Jan 12 20:57:37 UTC 2018 in 140 milliseconds
> The filesystem under path '/test-warehouse' is HEALTHY
> Some HDFS blocks are still under replicated after 30s.
> Some tests cannot pass without fully replicated blocks (IMPALA-3887).
> Failing the data loading.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
