Tim Armstrong created IMPALA-6394:
-------------------------------------
Summary: GVO failed in "Waiting for HDFS replication"
Key: IMPALA-6394
URL: https://issues.apache.org/jira/browse/IMPALA-6394
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 2.12.0
Reporter: Tim Armstrong
Assignee: Tianyi Wang
Attachments: wait-hdfs-replication.log
A GVO failed at this step in data loading:
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/942/console . I
attached the log tail is:
{noformat}
/test-warehouse/zipcode_incomes_seq_gzip/000000_0: Under replicated
BP-611512709-127.0.0.1-1515787500736:blk_1073751054_10230. Target Replicas is 3
but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
replica(s).
..
/test-warehouse/zipcode_incomes_seq_record_def/000000_0: Under replicated
BP-611512709-127.0.0.1-1515787500736:blk_1073750916_10092. Target Replicas is 3
but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
replica(s).
..
/test-warehouse/zipcode_incomes_seq_record_snap/000000_0: Under replicated
BP-611512709-127.0.0.1-1515787500736:blk_1073751714_10890. Target Replicas is 3
but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
replica(s).
...
/test-warehouse/zipcode_incomes_text_gzip/000000_0.gz: Under replicated
BP-611512709-127.0.0.1-1515787500736:blk_1073748020_7196. Target Replicas is 3
but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
replica(s).
..
/test-warehouse/zipcode_incomes_text_lzo/000000_0.lzo.index: Under replicated
BP-611512709-127.0.0.1-1515787500736:blk_1073752971_12147. Target Replicas is 3
but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning
replica(s).
.Status: HEALTHY
Total size: 8422045399 B
Total dirs: 8441
Total files: 10228
Total symlinks: 0
Total blocks (validated): 10249 (avg. block size 821743 B)
Minimally replicated blocks: 10249 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 1523 (14.859986 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 2.8514001
Corrupt blocks: 0
Missing replicas: 1523 (4.9533286 %)
Number of data-nodes: 3
Number of racks: 1
FSCK ended at Fri Jan 12 20:57:37 UTC 2018 in 140 milliseconds
The filesystem under path '/test-warehouse' is HEALTHY
Some HDFS blocks are still under replicated after 30s.
Some tests cannot pass without fully replicated blocks (IMPALA-3887).
Failing the data loading.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
