[
https://issues.apache.org/jira/browse/BOOKKEEPER-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411538#comment-13411538
]
Brahma Reddy Battula commented on BOOKKEEPER-337:
-------------------------------------------------
I had faced this in HDFS-HA,where ANN got shutdown by throwing following..Here
availoble bookies are seven even then addentry failed because continuous fail
bookie fail which are in ensemble
{noformat}
2012-07-11 04:02:50,445 ERROR
hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Invoked
callback method: 1058
2012-07-11 04:02:50,445 ERROR
hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Could not
write request for adding entry: 1058 ledger-id: 364 bookie: /HOST****.168:3183
2012-07-11 04:02:50,445 WARN
hidden.bkjournal.org.apache.bookkeeper.client.PendingAddOp: Write did not
succeed: 364, 1058
2012-07-11 04:02:50,445 ERROR
hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Invoked
callback method: 1058
2012-07-11 04:02:50,445 ERROR
hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Unexpected
add response received from bookie: /HOST****.168:3183 for ledger: 364, entry:
1058 , ignoring
2012-07-11 04:02:50,445 ERROR
hidden.bkjournal.org.apache.bookkeeper.proto.PerChannelBookieClient: Unexpected
add response received from bookie: /HOST****.168:3182 for ledger: 364, entry:
1058 , ignoring
2012-07-11 04:02:50,981 ERROR
hidden.bkjournal.org.apache.bookkeeper.client.LedgerHandle: Could not resolve
ledger metadata conflict while changing ensemble to: [/HOST****.20:3181,
/HOST****.168:3181, /10.18.52.55:3187], old meta data is
BookieMetadataFormatVersion 1
2
3
0
0 HOST****.168:3183 HOST****.168:3181 HOST****.168:3182
, new meta data is
BookieMetadataFormatVersion 1
2
3
0
0 HOST****.168:3183 HOST****.168:3181 HOST****.168:3182
1058 HOST****.168:3183 HOST****.168:3181 10.18.52.55:3187
,closing ledger
2012-07-11 04:02:51,366 FATAL org.apache.hadoop.hdfs.server.namenode.FSEditLog:
Error: flush failed for required journal
(JournalAndStream(mgr=org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager@f73f2c5,
stream=org.apache.hadoop.contrib.bkjournal.BookKeeperEditLogOutputStream@5476b70))
java.io.IOException: Failed to write to bookkeeper; Error is (-9) Error while
using ZooKeeper
at
org.apache.hadoop.contrib.bkjournal.BookKeeperEditLogOutputStream.flushAndSync(BookKeeperEditLogOutputStream.java:141)
at
org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream.flush(EditLogOutputStream.java:105)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalSetOutputStream$8.apply(JournalSet.java:460)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:319)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet.access$200(JournalSet.java:50)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalSetOutputStream.flush(JournalSet.java:456)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLog.logSync(FSEditLog.java:550)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2007)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:471)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:292)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42676)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
{noformat}
> Add entry fails with MetadataVersionException when last ensemble has morethan
> one bookie failures
> -------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-337
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-337
> Project: Bookkeeper
> Issue Type: Bug
> Reporter: Brahma Reddy Battula
> Assignee: Rakesh R
>
> Scenario:
> ========
> Start Five BK's
> Write ledger's with ensemble three and quroum size=2
> while write inprogress down two bookies(Bookies should be in ensemble)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
