Re: Review Request 39128: Journal node went in critical state on Ambari : message on UI : Connection failed: [Errno 111] Connection refused to 0.0.0.0:8480

[Andrew Onischuk]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39128/
-----------------------------------------------------------

(Updated Oct. 12, 2015, 10:02 a.m.)


Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-13355
    https://issues.apache.org/jira/browse/AMBARI-13355


Repository: ambari


Description
-------

Ambari UI is showing alert for HDFS->Journal Node Process.  
By alert definition "This host-level alert is triggered if the JournalNode
process cannot be confirmed to be up and listening on the network."

When i check on one of the journal node last successful call was made at -  
2015-09-29 08:07:11,892

    
    
    
    2015-09-29 08:07:11,892 INFO  namenode.FSImage 
(FSEditLogLoader.java:loadFSEdits(145)) - Edits file 
/grid/0/hadoop/hdfs/namenode/current/edits_0000000000000002757-0000000000000002781,
 
http://scale-test-20-nodes-part1-1.novalocal:8480/getJournal?jid=nameservice&segmentTxId=2757&storageInfo=-63%3A1242882785%3A0%3ACID-6fd83300-71b7-41e3-aa2f-c40614cae059,
 
http://scale-test-20-nodes-part1-11.novalocal:8480/getJournal?jid=nameservice&segmentTxId=2757&storageInfo=-63%3A1242882785%3A0%3ACID-6fd83300-71b7-41e3-aa2f-c40614cae059
 of size 1048576 edits # 25 loaded in 0 seconds
    

But the JournalNode process was running

    
    
    
    [root@scale-test-20-nodes-part1-1 hdfs]# /usr/jdk64/jdk1.7.0_67/bin/jps
    21731
    23257 DFSZKFailoverController
    3220 ApplicationHistoryServer
    25424 QuorumPeerMain
    27350 NodeManager
    23011
    5427 -- process information unavailable
    26990 ResourceManager
    5354 Application
    27938 Jps
    6385 Application
    22143 JournalNode
    23514 NameNode
    3176 HMaster


Diffs (updated)
-----

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 62cd868 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
 cd57df1 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java
 dcdbb85 
  
ambari-server/src/main/java/org/apache/ambari/server/utils/EventBusSynchronizer.java
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
2ea9446 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java
 a54dee2 
  
ambari-server/src/test/java/org/apache/ambari/server/utils/EventBusSynchronizer.java
 d9be622 

Diff: https://reviews.apache.org/r/39128/diff/


Testing
-------

mvn clean test


Thanks,

Andrew Onischuk