Hi, We consider to update the version of Hbase.
The VM spec is below. CPU:2 Core MEMORY:4GB We don't know the hardware spec of host server, because we owe the VM from a VPS provider. Additionally, I check vmstat for the time spanning the isuue. No batch and online job worked on this server for this time. But the cpu usage was very high suddenly. Is it a clue? 2013/01/17 16:05:51 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ 2013/01/17 16:05:51 r b swpd free buff cache si so bi bo in cs us sy id wa st 2013/01/18 09:15:45 0 1 0 3113832 126920 573332 0 0 0 3 1001 121 0 0 100 0 0 2013/01/18 09:15:50 0 1 0 3113832 126924 573332 0 0 0 18 1002 125 0 0 100 0 0 2013/01/18 09:15:55 0 1 0 3113832 126924 573332 0 0 0 2 1001 123 0 0 100 0 0 2013/01/18 09:16:00 0 1 0 3113832 126924 573332 0 0 0 15 1001 122 0 0 100 0 0 2013/01/18 09:16:05 0 0 0 3113708 126940 573352 0 0 0 35 1002 140 0 0 100 0 0 2013/01/18 09:16:10 0 0 0 3113708 126944 573352 0 0 0 7 1001 122 0 0 100 0 0 2013/01/18 09:16:15 0 1 0 3113708 126944 573352 0 0 0 6 1001 122 0 0 100 0 0 2013/01/18 09:16:20 0 1 0 3113708 126944 573352 0 0 0 1 1001 122 0 0 100 0 0 2013/01/18 09:16:25 0 1 0 3113708 126948 573348 0 0 0 7 1001 123 0 0 100 0 0 2013/01/18 09:16:30 0 1 0 3113708 126948 573352 0 0 0 28 1001 130 0 0 100 0 0 2013/01/18 09:16:35 0 1 0 3113708 126948 573352 0 0 0 12 1001 128 0 0 100 0 0 2013/01/18 09:16:40 0 1 0 3113708 126956 573352 0 0 0 9 1001 127 0 0 100 0 0 2013/01/18 09:16:45 0 1 0 3113708 126956 573352 0 0 0 5 1001 127 0 0 100 0 0 2013/01/18 09:16:50 0 1 0 3113708 126956 573352 0 0 0 3 1001 128 0 0 100 0 0 2013/01/18 09:16:55 0 1 0 3113708 126964 573348 0 0 0 6 1001 128 0 0 99 0 0 2013/01/18 09:17:00 0 1 0 3113708 126972 573348 0 0 0 3 1001 127 0 0 100 0 0 2013/01/18 09:17:05 0 0 0 3113708 126976 573372 0 0 0 25 1002 142 0 0 100 0 0 2013/01/18 09:17:10 0 0 0 3113708 126976 573372 0 0 0 53 1002 128 0 0 100 0 0 2013/01/18 09:17:15 0 0 0 3113708 126980 573372 0 0 0 11 1001 127 0 0 100 0 0 2013/01/18 09:17:20 0 0 0 3113708 126984 573368 0 0 0 3 1001 126 0 0 100 0 0 2013/01/18 09:17:25 0 1 0 3113708 126992 573364 0 0 0 6 1001 127 0 0 100 0 0 2013/01/18 09:17:30 0 1 0 3113708 126992 573376 0 0 0 2 1001 125 0 0 100 0 0 2013/01/18 09:17:35 0 1 0 3113708 127000 573372 0 0 0 10 1001 128 0 0 100 0 0 2013/01/18 09:17:40 0 1 0 3113708 127000 573376 0 0 0 2 1001 126 0 0 100 0 0 2013/01/18 09:17:46 2 1 0 3113692 127004 573376 0 0 0 11 806 29 2 2 95 2 0 2013/01/18 09:17:53 1 1 0 3113708 127008 573376 0 0 0 10 1043 15 0 10 40 50 0 2013/01/18 09:18:00 1 1 0 3113708 127008 573376 0 0 0 4 1364 15 8 0 67 25 0 2013/01/18 09:18:06 3 0 0 3113584 127008 573396 0 0 0 26 1591 32 8 17 67 8 0 2013/01/18 09:18:12 1 0 0 3113584 127012 573396 0 0 0 21 1004 10 0 0 89 11 0 2013/01/18 09:18:17 1 1 0 3113584 127020 573388 0 0 0 34 1392 12 10 0 90 0 0 2013/01/18 09:18:53 2 2 0 3113584 127020 573396 0 0 0 6 1075 15 0 25 75 0 0 2013/01/18 09:18:58 1 1 0 3113584 127020 573396 0 0 0 14 6534 18 2 0 98 0 0 2013/01/18 09:19:04 2 0 0 3113576 127032 573392 0 0 0 9 1604 20 0 9 27 64 0 2013/01/18 09:19:11 3 1 0 3113584 127044 573416 0 0 0 41 1579 29 17 17 67 0 0 2013/01/18 09:19:17 1 1 0 3113584 127048 573416 0 0 0 3 705 10 0 0 50 50 0 2013/01/18 09:19:23 1 1 0 3113584 127060 573412 0 0 0 7 1684 15 0 6 88 6 0 2013/01/18 09:19:28 1 2 0 3113584 127060 573416 0 0 0 33 1153 21 10 20 60 10 0 2013/01/18 09:19:33 1 1 0 3113584 127060 573416 0 0 0 6 861 16 0 0 44 56 0 2013/01/18 09:19:39 1 1 0 3113584 127068 573416 0 0 0 6 1036 16 0 0 78 22 0 2013/01/18 09:19:44 0 1 0 3113584 127072 573416 0 0 0 3 1007 10 14 14 57 14 0 2013/01/18 09:19:49 0 1 0 3113584 127076 573416 0 0 0 6 1369 18 0 0 91 9 0 2013/01/18 09:19:56 2 0 0 3113584 127084 573408 0 0 0 8 596 11 20 20 60 0 0 2013/01/18 09:20:01 1 1 0 3113584 127084 573416 0 0 0 6 1421 10 0 8 92 0 0 2013/01/18 09:20:16 2 2 0 3113444 127096 573432 0 0 0 60 804 26 20 0 20 60 0 2013/01/18 09:20:21 0 1 0 3113460 127100 573432 0 0 0 10 3578 20 4 4 32 61 0 2013/01/18 09:20:26 1 1 0 3113460 127100 573432 0 0 0 13 833 16 0 14 71 14 0 2013/01/18 09:20:32 1 1 0 3113460 127100 573432 0 0 0 0 802 8 0 0 0 100 0 2013/01/18 09:20:40 1 1 0 3113460 127104 573432 0 0 0 8 1175 9 10 10 0 80 0 2013/01/18 09:20:46 1 1 0 3113460 127108 573432 0 0 0 7 1850 14 0 0 87 13 0 2013/01/18 09:20:51 0 1 0 3113460 127112 573432 0 0 0 14 817 7 14 0 57 29 0 2013/01/18 09:20:56 2 1 0 3113460 127120 573432 0 0 0 6 1250 16 0 0 56 44 0 2013/01/18 09:21:05 5 1 0 3113452 127128 573432 0 0 0 3 1302 16 0 0 100 0 0 2013/01/18 09:21:14 1 1 0 3113336 127136 573460 0 0 0 48 1400 27 20 20 60 0 0 2013/01/18 09:21:23 1 1 0 3113336 127136 573460 0 0 0 35 1899 11 0 0 100 0 0 2013/01/18 09:21:28 0 1 0 3113336 127144 573456 0 0 0 10 1963 17 5 5 21 68 0 2013/01/18 09:21:33 1 1 0 3113336 127148 573460 0 0 0 2 761 13 0 20 40 40 0 2013/01/18 09:21:38 0 1 0 3113336 127148 573460 0 0 0 0 1346 17 10 10 0 80 0 2013/01/18 09:21:44 1 1 0 3113336 127148 573460 0 0 0 2 622 12 0 0 0 100 0 2013/01/18 09:21:49 1 0 0 3113336 127156 573460 0 0 0 11 1433 14 10 0 70 20 0 2013/01/18 09:21:54 0 0 0 3113336 127160 573456 0 0 0 11 1190 13 0 0 80 20 0 2013/01/18 09:22:00 3 1 0 3113336 127160 573460 0 0 0 2 981 14 0 9 91 0 0 2013/01/18 09:22:09 1 0 0 3113336 127168 573476 0 0 0 45 1125 26 25 38 25 13 0 2013/01/18 09:22:15 1 1 0 3113336 127168 573480 0 0 0 1 1294 9 0 9 91 0 0 2013/01/18 09:22:21 1 1 0 3113336 127172 573480 0 0 0 11 1696 14 8 8 85 0 0 2013/01/18 09:22:26 0 1 0 3113336 127176 573480 0 0 0 8 1123 12 0 0 57 43 0 2013/01/18 09:22:31 1 2 0 3113336 127184 573480 0 0 0 33 1000 15 10 0 70 20 0 2013/01/18 09:22:36 1 1 0 3113336 127188 573480 0 0 0 3 880 12 0 14 43 43 0 2013/01/18 09:22:41 0 1 0 3113336 127188 573480 0 0 0 7 1399 32 0 1 96 3 0 2013/01/18 09:22:46 0 1 0 3113336 127188 573480 0 0 0 6 1001 121 0 0 100 0 0 2013/01/18 09:22:51 0 1 0 3113336 127196 573476 0 0 0 3 1001 121 0 0 100 0 0 2013/01/18 09:22:56 0 1 0 3113336 127196 573480 0 0 0 10 1001 127 0 0 100 0 0 2013/01/18 09:23:01 0 0 0 3113336 127204 573476 0 0 0 33 1002 141 0 0 100 0 0 2013/01/18 09:23:06 0 0 0 3113336 127204 573500 0 0 0 7 1001 127 0 0 100 0 0 2013/01/18 09:23:11 0 1 0 3113336 127204 573500 0 0 0 26 1000 124 0 0 100 0 0 2013/01/18 09:23:16 0 0 0 3113336 127212 573496 0 0 0 8 1002 128 0 0 100 0 0 2013/01/18 09:23:21 0 1 0 3113336 127216 573496 0 0 0 3 1001 124 0 0 100 0 0 2013/01/18 09:23:26 0 1 0 3113336 127216 573500 0 0 0 6 1001 127 0 0 100 0 0 2013/01/18 09:23:31 0 0 0 3113336 127220 573496 0 0 0 9 1002 128 0 0 100 0 0 2013/01/18 09:23:36 0 1 0 3113336 127224 573496 0 0 0 6 1001 126 0 0 100 0 0 2013/01/18 09:23:41 1 1 0 3113336 127232 573496 0 0 0 4 1001 115 0 0 100 0 0 -------------------------------------------- On 2013/02/14 22:10, Michel Segel [via Apache HBase] wrote: > > > First, > In the world of Hadoop, if it ain't broke don't fix it, may not be the best > advice. > HBase is still evolving at a good pace and you want to be closer to the > latest releases. > CDH4 is stable so that I would agree that going to CDH4 would be best. > > Second. > You are running this as a single machine within the VM. > > What does the hardware look like? > Number of cores, physical or virtual ? > How much memory? > > What type of disks and where are they? > ( attached or SAN?) > > > > > Sent from a remote device. Please excuse any typos... > > Mike Segel > > On Feb 14, 2013, at 1:16 AM, So Hibino <[email protected]> wrote: > >> Hi, >> >> The VM was rebooted last night because of the host server memory error. >> So the dmesg was cleared at the time of the isuue. >> >> The below dmesg is new and the issue has not been happened yet. >> >> Thank you, >> So >> >> On 2013/02/14 15:10, So Hibino wrote: >>> Hi, >>> >>> I pasted dmseg and syslog on pastebin.com. >>> >>> URL is below. >>> >>> dmseg >>> http://pastebin.com/t3i3KRJ8 >>> >>> syslog >>> http://pastebin.com/LcBRYMyX >>> >>> Any messages wasn't output for the time spanning the isuue. >>> >>> thank you, >>> So >>> >>> >>> On 2013/02/13 9:31, Jean-Daniel Cryans [via Apache HBase] wrote: >>>> >>>> >>>> messages and syslog please, just for the time spanning the issue. >>>> >>>> J-D >>>> >>>> On Tue, Feb 12, 2013 at 4:10 PM, So Hibino <[email protected]> wrote: >>>>> Hi, >>>>> >>>>>> Can you post the dmesg and syslog spanning the time of the issue? >>>>> Is the syslog you mean /var/log/message ? >>>>> >>>>> >>>>> On 2013/02/13 4:00, Jean-Daniel Cryans [via Apache HBase] wrote: >>>>>> On Mon, Feb 11, 2013 at 8:06 PM, So Hibino <[hidden email] >>>>>> </user/SendEmail.jtp?type=node&node=4038442&i=0>> wrote: >>>>>> >>>>>>> master log >>>>>> >>>>>> Unfortunately there's not much in there apart from the weird sleep >>>>>> times. At this point I'd consider OS-level issues. >>>>>> >>>>>>> We doesn't use EC2,but this server works with KVM. >>>>>> >>>>>> Can you post the dmesg and syslog spanning the time of the issue? >>>>>> Please use pastebin.com as it can be really hard to read the logs in >>>>>> emails. >>>>>> >>>>>> Thx, >>>>>> >>>>>> J-D >>>>>> >>>>>> >>>>>> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------! --! >> -! >>> --! >>>>> --- >>>>>> If you reply to this email, your message will be added to the discussion >>>>>> below: >>>>>> http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038442.html >>>>>> To unsubscribe from hbase-master-server slept, click here >>>>>> <http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4038192&code=aGliaW5vLnNvQGxhYi5udHQuY28uanB8NDAzODE5MnwtNjAxOTg5MzI0>. >>>>>> NAML >>>>>> <http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>>>> >>>>> >>>>> -- >>>>> H24.07より組織名が変わりました。 >>>>> >>>>> NTTサービスエボリューション研究所 >>>>> 日比野 壮 （So Hibino） >>>>> >>>>> 〒239-0847 横須賀市光の丘1-1 (320C) >>>>> TEL : 046-859-4582 FAX : 046-855-1282 >>>>> e-mail : [email protected] >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038459.html >>>>> Sent from the HBase User mailing list archive at Nabble.com. >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> If you reply to this email, your message will be added to the discussion >>>> below: >>>> http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038461.html >>>> >>>> To unsubscribe from hbase-master-server slept, visit >>>> http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4038192&code=aGliaW5vLnNvQGxhYi5udHQuY28uanB8NDAzODE5MnwtNjAxOTg5MzI0 >> >> >> -- >> H24.07より組織名が変わりました。 >> >> NTTサービスエボリューション研究所 >> 日比野 壮 （So Hibino） >> >> 〒239-0847 横須賀市光の丘1-1 (320C) >> TEL : 046-859-4582 FAX : 046-855-1282 >> e-mail : [email protected] >> >> >> >> >> >> -- >> View this message in context: >> http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038516.html >> Sent from the HBase User mailing list archive at Nabble.com. > > > > > _______________________________________________ > If you reply to this email, your message will be added to the discussion > below: > http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038520.html > > To unsubscribe from hbase-master-server slept, visit > http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4038192&code=aGliaW5vLnNvQGxhYi5udHQuY28uanB8NDAzODE5MnwtNjAxOTg5MzI0 > -- H24.07より組織名が変わりました。 NTTサービスエボリューション研究所 日比野 壮 （So Hibino） 〒239-0847 横須賀市光の丘1-1 (320C) TEL : 046-859-4582 FAX : 046-855-1282 e-mail : [email protected] -- View this message in context: http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038559.html Sent from the HBase User mailing list archive at Nabble.com.
