JNI native-method warning in HDP 2.0

2014-10-16 Thread John Lilley
We are seeing a warning deep in HDFS code, I was wondering if anyone knows of this or if a JIRA has been filed or fixed? Searching on the warning text didn't crop up anything. It is not harmful AFAIK. John [Dynamic-linking native method java.net.NetworkInterface.init ... JNI] WARNING in

decommissioning disks on a data node

2014-10-16 Thread Colin Kincaid Williams
We have been seeing some of the disks on our cluster having bad blocks, and then failing. We are using some dell PERC H700 disk controllers that create virtual devices. Our hosting manager uses a dell utility which reports virtual device bad blocks. He has suggested that we use the dell tool to

hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?

2014-10-16 Thread Manoj Samel
Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade from 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and write the new 2.4 protobuf format. After using 2.4, if there is a need to downgrade back to 2.3, how would that work ? Thanks,

Re: decommissioning disks on a data node

2014-10-16 Thread Travis
On Thu, Oct 16, 2014 at 7:03 PM, Colin Kincaid Williams disc...@uw.edu wrote: We have been seeing some of the disks on our cluster having bad blocks, and then failing. We are using some dell PERC H700 disk controllers that create virtual devices. Are you doing a bunch of single-disk RAID0

Re: decommissioning disks on a data node

2014-10-16 Thread Colin Kincaid Williams
For some reason he seems intent on resetting the bad Virtual blocks, and giving the drives another shot. From what he told me, nothing is under warranty anymore. My first suggestion was to get rid of the disks. Here's the command: /opt/dell/srvadmin/bin/omconfig storage vdisk

Re: decommissioning disks on a data node

2014-10-16 Thread Travis
On Thu, Oct 16, 2014 at 10:01 PM, Colin Kincaid Williams disc...@uw.edu wrote: For some reason he seems intent on resetting the bad Virtual blocks, and giving the drives another shot. From what he told me, nothing is under warranty anymore. My first suggestion was to get rid of the disks.

Re: decommissioning disks on a data node

2014-10-16 Thread Colin Kincaid Williams
Hi Travis, Thanks for your input. I forgot to mention that the drives are most likely in the single drive configuration that you describe. I think what I've found is that restarting the datanodes in the manner I describe shows that the mount points on the drives with the reset blocks and newly