On Mar 8, 2012, at 3:12 AM, [email protected] wrote: > Hi ceph-devel, > I came up with the following problems when I tried to let Hadoop use Ceph. > Could you give any advice? Thanks firstly! > I installed ceph-0.20.2 and hadoop.0.20.205.0 with the > hadoop.0.20.205.0-cephfs1.patch successfully. Both ceph and hadoop(using > HDFS) can work well separately. Besides, I could run command like 'hadoop fs > -ls /', 'hadoop fs -put /localfilepath /cephfilepath' correctly. But when I > run a mapreduce demo: wordcount with a small file containing about two > thousand words, reduce task crashed reporting information like: > A fatal error has been detected by the Java Runtime > Environment:...Problematic frame:C [libceph.so.1.0.0+0x898e7]…
Hi Xiaobing, It's a good sign that you are able to use 'hadoop fs -put' to load data into Ceph. There are some edge cases with library loading that aren't quite robust enough yet. It would be great if you could provide some more information: 1. What distribution are you using? 2. When the crash occurs there should be a large block of text along side the "Problematic frame…" message that the JVM dumps out. Could you post that here? 3. The JVM should be producing a file that looks like "hs_err_pid1201.log". Attaching that is also helpful. You are definitely correct that having version mis-matches can cause problems, so making sure to have a clean installation is important when you are switching versions of Ceph and Hadoop. On the wiki http://ceph.newdream.net/wiki/Hadoop_FileSystem there are specific Git SHA1 versions that are known to work on Ubuntu 11.10. Thanks, Noah-- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
