i'm sure i close all the files in the reduce step. Any other reasons cause this problem?
2008/6/18 Konstantin Shvachko <[EMAIL PROTECTED]>: > Did you close those files? > If not they may be empty. > > > > ??? wrote: > >> Dears, >> >> I use hadoop-0.16.4 to do some work and found a error which i can't get >> the >> reasons. >> >> The scenario is like this: In the reduce step, instead of using >> OutputCollector to write result, i use FSDataOutputStream to write result >> to >> files on HDFS(becouse i want to split the result by some rules). After the >> job finished, i found that *some* files(but not all) are empty on HDFS. >> But >> i'm sure in the reduce step the files are not empty since i added some >> logs >> to read the generated file. It seems that some file's contents are lost >> after the reduce step. Is anyone happen to face such errors? or it's a >> hadoop bug? >> >> Please help me to find the reason if you some guys know >> >> Thanks & Regards >> Guangfeng >> >> -- Guangfeng Jin Software Engineer iZENEsoft (Shanghai) Co., Ltd Room 601 Marine Tower, No. 1 Pudong Ave. Tel:86-21-68860698 Fax:86-21-68860699 Mobile: 86-13621906422 Company Website:www.izenesoft.com
