Hi, > Ah... I remember that there is a file whose name is "version-xxx", in which the "xxx" is a counter, and will be updated if the version difference between the disk and memory >= 50.
Yes, each storage group has a file named version-xxx under system/storage_group_xxx folder. > I am not sure whether it is correct that "renaming is not very frequent" because each ChunkGroup has a version (am I right?).. Right, each ChunkGroup will increase this version, and each new data file will increase this version. Currently, renaming only occurs in two case (1) updating the version files for each storage group periodically in system/storage_group_x/version-xxx (2) updating the compression-ratio file when dynamically adjusting parameters It's hard to say whether it is frequent or not, we can compare the performance between local file system and HDFS after supporting HDFS. Best, -- Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院 > -----原始邮件----- > 发件人: "Xiangdong Huang" <[email protected]> > 发送时间: 2019-09-01 23:10:08 (星期日) > 收件人: [email protected] > 抄送: > 主题: Re: [IoTDB-189] question about the feature of compatibility of HDFS > > Hi, > > > since renaming is not very frequent, maybe we do not need to worry about > performance. > > Ah... I remember that there is a file whose name is "version-xxx", in which > the "xxx" is a counter, and will be updated if the version difference > between the disk and memory >= 50. > > I am not sure whether it is correct that "renaming is not very frequent" > because each ChunkGroup has a version (am I right?).. > > But, I am not saying that writing on HDFS is bad, I just want to remind you > that reducing renaming operations, or proving that renaming on HDFS is not > a time consuming operation. > > Best, > ----------------------------------- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Zesong Sun <[email protected]> 于2019年8月31日周六 下午8:28写道： > > > Hi, > > > > > > This issue is a sub-task of IoTDB-187 [1], which enables users to choose > > whether files are storied in local file system or HDFS. > > > > > > In this sub-task, I intend to use an encapsulated "File" to replace > > existed Java File in our project. > > > > > > Currently in our design, all files could be written on HDFS. > > > > > > HDFS supports truncating files, and since renaming is not very frequent, > > maybe we do not need to worry about performance. > > > > > > If anyone has any other ideas, please discuss with me. > > > > > > [1] https://issues.apache.org/jira/browse/IOTDB-187 > > > > > > ------------------ > > Zesong Sun > > School of Software, Tsinghua University > > > > 孙泽嵩 > > 清华大学 软件学院 > > > > > > > > > > > > > > > > ------------------ 原始邮件 ------------------ > > 发件人: "Xiangdong Huang"<[email protected]>; > > 发送时间: 2019年8月31日(星期六) 晚上10:50 > > 收件人: "dev"<[email protected]>; > > > > 主题: [IoTDB-189] question about the feature of compatibility of HDFS > > > > > > > > Hi, > > > > This issue only has a title.. without any description... > > > > I think it is for letting IoTDB write TsFiles on HDFS directly. > > > > My question is, is only TsFile written on HDFS? or all files (e.g., wal, > > system data file)? > > > > Besides, if all files are on HDFS, do we need to avoid using File name as a > > mark or a lock (i.e., create a file and then rename it) for performance > > consideration? > > > > (I am on my trip and then I realize clearly that current info on issue is > > not enough for guys who are on the internet) > > > > Best, > > ----------------------------------- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院
