[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464005#comment-13464005 ] Lars Hofhansl commented on HDFS-744: BTW. I filed HDFS-3979 to do that.

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464004#comment-13464004 ] Lars Hofhansl commented on HDFS-744: I see. In that case we wouldn't ack back until all

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463964#comment-13463964 ] Kan Zhang commented on HDFS-744: bq. I think the problem is that we have to enqueue the seqn

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463512#comment-13463512 ] Lars Hofhansl commented on HDFS-744: Another approach would be to wait in the responder

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463505#comment-13463505 ] Lars Hofhansl commented on HDFS-744: I think the problem is that we have to enqueue the

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463373#comment-13463373 ] Kan Zhang commented on HDFS-744: bq. As long as the syncs (or flushes) are not serialized it

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462869#comment-13462869 ] Lars Hofhansl commented on HDFS-744: In any case, hsync and hflush should be fixed toget

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462855#comment-13462855 ] Lars Hofhansl commented on HDFS-744: You are right Luke. I implemented this in the conte

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462578#comment-13462578 ] Luke Lu commented on HDFS-744: -- I think that it's important enough to clarify: bq. When hsync

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422901#comment-13422901 ] Todd Lipcon commented on HDFS-744: -- 2.0.x is supposed to be for critical bug fixes and secu

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422842#comment-13422842 ] Lars Hofhansl commented on HDFS-744: Just noticed (because of HDFS-3721) that this is no

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288732#comment-13288732 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - Hi Lars, I just have merged this

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288691#comment-13288691 ] Lars Hofhansl commented on HDFS-744: Two responses on DEV list on port to 2.0, both in f

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286579#comment-13286579 ] Hudson commented on HDFS-744: - Integrated in Hadoop-Mapreduce-trunk #1096 (See [https://builds.

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286496#comment-13286496 ] Hudson commented on HDFS-744: - Integrated in Hadoop-Hdfs-trunk #1062 (See [https://builds.apach

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286457#comment-13286457 ] Lars Hofhansl commented on HDFS-744: I'm happy to send an email to DEV list, BTW.

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286428#comment-13286428 ] Lars Hofhansl commented on HDFS-744: The principle change for 1.0 is the same, but the F

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286270#comment-13286270 ] Eli Collins commented on HDFS-744: -- We're still not out of alpha for 2.0, IMO this seems re

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286255#comment-13286255 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - For backporting it to 1.0, it req

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286067#comment-13286067 ] Lars Hofhansl commented on HDFS-744: I agree. Or at least Hadoop 2.0.x.

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286062#comment-13286062 ] Zhihong Yu commented on HDFS-744: - Backport to hadoop 1.0 would be nice. >

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285992#comment-13285992 ] Hudson commented on HDFS-744: - Integrated in Hadoop-Mapreduce-trunk-Commit #2320 (See [https://

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285963#comment-13285963 ] Lars Hofhansl commented on HDFS-744: Thanks Nicholas! > Support hsync i

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285940#comment-13285940 ] Hudson commented on HDFS-744: - Integrated in Hadoop-Hdfs-trunk-Commit #2375 (See [https://build

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285936#comment-13285936 ] Hudson commented on HDFS-744: - Integrated in Hadoop-Common-trunk-Commit #2302 (See [https://bui

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285926#comment-13285926 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - If Todd is not around, then I wil

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285895#comment-13285895 ] Eli Collins commented on HDFS-744: -- @Lars, Todd is on vacation this week, might take him so

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285499#comment-13285499 ] Lars Hofhansl commented on HDFS-744: @Todd: Could you have a quick look? There are possi

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283685#comment-13283685 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - +1 patch looks good. Todd, do yo

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13282068#comment-13282068 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281942#comment-13281942 ] Todd Lipcon commented on HDFS-744: -- Looks pretty good! Just a few small things: - looks li

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279637#comment-13279637 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279619#comment-13279619 ] Lars Hofhansl commented on HDFS-744: Upon further inspection it turns out the issue with

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279501#comment-13279501 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279393#comment-13279393 ] Lars Hofhansl commented on HDFS-744: I like SequenceFile.Writer implementing Syncable.

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279383#comment-13279383 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - Hi Todd, you make a good point th

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279378#comment-13279378 ] Todd Lipcon commented on HDFS-744: -- Here's my opinion: let's create a {{syncFs(SyncType typ

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279323#comment-13279323 ] Lars Hofhansl commented on HDFS-744: Oh, and I'll add the metrics too.

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279321#comment-13279321 ] Lars Hofhansl commented on HDFS-744: Thanks Todd and Nicholas. * To your first point, t

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279255#comment-13279255 ] Todd Lipcon commented on HDFS-744: -- Quickly looked at the patch. A few notes: - I don't th

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279166#comment-13279166 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - The new javac warning is [WARNIN

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278633#comment-13278633 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278544#comment-13278544 ] Lars Hofhansl commented on HDFS-744: Alternatively we could add flushFS() to SequenceFil

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278488#comment-13278488 ] Lars Hofhansl commented on HDFS-744: Thanks Nicholas! I'll rename the flag and the flus

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278444#comment-13278444 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - Thanks a lot, Lars! The patch lo

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276353#comment-13276353 ] Lars Hofhansl commented on HDFS-744: In HBASE-5954 I added some performance tests on HBa

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274046#comment-13274046 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273866#comment-13273866 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273826#comment-13273826 ] Lars Hofhansl commented on HDFS-744: Looking at the test failures. > Su

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273749#comment-13273749 ] Hadoop QA commented on HDFS-744: -1 overall. Here are the results of testing the latest att

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270239#comment-13270239 ] Lars Hofhansl commented on HDFS-744: Thanks Todd. I'll create a trunk patch tomorrow (ho

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270233#comment-13270233 ] Todd Lipcon commented on HDFS-744: -- One quick note, though: in order for this to be reviewe

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270228#comment-13270228 ] Todd Lipcon commented on HDFS-744: -- Hi Lars. Sorry, I wasn't watching this before so I miss

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269798#comment-13269798 ] Lars Hofhansl commented on HDFS-744: Thanks Stack. The sync flags does two things: it c

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269701#comment-13269701 ] stack commented on HDFS-744: Patch looks good to me. Were you going to expose a file mode inste

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269147#comment-13269147 ] Lars Hofhansl commented on HDFS-744: Turns out no new DataTransferProtocol option is nee

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268803#comment-13268803 ] Lars Hofhansl commented on HDFS-744: As Dhruba said, we need this for HBase. One soluti

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017633#comment-13017633 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - > ... Could we do it in a separat

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017598#comment-13017598 ] Hairong Kuang commented on HDFS-744: @Linyin, I am ok if we simply sync every packet to

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017010#comment-13017010 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - Hi Hairong, We also don't have a

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016873#comment-13016873 ] Liyin Tang commented on HDFS-744: - Since we are now using new Create API to force sync, ther

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016683#comment-13016683 ] stack commented on HDFS-744: @Hairong Flag on create sounds right. > Support hsync in HDFS > --

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016678#comment-13016678 ] dhruba borthakur commented on HDFS-744: --- @Nicholas: yeah, machines were hard-rebooted

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016652#comment-13016652 ] Tsz Wo (Nicholas), SZE commented on HDFS-744: - Hi Dhruba, how the datanode power

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016626#comment-13016626 ] Hairong Kuang commented on HDFS-744: I thought more about this. It seems to me it is not

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016259#comment-13016259 ] Liyin Tang commented on HDFS-744: - Shall we do this for avatar client as well? > Support hs

[jira] [Commented] (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016258#comment-13016258 ] dhruba borthakur commented on HDFS-744: --- We need this functionality for HBase clusters

[jira] Commented: (HDFS-744) Support hsync in HDFS

[ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772431#action_12772431 ] dhruba borthakur commented on HDFS-744: --- do you plan this now or is this a placeholder