[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13427476#comment-13427476 ] Eli Collins commented on HDFS-3697: --- Agree with Brandon that this seems reasonable for branch-1, I'll merge the patch. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13427485#comment-13427485 ] Eli Collins commented on HDFS-3697: --- +1 to Todd's branch-1 patch. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422767#comment-13422767 ] Suresh Srinivas commented on HDFS-3697: --- Todd, I think we should add appropriate release notes, even though this is no incompatible, to call out attention to enabling this feature by default. BTW this is a good change! Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421374#comment-13421374 ] Hudson commented on HDFS-3697: -- Integrated in Hadoop-Hdfs-trunk #1114 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1114/]) HDFS-3697. Enable fadvise readahead by default. Contributed by Todd Lipcon. (Revision 1364698) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1364698 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421430#comment-13421430 ] Hudson commented on HDFS-3697: -- Integrated in Hadoop-Mapreduce-trunk #1146 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1146/]) HDFS-3697. Enable fadvise readahead by default. Contributed by Todd Lipcon. (Revision 1364698) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1364698 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420763#comment-13420763 ] Hudson commented on HDFS-3697: -- Integrated in Hadoop-Hdfs-trunk-Commit #2576 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2576/]) HDFS-3697. Enable fadvise readahead by default. Contributed by Todd Lipcon. (Revision 1364698) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1364698 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420764#comment-13420764 ] Hudson commented on HDFS-3697: -- Integrated in Hadoop-Common-trunk-Commit #2511 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2511/]) HDFS-3697. Enable fadvise readahead by default. Contributed by Todd Lipcon. (Revision 1364698) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1364698 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420768#comment-13420768 ] Hadoop QA commented on HDFS-3697: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537579/hdfs-3697-branch-1.txt against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2887//console This message is automatically generated. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420792#comment-13420792 ] Brandon Li commented on HDFS-3697: -- Do we have any good way to enable/disable readahead, drop cache and sync without restarting namenode? Just thinking when the workload changes, the customer has to restart the namenode to take advantage of the performance enhancement. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420793#comment-13420793 ] Todd Lipcon commented on HDFS-3697: --- You mean the datanode, right? It would be worth considering adding these as per-stream flags - e.g an HDFS fadvise-like API on DFSInputStream/DFSOutputStream that changes the behavior. Then, the datanode-wide flags would just serve as defaults. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420796#comment-13420796 ] Brandon Li commented on HDFS-3697: -- [quote]You mean the datanode, right?[quote] ops, yes. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420799#comment-13420799 ] Hudson commented on HDFS-3697: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2532 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2532/]) HDFS-3697. Enable fadvise readahead by default. Contributed by Todd Lipcon. (Revision 1364698) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1364698 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421051#comment-13421051 ] Todd Lipcon commented on HDFS-3697: --- Hey Brandon. In the absence of the improvement discussed above, do you think we should commit to branch-1? I ask, given you've been doing the other fadvise backports into branch-1. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421075#comment-13421075 ] Brandon Li commented on HDFS-3697: -- Hey Todd, I've seen some performance improvement in my branch-1 test with the fadvise support enabled. I think should commit it to branch-1. The thing I am not sure is the readahead step size since I didn't try out with enough workloads and step sizes in my tests. Given 4MB is good in your experiments and it's also configurable, we can start with it. I just feel it's not trivial sometimes to guess out a good step size beforehand. :-) If it's possible in the future, we might want to make readahead adjust its step size based on observed access pattern(however, tracking multiple parallel streams' access pattern can be same challenging :-( ). Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3697-branch-1.txt, hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419474#comment-13419474 ] Hadoop QA commented on HDFS-3697: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537392/hdfs-3697.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2879//console This message is automatically generated. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable them by default in future versions so that users get this benefit without any manual configuration required. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419539#comment-13419539 ] Eli Collins commented on HDFS-3697: --- +1 pending jenkins. It's worth calling out in the description that this only enables readahead, not drop cache and sync. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable them by default in future versions so that users get this benefit without any manual configuration required. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3697) Enable fadvise readahead by default
[ https://issues.apache.org/jira/browse/HDFS-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419664#comment-13419664 ] Hadoop QA commented on HDFS-3697: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537405/hdfs-3697.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2880//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2880//console This message is automatically generated. Enable fadvise readahead by default --- Key: HDFS-3697 URL: https://issues.apache.org/jira/browse/HDFS-3697 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, performance Affects Versions: 3.0.0, 2.2.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3697.txt, hdfs-3697.txt The fadvise features have been implemented for some time, and we've enabled them in production at a lot of customer sites without difficulty. I'd like to enable the readahead feature by default in future versions so that users get this benefit without any manual configuration required. The other fadvise features seem to be more workload-dependent and need further testing before enabling by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
