[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-04-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13965839#comment-13965839 ] Chris Nauroth commented on HDFS-5776: - FYI, I've discovered that the {{DFSClient}} can

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911477#comment-13911477 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #492 (See

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911577#comment-13911577 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1684 (See

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911633#comment-13911633 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1709 (See

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-24 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13910935#comment-13910935 ] Hudson commented on HDFS-5776: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5216 (See

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907451#comment-13907451 ] stack commented on HDFS-5776: - Thanks for the numbers [~xieliang007] I ran some loadings

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907863#comment-13907863 ] Liang Xie commented on HDFS-5776: - bq. little discernible overall difference in spite of my

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906242#comment-13906242 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-19 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906611#comment-13906611 ] Liang Xie commented on HDFS-5776: - Huge thanks, [~saint@gmail.com] ! Here is the YCSB

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13901843#comment-13901843 ] Jing Zhao commented on HDFS-5776: - Thanks for updating the patch [~stack]. The latest patch

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-14 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13902288#comment-13902288 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-13 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13900133#comment-13900133 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-12 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13899961#comment-13899961 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896288#comment-13896288 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895423#comment-13895423 ] Liang Xie commented on HDFS-5776: - I am from holiday now, will try to catch up the above

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895441#comment-13895441 ] Liang Xie commented on HDFS-5776: - Thank you, Michael! Support 'hedged' reads in

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-07 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895461#comment-13895461 ] Liang Xie commented on HDFS-5776: - We need to have a NPE check here since the chosenNode be

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-02-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891912#comment-13891912 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-30 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887039#comment-13887039 ] stack commented on HDFS-5776: - [~jingzhao] Thanks for taking the time to look and the great

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-30 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13887329#comment-13887329 ] Liang Xie commented on HDFS-5776: - [~saint@gmail.com], yeh, i need your help, thanks!

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13885512#comment-13885512 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13885813#comment-13885813 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886072#comment-13886072 ] Jing Zhao commented on HDFS-5776: - Thanks for updating the patch, [~xieliang007] and

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886107#comment-13886107 ] stack commented on HDFS-5776: - [~jingzhao] Thanks for the new input. Please help me better

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886142#comment-13886142 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-29 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13886172#comment-13886172 ] Jing Zhao commented on HDFS-5776: - Thanks for the feedback [~stack]. My first question

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883928#comment-13883928 ] Jing Zhao commented on HDFS-5776: - bq. it's more flexible if we provide instance level

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883940#comment-13883940 ] Jing Zhao commented on HDFS-5776: - Another thing for enoughNodesForHedgedRead. The current

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883944#comment-13883944 ] Liang Xie commented on HDFS-5776: - Could we create another JIRA to track those

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13884351#comment-13884351 ] Arpit Agarwal commented on HDFS-5776: - [~stack] I am basically +1 on the v9 patch at

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13884361#comment-13884361 ] Suresh Srinivas commented on HDFS-5776: --- bq. Could we create another JIRA to track

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13884554#comment-13884554 ] Colin Patrick McCabe commented on HDFS-5776: [~arpitagarwal] : if I understand

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13884573#comment-13884573 ] Suresh Srinivas commented on HDFS-5776: --- bq. We do not check other configuration

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13884694#comment-13884694 ] stack commented on HDFS-5776: - Thanks lads. We are almost there. [~xieliang007] It is better

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13884710#comment-13884710 ] Arpit Agarwal commented on HDFS-5776: - I've stated my concerns but if there is broad

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13885083#comment-13885083 ] Liang Xie commented on HDFS-5776: - bq. we do not check threadpool in enableHedgedReads.

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13885091#comment-13885091 ] Liang Xie commented on HDFS-5776: - bq. I think a better way is to add this check in

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13885092#comment-13885092 ] stack commented on HDFS-5776: - bq. what do you think ? That looks good to me [~xieliang007]

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882655#comment-13882655 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883079#comment-13883079 ] Colin Patrick McCabe commented on HDFS-5776: bq. Fine, let me introduce a hard

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883666#comment-13883666 ] Jing Zhao commented on HDFS-5776: - Thanks for the work [~xieliang007]! I will review your

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883685#comment-13883685 ] Arpit Agarwal commented on HDFS-5776: - {quote} Yes, that would be perfect sometimes,

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883717#comment-13883717 ] stack commented on HDFS-5776: - [~arpitagarwal] Would v10 be palatable? You say OK to v9 above

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883746#comment-13883746 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-27 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13883823#comment-13883823 ] Liang Xie commented on HDFS-5776: - bq. Isn't the call to actualGetFromOneDataNode wrapped

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-26 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882606#comment-13882606 ] Liang Xie commented on HDFS-5776: - [~saint@gmail.com] bq. If we are too often running

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881827#comment-13881827 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882037#comment-13882037 ] stack commented on HDFS-5776: - bq. I might be misunderstanding, but it seems like this should

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881281#comment-13881281 ] stack commented on HDFS-5776: - So, to enable, we set DFS_DFSCLIENT_HEDGED_READ_THREADPOOL_SIZE

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881305#comment-13881305 ] Arpit Agarwal commented on HDFS-5776: - {quote} I don't see why we wouldn't expose this

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881341#comment-13881341 ] Arpit Agarwal commented on HDFS-5776: - I reviewed the v8 patch. The implementation of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881661#comment-13881661 ] Colin Patrick McCabe commented on HDFS-5776: I might be misunderstanding, but

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879680#comment-13879680 ] Liang Xie commented on HDFS-5776: - [~arpitagarwal], thanks for your nice review! bq. The

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880279#comment-13880279 ] Jing Zhao commented on HDFS-5776: - # In DFSClient, I agree with Arpit that we should remove

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880280#comment-13880280 ] Colin Patrick McCabe commented on HDFS-5776: bq. One note on 1.. Is static ever

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880284#comment-13880284 ] Colin Patrick McCabe commented on HDFS-5776: By the way, my previous comment

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880746#comment-13880746 ] stack commented on HDFS-5776: - bq. An alternative way to do this is to have an

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880757#comment-13880757 ] stack commented on HDFS-5776: - [~xieliang007] what you think of the new comments above by the

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880762#comment-13880762 ] Liang Xie commented on HDFS-5776: - bq. Can this scenario be possible? In

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-23 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880783#comment-13880783 ] Liang Xie commented on HDFS-5776: - v8 add the enoughNodesForHedgedRead() function to sanity

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878459#comment-13878459 ] Liang Xie commented on HDFS-5776: - Added a bit codes into YCSB to get 99.9thPercentile

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878485#comment-13878485 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878997#comment-13878997 ] Enis Soztutar commented on HDFS-5776: - bq. About naming, personally i'd like to keep

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879324#comment-13879324 ] Colin Patrick McCabe commented on HDFS-5776: I like the idea of calling this

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879326#comment-13879326 ] Arpit Agarwal commented on HDFS-5776: - Hi Liang, thanks for this contribution to HDFS!

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879476#comment-13879476 ] stack commented on HDFS-5776: - [~arpitagarwal] Great review (from a bystander). One note on

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877344#comment-13877344 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877996#comment-13877996 ] Enis Soztutar commented on HDFS-5776: - Nice work Liang. You have beat us to implement

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878028#comment-13878028 ] stack commented on HDFS-5776: - (Trying to help out w/ naming...) [~enis] You don't like

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878184#comment-13878184 ] Liang Xie commented on HDFS-5776: - [~enis], thanks for your nice comments. bq. sleeping is

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-21 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878382#comment-13878382 ] Liang Xie commented on HDFS-5776: - About naming, personally i'd like to keep hedged

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13876388#comment-13876388 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13876733#comment-13876733 ] stack commented on HDFS-5776: - Nice numbers [~xieliang007] When you get a chance, there were a

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13876777#comment-13876777 ] Colin Patrick McCabe commented on HDFS-5776: Let's keep TestPread as a test of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13876802#comment-13876802 ] stack commented on HDFS-5776: - bq. I think that when the data being read is local, you will not

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13876842#comment-13876842 ] Colin Patrick McCabe commented on HDFS-5776: bq. Having hedged read on could

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877145#comment-13877145 ] Liang Xie commented on HDFS-5776: - [~saint@gmail.com], [~cmccabe], hedged reads

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877146#comment-13877146 ] Liang Xie commented on HDFS-5776: - bq. I guess we probably don't need to special-case local

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877154#comment-13877154 ] Liang Xie commented on HDFS-5776: - bq. Let's keep TestPread as a test of just pread, and

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877161#comment-13877161 ] Liang Xie commented on HDFS-5776: - bq. Nit: This seems extraneous in the new Callable: +

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877173#comment-13877173 ] Liang Xie commented on HDFS-5776: - {quote} Nit: Should the '60' below here: +

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877177#comment-13877177 ] Liang Xie commented on HDFS-5776: - bq. Shut down this executor in the finally? Don't want

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877231#comment-13877231 ] Liang Xie commented on HDFS-5776: - {quote} getHedgedReadsThreadPool Should be kept internal

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877255#comment-13877255 ] Liang Xie commented on HDFS-5776: - bq. For sure this will always trigger though the number

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877263#comment-13877263 ] Liang Xie commented on HDFS-5776: - bq. When we get to the end of the pipeline here; i.e.

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-20 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877268#comment-13877268 ] Liang Xie commented on HDFS-5776: - bq. I suppose moving this into finally would be messier

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874620#comment-13874620 ] Hadoop QA commented on HDFS-5776: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874818#comment-13874818 ] Liang Xie commented on HDFS-5776: - all are green now, nice:) Support 'hedged' reads in

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875161#comment-13875161 ] stack commented on HDFS-5776: - [~xieliang007] Nit: The 'return null' necessary in below given

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875453#comment-13875453 ] stack commented on HDFS-5776: - This looks like copy paste issue: + // assert that there

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-16 Thread Liang Xie (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13873136#comment-13873136 ] Liang Xie commented on HDFS-5776: - mvn clean test -Dtest=TestPread passed locally:)

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13873239#comment-13873239 ] Hadoop QA commented on HDFS-5776: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient

2014-01-15 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13872402#comment-13872402 ] stack commented on HDFS-5776: - I had a go at the subject for you [~xieliang007] I am a native