[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881830#comment-13881830 ] Hudson commented on HDFS-4949: -- FAILURE: Integrated in Hadoop-Yarn-trunk #461 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881857#comment-13881857 ] Hudson commented on HDFS-4949: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1678 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881875#comment-13881875 ] Hudson commented on HDFS-4949: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1653 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879143#comment-13879143 ] Colin Patrick McCabe commented on HDFS-4949: branch-2 patch looks good to me;

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-22 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879198#comment-13879198 ] Andrew Wang commented on HDFS-4949: --- Thanks Colin. The branch-2 test run I did with this

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-22 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879235#comment-13879235 ] Andrew Wang commented on HDFS-4949: --- I've committed this to branch-2 and fixed up the

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-22 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879301#comment-13879301 ] Hudson commented on HDFS-4949: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5035 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2014-01-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877995#comment-13877995 ] Hadoop QA commented on HDFS-4949: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-12-23 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13855882#comment-13855882 ] Arun C Murthy commented on HDFS-4949: - We've been discussing through a proposal

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-12-03 Thread Fengdong Yu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13838467#comment-13838467 ] Fengdong Yu commented on HDFS-4949: --- [~cnauroth], yes, I can find CacheManager in the

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-12-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13838471#comment-13838471 ] Andrew Wang commented on HDFS-4949: --- Hey [~azuryy], since the merge, we've been

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-12-03 Thread Fengdong Yu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13838481#comment-13838481 ] Fengdong Yu commented on HDFS-4949: --- Thanks, Andrew. Centralized cache management in

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-12-02 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836687#comment-13836687 ] Chris Nauroth commented on HDFS-4949: - Hi, [~azuryy]. The merge vote passed and

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-12-01 Thread Fengdong Yu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13836321#comment-13836321 ] Fengdong Yu commented on HDFS-4949: --- Vote thread has been started for a month, but

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-29 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807867#comment-13807867 ] Hudson commented on HDFS-4949: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #377 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-29 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807887#comment-13807887 ] Hudson commented on HDFS-4949: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1567 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-29 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807952#comment-13807952 ] Hudson commented on HDFS-4949: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1593 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-28 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807135#comment-13807135 ] Chris Nauroth commented on HDFS-4949: - +1 for the merge. Thanks again, Andrew and

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-28 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807547#comment-13807547 ] Hudson commented on HDFS-4949: -- SUCCESS: Integrated in Hadoop-trunk-Commit #4664 (See

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-25 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805435#comment-13805435 ] Suresh Srinivas commented on HDFS-4949: --- Given that the merge vote thread has been

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-25 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805463#comment-13805463 ] Chris Nauroth commented on HDFS-4949: - Here is a list of items discussed in the design

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805791#comment-13805791 ] Colin Patrick McCabe commented on HDFS-4949: quick note: The reason why

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804816#comment-13804816 ] Hadoop QA commented on HDFS-4949: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-24 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804828#comment-13804828 ] Chris Nauroth commented on HDFS-4949: - Thank you, Colin and Stephen. The design doc

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804840#comment-13804840 ] Hadoop QA commented on HDFS-4949: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-10-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805021#comment-13805021 ] Hadoop QA commented on HDFS-4949: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-14 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740145#comment-13740145 ] Andrew Wang commented on HDFS-4949: --- Hi Arun, On the read path comments, it might be

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-10 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736019#comment-13736019 ] Arun C Murthy commented on HDFS-4949: - bq. 1. The main reason we added auto-caching of

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-10 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736020#comment-13736020 ] Arun C Murthy commented on HDFS-4949: - bq. Tying in YARN would definitely be great.

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-09 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13734926#comment-13734926 ] Andrew Wang commented on HDFS-4949: --- Hi Tsuyoshi, HDFS-4953 allows applications to do

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-09 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735382#comment-13735382 ] Suresh Srinivas commented on HDFS-4949: --- bq. As a meta-point, I think much of the

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-09 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735632#comment-13735632 ] Arun C Murthy commented on HDFS-4949: - bq. As a meta-point, I think much of the

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-09 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735643#comment-13735643 ] Arun C Murthy commented on HDFS-4949: - [~andrew.wang] overall it's looks great, some

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-09 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735723#comment-13735723 ] Andrew Wang commented on HDFS-4949: --- Hey Arun, thanks for taking a look! Tying in YARN

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731077#comment-13731077 ] Colin Patrick McCabe commented on HDFS-4949: I created a branch for this

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-06 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731143#comment-13731143 ] Suresh Srinivas commented on HDFS-4949: --- My notes from the meeting: Enabling this

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-06 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731158#comment-13731158 ] Chris Nauroth commented on HDFS-4949: - bq. mlock there is no windows equivalent? I

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-08-06 Thread Tsuyoshi OZAWA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731632#comment-13731632 ] Tsuyoshi OZAWA commented on HDFS-4949: -- Hi, Are there any plan to add APIs to access

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-31 Thread David S. Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725704#comment-13725704 ] David S. Wang commented on HDFS-4949: - HDFS-4949 meeting, July 29, 2013 2 PM @

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-29 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723306#comment-13723306 ] Suresh Srinivas commented on HDFS-4949: --- As discussed in the comments earlier, few of

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-18 Thread Sanjay Radia (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13712529#comment-13712529 ] Sanjay Radia commented on HDFS-4949: Caching partial blocks: There is no problem with a

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13712579#comment-13712579 ] Colin Patrick McCabe commented on HDFS-4949: As Todd, Andrew, and I said

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-18 Thread Sanjay Radia (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13712856#comment-13712856 ] Sanjay Radia commented on HDFS-4949: bq. we have to use mmap of a file on disk. Please

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-18 Thread Sanjay Radia (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713117#comment-13713117 ] Sanjay Radia commented on HDFS-4949: To converge on this could we do a meetup?

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713327#comment-13713327 ] Colin Patrick McCabe commented on HDFS-4949: A meetup is a good idea. I will

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-16 Thread Chu Tong (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709911#comment-13709911 ] Chu Tong commented on HDFS-4949: To maximize memory usage, should we consider to compress

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710097#comment-13710097 ] Colin Patrick McCabe commented on HDFS-4949: For most of the applications we're

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Sanjay Radia (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707288#comment-13707288 ] Sanjay Radia commented on HDFS-4949: I think we can treat the RAM copies as replicas -

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707389#comment-13707389 ] Colin Patrick McCabe commented on HDFS-4949: I agree that there are some

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707467#comment-13707467 ] Andrew Wang commented on HDFS-4949: --- Sanjay, thanks for your comments! I need to look

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707536#comment-13707536 ] Suresh Srinivas commented on HDFS-4949: --- bq. However, tiered storage management

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707546#comment-13707546 ] Todd Lipcon commented on HDFS-4949: --- Hey folks. I agree that HSM is a much bigger task

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707572#comment-13707572 ] Suresh Srinivas commented on HDFS-4949: --- The problem for partical cache is though,

[jira] [Commented] (HDFS-4949) Centralized cache management in HDFS

2013-07-12 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707576#comment-13707576 ] Todd Lipcon commented on HDFS-4949: --- Yes, I think the explicit caching policies would