[jira] [Commented] (HDFS-8999) Namenode need not wait for {{blockReceived}} for the last block before completing a file.

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116636#comment-15116636 ] Hadoop QA commented on HDFS-8999: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread GAO Rui (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116748#comment-15116748 ] GAO Rui commented on HDFS-9494: --- [~szetszwo],[~rakeshr], thanks a lot for your advice. For

[jira] [Commented] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

2016-01-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116846#comment-15116846 ] Hudson commented on HDFS-8071: -- FAILURE: Integrated in Hadoop-trunk-Commit #9186 (See

[jira] [Commented] (HDFS-9690) addBlock is not idempotent

2016-01-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116847#comment-15116847 ] Hudson commented on HDFS-9690: -- FAILURE: Integrated in Hadoop-trunk-Commit #9186 (See

[jira] [Created] (HDFS-9704) terminate progress after namenode recover finished

2016-01-25 Thread Liao, Xiaoge (JIRA)
Liao, Xiaoge created HDFS-9704: -- Summary: terminate progress after namenode recover finished Key: HDFS-9704 URL: https://issues.apache.org/jira/browse/HDFS-9704 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116767#comment-15116767 ] Hadoop QA commented on HDFS-9494: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9503: Attachment: HDFS-9053.002.patch > Replace -namenode option with -fs for NNThroughputBenchmark >

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-01-25 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116671#comment-15116671 ] Sangjin Lee commented on HDFS-9579: --- I went over the existing version of the patch. First, I don't think

[jira] [Commented] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116830#comment-15116830 ] Xiao Chen commented on HDFS-9701: - Patch 2 fixes checkstyle, findbugs issues, and added some javadocs. For

[jira] [Updated] (HDFS-9690) addBlock is not idempotent

2016-01-25 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9690: -- Attachment: h9690_20160124b_branch-2.7.patch Thanks Vinay for reviewing and trying

[jira] [Commented] (HDFS-8999) Namenode need not wait for {{blockReceived}} for the last block before completing a file.

2016-01-25 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116845#comment-15116845 ] Tsz Wo Nicholas Sze commented on HDFS-8999: --- I have committed this to trunk. Will leave this

[jira] [Commented] (HDFS-9663) Optimize some RPC call using lighter weight construct than DatanodeInfo

2016-01-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116634#comment-15116634 ] Colin Patrick McCabe commented on HDFS-9663: Is that stuff actually sent over the wire in every

[jira] [Updated] (HDFS-9704) terminate progress after namenode recover finished

2016-01-25 Thread Liao, Xiaoge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liao, Xiaoge updated HDFS-9704: --- Attachment: HDFS-9704.001.patch > terminate progress after namenode recover finished >

[jira] [Commented] (HDFS-8999) Namenode need not wait for {{blockReceived}} for the last block before completing a file.

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116762#comment-15116762 ] Hadoop QA commented on HDFS-8999: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9655) NN should start JVM pause monitor before loading fsimage

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-9655: Labels: supportability (was: ) > NN should start JVM pause monitor before loading fsimage >

[jira] [Updated] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread GAO Rui (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GAO Rui updated HDFS-9494: -- Status: In Progress (was: Patch Available) > Parallel optimization of DFSStripedOutputStream#flushAllInternals(

[jira] [Updated] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread GAO Rui (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GAO Rui updated HDFS-9494: -- Status: Patch Available (was: In Progress) > Parallel optimization of DFSStripedOutputStream#flushAllInternals(

[jira] [Updated] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread GAO Rui (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GAO Rui updated HDFS-9494: -- Attachment: HDFS-9494-origin-trunk.05.patch > Parallel optimization of DFSStripedOutputStream#flushAllInternals(

[jira] [Commented] (HDFS-9690) addBlock is not idempotent

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116851#comment-15116851 ] Hadoop QA commented on HDFS-9690: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-7694) FSDataInputStream should support "unbuffer"

2016-01-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116622#comment-15116622 ] Colin Patrick McCabe commented on HDFS-7694: Hi, [~djp]. This change is compatible, since

[jira] [Commented] (HDFS-9663) Optimize some RPC call using lighter weight construct than DatanodeInfo

2016-01-25 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116701#comment-15116701 ] Kai Zheng commented on HDFS-9663: - Thanks for your comments, Colin. bq. Is that stuff actually sent over

[jira] [Updated] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9701: Attachment: HDFS-9701.02.patch > DN may deadlock when hot-swapping under load >

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-01-25 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116791#comment-15116791 ] Ming Ma commented on HDFS-9579: --- Thanks [~sjlee0]! Good point about the thread visibility issue. The reason I

[jira] [Updated] (HDFS-6489) DFS Used space is not correct computed on frequent append operations

2016-01-25 Thread Bogdan Raducanu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bogdan Raducanu updated HDFS-6489: -- Attachment: HDFS6489.java > DFS Used space is not correct computed on frequent append operations

[jira] [Updated] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-25 Thread Staffan Friberg (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Staffan Friberg updated HDFS-9260: -- Attachment: HDFS-9260.014.patch > Improve performance and GC friendliness of startup and FBRs >

[jira] [Updated] (HDFS-6489) DFS Used space is not correct computed on frequent append operations

2016-01-25 Thread Bogdan Raducanu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bogdan Raducanu updated HDFS-6489: -- Affects Version/s: 2.7.1 > DFS Used space is not correct computed on frequent append operations

[jira] [Commented] (HDFS-6489) DFS Used space is not correct computed on frequent append operations

2016-01-25 Thread Bogdan Raducanu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115206#comment-15115206 ] Bogdan Raducanu commented on HDFS-6489: --- I've recently hit this bug in 2.7.1. I attached repro code.

[jira] [Created] (HDFS-9695) HTTPFS - CHECKACCESS operation missing

2016-01-25 Thread Bert Hekman (JIRA)
Bert Hekman created HDFS-9695: - Summary: HTTPFS - CHECKACCESS operation missing Key: HDFS-9695 URL: https://issues.apache.org/jira/browse/HDFS-9695 Project: Hadoop HDFS Issue Type: Bug

[jira] [Comment Edited] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

2016-01-25 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115072#comment-15115072 ] Vinayakumar B edited comment on HDFS-9689 at 1/25/16 1:22 PM: -- One possible

[jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-25 Thread Staffan Friberg (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115356#comment-15115356 ] Staffan Friberg commented on HDFS-9260: --- Fixed checkstyle on TreeSet. Should I convert storages

[jira] [Created] (HDFS-9694) Make existing DFSClient#getFileChecksum() work for striped blocks

2016-01-25 Thread Kai Zheng (JIRA)
Kai Zheng created HDFS-9694: --- Summary: Make existing DFSClient#getFileChecksum() work for striped blocks Key: HDFS-9694 URL: https://issues.apache.org/jira/browse/HDFS-9694 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9693) Trim the user config of `dfs.ha.namenode.id`

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115244#comment-15115244 ] Hadoop QA commented on HDFS-9693: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8430) Erasure coding: compute file checksum for stripe files

2016-01-25 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115135#comment-15115135 ] Kai Zheng commented on HDFS-8430: - To break down, opened HDFS-9694 to make the existing API also works for

[jira] [Commented] (HDFS-9525) hadoop utilities need to support provided delegation tokens

2016-01-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115355#comment-15115355 ] Kihwal Lee commented on HDFS-9525: -- Is anyone reverting it or reworking on the fix? > hadoop utilities

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-25 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115780#comment-15115780 ] Jing Zhao commented on HDFS-9406: - Thanks for reporting the issue, [~stanislav.an...@gmail.com]. The

[jira] [Updated] (HDFS-9262) Support reconfiguring dfs.datanode.lazywriter.interval.sec without DN restart

2016-01-25 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-9262: Attachment: HDFS-9262-HDFS-9000.004.patch V004 fixed a large quantity of unit failures as a result

[jira] [Created] (HDFS-9699) libhdfs++: Add appropriate catch blocks for ASIO operations that throw

2016-01-25 Thread James Clampffer (JIRA)
James Clampffer created HDFS-9699: - Summary: libhdfs++: Add appropriate catch blocks for ASIO operations that throw Key: HDFS-9699 URL: https://issues.apache.org/jira/browse/HDFS-9699 Project: Hadoop

[jira] [Commented] (HDFS-9672) o.a.h.hdfs.TestLeaseRecovery2 fails intermittently

2016-01-25 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115862#comment-15115862 ] Jitendra Nath Pandey commented on HDFS-9672: +1 > o.a.h.hdfs.TestLeaseRecovery2 fails

[jira] [Commented] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115745#comment-15115745 ] Rakesh R commented on HDFS-9494: Thanks [~demongaorui] for the patch. I've a minor comment, please consider

[jira] [Created] (HDFS-9698) Long running Balancer should renew TGT

2016-01-25 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9698: --- Summary: Long running Balancer should renew TGT Key: HDFS-9698 URL: https://issues.apache.org/jira/browse/HDFS-9698 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115866#comment-15115866 ] Yongjun Zhang commented on HDFS-9406: - Thanks [~kihwal] and [~jingzhao]. Hi Jing, I have got a set

[jira] [Comment Edited] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115818#comment-15115818 ] Jing Zhao edited comment on HDFS-9696 at 1/25/16 7:32 PM: -- Currently I think

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115818#comment-15115818 ] Jing Zhao commented on HDFS-9696: - Currently I think HDFS-9406 and HDFS-9697 may both be caused by some

[jira] [Updated] (HDFS-9330) Support reconfiguring dfs.datanode.duplicate.replica.deletion without DN restart

2016-01-25 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-9330: Attachment: HDFS-9330-HDFS-9000.003.patch Similarly, V003 fixed a large quantity of unit failures as

[jira] [Commented] (HDFS-9691) o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode#testCheckSafeMode fails intermittently

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115776#comment-15115776 ] Mingliang Liu commented on HDFS-9691: - The failing test is not related, and seems flaky which is

[jira] [Updated] (HDFS-9698) Long running Balancer should renew TGT

2016-01-25 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9698: Attachment: HDFS-9698.00.patch A similar fix as HADOOP-12559, but in the {{Balancer}}. Adding the renewal

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115795#comment-15115795 ] Kihwal Lee commented on HDFS-9406: -- HDFS-9696 might be related. > FSImage corruption after taking

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115796#comment-15115796 ] Yongjun Zhang commented on HDFS-9696: - Thanks Kihwal. Yes, agree. While I have been investigating, I

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115884#comment-15115884 ] Yongjun Zhang commented on HDFS-9696: - And I have a solution for HDFS-9697, for the case I created. Yet

[jira] [Updated] (HDFS-9094) Add command line option to ask NameNode reload configuration.

2016-01-25 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-9094: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status: Resolved

[jira] [Commented] (HDFS-9094) Add command line option to ask NameNode reload configuration.

2016-01-25 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115906#comment-15115906 ] Hudson commented on HDFS-9094: -- FAILURE: Integrated in Hadoop-trunk-Commit #9180 (See

[jira] [Commented] (HDFS-9494) Parallel optimization of DFSStripedOutputStream#flushAllInternals( )

2016-01-25 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115531#comment-15115531 ] Tsz Wo Nicholas Sze commented on HDFS-9494: --- In the last for-loop, the finally-block will be

[jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115578#comment-15115578 ] Hadoop QA commented on HDFS-9260: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9118) Add logging system for libdhfs++

2016-01-25 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115421#comment-15115421 ] James Clampffer commented on HDFS-9118: --- I'm going to take a shot at this. Things I'm planning on

[jira] [Commented] (HDFS-9118) Add logging system for libdhfs++

2016-01-25 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115434#comment-15115434 ] Bob Hansen commented on HDFS-9118: -- Grabbing the stack address is a fairly expensive operation. I would

[jira] [Commented] (HDFS-9607) Advance Hadoop Architecture (AHA) - HDFS Update (write-in-place)

2016-01-25 Thread Dinesh S. Atreya (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115548#comment-15115548 ] Dinesh S. Atreya commented on HDFS-9607: Please see updated API from above below:

[jira] [Commented] (HDFS-9607) Advance Hadoop Architecture (AHA) - HDFS Update (write-in-place)

2016-01-25 Thread Dinesh S. Atreya (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115577#comment-15115577 ] Dinesh S. Atreya commented on HDFS-9607: Alternatively, assuming {{getPos()}} and {{seek}} as given

[jira] [Created] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-9696: Summary: Garbage snapshot records lingering forever Key: HDFS-9696 URL: https://issues.apache.org/jira/browse/HDFS-9696 Project: Hadoop HDFS Issue Type: Bug

[jira] [Assigned] (HDFS-9118) Add logging system for libdhfs++

2016-01-25 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Clampffer reassigned HDFS-9118: - Assignee: James Clampffer > Add logging system for libdhfs++ >

[jira] [Commented] (HDFS-9607) Advance Hadoop Architecture (AHA) - HDFS Update (write-in-place)

2016-01-25 Thread Dinesh S. Atreya (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115572#comment-15115572 ] Dinesh S. Atreya commented on HDFS-9607: Correction {code} int write(long position, byte[]

[jira] [Commented] (HDFS-9690) addBlock is not idempotent

2016-01-25 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115568#comment-15115568 ] Vinayakumar B commented on HDFS-9690: - Tried committing, patch doesnt apply to branch-2.7 as there is

[jira] [Commented] (HDFS-9684) DataNode stopped sending heartbeat after getting OutOfMemoryError form DataTransfer thread.

2016-01-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115523#comment-15115523 ] Kihwal Lee commented on HDFS-9684: -- That usually means the ulimit is reached. What is the max Xceiver

[jira] [Updated] (HDFS-9690) addBlock is not idempotent

2016-01-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9690: - Target Version/s: 2.7.3 > addBlock is not idempotent > -- > > Key:

[jira] [Assigned] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang reassigned HDFS-9696: --- Assignee: Yongjun Zhang Hi Kihwal, Since I am working on, do you mind if I assign it to

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115754#comment-15115754 ] Kihwal Lee commented on HDFS-9696: -- bq. do you mind if I assign it to myself? I don't. But I noticed that

[jira] [Updated] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-9697: Summary: NN fails to restart due to corrupt fsimage caused by snapshot handling (was: NN fails to

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115595#comment-15115595 ] Kihwal Lee commented on HDFS-9696: -- {code:xml} 0 ... 1638543008443-10action-data.seq

[jira] [Created] (HDFS-9697) NN fails to restart due to corrupt fsimage

2016-01-25 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-9697: --- Summary: NN fails to restart due to corrupt fsimage Key: HDFS-9697 URL: https://issues.apache.org/jira/browse/HDFS-9697 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115690#comment-15115690 ] Yongjun Zhang commented on HDFS-9696: - Hi [~kihwal], Thanks much for reporting this issue. I have been

[jira] [Commented] (HDFS-8999) Namenode need not wait for {{blockReceived}} for the last block before completing a file.

2016-01-25 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115710#comment-15115710 ] Jing Zhao commented on HDFS-8999: - Thanks for verifying the test failures, Nicholas! +1 committing the

[jira] [Updated] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-25 Thread Gary Helmling (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HDFS-9700: Attachment: HDFS-9700_branch-2.7.patch The attached patch is against branch-2.7. For an HBase

[jira] [Commented] (HDFS-9669) TcpPeerServer should respect ipc.server.listen.queue.size

2016-01-25 Thread Elliott Clark (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116012#comment-15116012 ] Elliott Clark commented on HDFS-9669: - Ping? This is running in production and removes thousands of

[jira] [Commented] (HDFS-9118) Add logging system for libdhfs++

2016-01-25 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116065#comment-15116065 ] James Clampffer commented on HDFS-9118: --- Shouldn't getting the stack address of a local variable boil

[jira] [Updated] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-25 Thread Gary Helmling (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HDFS-9700: Attachment: HDFS-9700-v1.patch Attaching a patch for the same changes against trunk. > DFSClient

[jira] [Updated] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-25 Thread Gary Helmling (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HDFS-9700: Status: Patch Available (was: Open) > DFSClient and DFSOutputStream do not respect TCP_NODELAY

[jira] [Created] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-9701: --- Summary: DN may deadlock when hot-swapping under load Key: HDFS-9701 URL: https://issues.apache.org/jira/browse/HDFS-9701 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116161#comment-15116161 ] Hadoop QA commented on HDFS-9700: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116149#comment-15116149 ] Xiao Chen commented on HDFS-9701: - Most notable jstacks: Reconfigure task: {noformat} "Reconfiguration

[jira] [Commented] (HDFS-9330) Support reconfiguring dfs.datanode.duplicate.replica.deletion without DN restart

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116162#comment-15116162 ] Hadoop QA commented on HDFS-9330: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-9702) DiskBalancer : getVolumeMap implementation

2016-01-25 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-9702: -- Summary: DiskBalancer : getVolumeMap implementation Key: HDFS-9702 URL: https://issues.apache.org/jira/browse/HDFS-9702 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-9702) DiskBalancer : getVolumeMap implementation

2016-01-25 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9702: --- Attachment: HDFS-9702-HDFS-1312.001.patch Adding patch for code review. This is dependent on HDFS-9683

[jira] [Updated] (HDFS-9703) DiskBalancer : getBandwidth implementation

2016-01-25 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9703: --- Attachment: HDFS-9703-HDFS-1312.001.patch Adding patch for code review. This is dependent on HDFS-9702

[jira] [Commented] (HDFS-9262) Support reconfiguring dfs.datanode.lazywriter.interval.sec without DN restart

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116176#comment-15116176 ] Hadoop QA commented on HDFS-9262: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9525) hadoop utilities need to support provided delegation tokens

2016-01-25 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116289#comment-15116289 ] Owen O'Malley commented on HDFS-9525: - [~daryn] I'm sorry, but I don't see what problem the patch

[jira] [Updated] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9701: Attachment: HDFS-9701.01.patch > DN may deadlock when hot-swapping under load >

[jira] [Updated] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9701: Status: Patch Available (was: Open) > DN may deadlock when hot-swapping under load >

[jira] [Commented] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116165#comment-15116165 ] Xiao Chen commented on HDFS-9701: - Thanks a lot to [~eddyxu] for the offline discussion about the general

[jira] [Commented] (HDFS-9698) Long running Balancer should renew TGT

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116184#comment-15116184 ] Hadoop QA commented on HDFS-9698: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-9703) DiskBalancer : getBandwidth implementation

2016-01-25 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-9703: -- Summary: DiskBalancer : getBandwidth implementation Key: HDFS-9703 URL: https://issues.apache.org/jira/browse/HDFS-9703 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-9476) TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115775#comment-15115775 ] Mingliang Liu commented on HDFS-9476: - It happens in recent build as well, see UT log at:

[jira] [Updated] (HDFS-9698) Long running Balancer should renew TGT

2016-01-25 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9698: Status: Patch Available (was: Open) > Long running Balancer should renew TGT >

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115802#comment-15115802 ] Yongjun Zhang commented on HDFS-9696: - Ah, I intended to write the request message in my prior comment

[jira] [Commented] (HDFS-9689) Test o.a.h.hdfs.TestRenameWhileOpen fails intermittently

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115838#comment-15115838 ] Mingliang Liu commented on HDFS-9689: - Thanks [~iwasakims] and [~vinayrpet] for your insightful

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-01-25 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15115879#comment-15115879 ] Yongjun Zhang commented on HDFS-9696: - Yes [~jingzhao], your analysis is correct to me per my study in

[jira] [Created] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-25 Thread Gary Helmling (JIRA)
Gary Helmling created HDFS-9700: --- Summary: DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots Key: HDFS-9700 URL: https://issues.apache.org/jira/browse/HDFS-9700 Project:

[jira] [Commented] (HDFS-9672) o.a.h.hdfs.TestLeaseRecovery2 fails intermittently

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116453#comment-15116453 ] Mingliang Liu commented on HDFS-9672: - Thanks for the discussion, review and commit, [~jnp]! >

[jira] [Commented] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116466#comment-15116466 ] Hadoop QA commented on HDFS-9701: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9579) Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level

2016-01-25 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116490#comment-15116490 ] Sangjin Lee commented on HDFS-9579: --- [~mingma], the patch no longer applies cleanly. Do you mind updating

[jira] [Commented] (HDFS-9664) TestRollingUpgrade.testRollback failed frequently

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116320#comment-15116320 ] Mingliang Liu commented on HDFS-9664: - Perhaps there is a fundamental problem in the rolling upgrade.

[jira] [Commented] (HDFS-9677) Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp

2016-01-25 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15116366#comment-15116366 ] Mingliang Liu commented on HDFS-9677: - The v1 patch basically renames the {{generationStampV1}} =>

  1   2   >