[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Status: Patch Available (was: In Progress) > Use TransmitFile for file to socket data

[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Component/s: datanode > Use TransmitFile for file to socket data transfer >

[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Labels: NIO Windows datanode (was: ) > Use TransmitFile for file to socket data transfer >

[jira] [Work started] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15151 started by Lukas Majercak. - > Use TransmitFile for file to socket data transfer >

[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Affects Version/s: 3.4.0 3.3.1 3.2.2

[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Description: Proposing to give an option to use TransmitFile Windows function for file to

[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Description: Proposing to give an option to use TransmitFile > Use TransmitFile for file to

[jira] [Updated] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15151: -- Description: Proposing to give an option to use TransmitFile Windows function for file to

[jira] [Created] (HDFS-15151) Use TransmitFile for file to socket data transfer

2020-01-30 Thread Lukas Majercak (Jira)
Lukas Majercak created HDFS-15151: - Summary: Use TransmitFile for file to socket data transfer Key: HDFS-15151 URL: https://issues.apache.org/jira/browse/HDFS-15151 Project: Hadoop HDFS

[jira] [Updated] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15055: -- Priority: Minor (was: Major) > Hedging clones client's buffer >

[jira] [Reopened] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak reopened HDFS-15055: --- > Hedging clones client's buffer > -- > > Key:

[jira] [Commented] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995256#comment-16995256 ] Lukas Majercak commented on HDFS-15055: --- Although I feel like this still could be an issue.

[jira] [Commented] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995254#comment-16995254 ] Lukas Majercak commented on HDFS-15055: --- Closing as we actually create a separate buffer for the

[jira] [Resolved] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak resolved HDFS-15055. --- Resolution: Not A Problem > Hedging clones client's buffer > --

[jira] [Updated] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15055: -- Description: Currently, DFSInputStream clones the buffer passed from the caller for every

[jira] [Updated] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15055: -- Priority: Major (was: Critical) > Hedging clones client's buffer >

[jira] [Updated] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15055: -- Description: Currently, DFSInputStream clones the buffer passed from the caller for every

[jira] [Updated] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-15055: -- Description: _emphasized text_ > Hedging clones client's buffer >

[jira] [Created] (HDFS-15055) Hedging clones client's buffer

2019-12-12 Thread Lukas Majercak (Jira)
Lukas Majercak created HDFS-15055: - Summary: Hedging clones client's buffer Key: HDFS-15055 URL: https://issues.apache.org/jira/browse/HDFS-15055 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-14882) Consider DataNode load when #getBlockLocation

2019-09-30 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941143#comment-16941143 ] Lukas Majercak commented on HDFS-14882: --- Overall looks okay to me, seems like an improvement of

[jira] [Commented] (HDFS-12288) Fix DataNode's xceiver count calculation

2019-09-09 Thread Lukas Majercak (Jira)
[ https://issues.apache.org/jira/browse/HDFS-12288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925976#comment-16925976 ] Lukas Majercak commented on HDFS-12288: --- [~zhangchen] not working on this right now, feel free to

[jira] [Commented] (HDFS-14545) RBF: Router should support GetUserMappingsProtocol

2019-06-14 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863822#comment-16863822 ] Lukas Majercak commented on HDFS-14545: --- Thanks [~ayushtkn], LGTM > RBF: Router should support

[jira] [Commented] (HDFS-14545) RBF: Router should support GetUserMappingsProtocol

2019-06-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858912#comment-16858912 ] Lukas Majercak commented on HDFS-14545: --- ConnectionPool lines 410, 411. Would be nice to either

[jira] [Commented] (HDFS-14447) RBF: Router should support RefreshUserMappingsProtocol

2019-05-16 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841554#comment-16841554 ] Lukas Majercak commented on HDFS-14447: --- patch09 lgtm > RBF: Router should support

[jira] [Commented] (HDFS-14447) RBF: Router should support RefreshUserMappingsProtocol

2019-05-14 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839720#comment-16839720 ] Lukas Majercak commented on HDFS-14447: --- There are a couple of syntax inconsistencies in 06.patch,

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-05-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834936#comment-16834936 ] Lukas Majercak commented on HDFS-14134: --- Hi [~John Smith]. I'm not sure if it's that simple. Say

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-05-03 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832730#comment-16832730 ] Lukas Majercak commented on HDFS-14134: --- Hi [~John Smith]. I'm not sure I'm following, what's the

[jira] [Commented] (HDFS-14326) Add CorruptFilesCount to JMX

2019-02-28 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780939#comment-16780939 ] Lukas Majercak commented on HDFS-14326: --- Can we add more test coverage? We could just assert the

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-11 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740810#comment-16740810 ] Lukas Majercak commented on HDFS-14134: --- + more people [~szetszwo], [~jingzhao]. > Idempotent

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-11 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740802#comment-16740802 ] Lukas Majercak commented on HDFS-14134: --- Thanks [~knanasi]. [~atm], [~eli], [~sureshms],

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739821#comment-16739821 ] Lukas Majercak commented on HDFS-14134: --- Patch 007 to fix checkstyle + whitespace warnings. >

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.007.patch > Idempotent operations throwing RemoteException should not

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.006.patch > Idempotent operations throwing RemoteException should not

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739719#comment-16739719 ] Lukas Majercak commented on HDFS-14134: --- Added patch006 together with

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: (was: HDFS-14134.006.patch) > Idempotent operations throwing RemoteException

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134_retrypolicy_change_proposal_1.pdf > Idempotent operations throwing

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.006.patch > Idempotent operations throwing RemoteException should not

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-09 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738702#comment-16738702 ] Lukas Majercak commented on HDFS-14134: --- Also note that previously, if a hedging request got

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2019-01-09 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738700#comment-16738700 ] Lukas Majercak commented on HDFS-14134: --- I see, that makes sense, I'm happy to change

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-13 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720564#comment-16720564 ] Lukas Majercak commented on HDFS-14134: --- I'll go through that discussion. > Idempotent operations

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-13 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720506#comment-16720506 ] Lukas Majercak commented on HDFS-14134: --- I agree non-remote IOExceptions could be network related,

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-13 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720507#comment-16720507 ] Lukas Majercak commented on HDFS-14134: --- I'd argue that this change is even safer, because

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-12 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719444#comment-16719444 ] Lukas Majercak commented on HDFS-14134: --- Retrying failed idempotent operations might be safe, but

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-11 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717754#comment-16717754 ] Lukas Majercak commented on HDFS-14134: --- Why should we retry if the operation is idempotent and the

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.005.patch > Idempotent operations throwing RemoteException should not

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715863#comment-16715863 ] Lukas Majercak commented on HDFS-14134: --- Patch 005 to fix minor checkstyle issue in

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.004.patch > Idempotent operations throwing RemoteException should not

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715705#comment-16715705 ] Lukas Majercak commented on HDFS-14134: --- Added tests to cover all cases (SocketExc, IOException,

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: (was: HDFS-14134.003.patch) > Idempotent operations throwing RemoteException

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715682#comment-16715682 ] Lukas Majercak commented on HDFS-14134: --- Fixed TestFailoverProxy as well, still might need to add

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.003.patch > Idempotent operations throwing RemoteException should not

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.003.patch > Idempotent operations throwing RemoteException should not

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715579#comment-16715579 ] Lukas Majercak commented on HDFS-14134: --- I realized I needed to change the mock expectations to fix

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715545#comment-16715545 ] Lukas Majercak commented on HDFS-14134: --- Thanks for the review [~knanasi]. I've uploaded patch002

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.002.patch > Idempotent operations throwing RemoteException should not

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713440#comment-16713440 ] Lukas Majercak commented on HDFS-14134: --- The unit tests are expected to fail, I can fix them once

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713358#comment-16713358 ] Lukas Majercak commented on HDFS-14134: --- For the retry policy changes, maybe it would make sense to

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.001.patch > Idempotent operations throwing RemoteException should not

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: (was: HDFS-14134.001.patch) > Idempotent operations throwing RemoteException

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713326#comment-16713326 ] Lukas Majercak commented on HDFS-14134: --- Reuploaded the patch, because this guy Yetus took my pdf

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713314#comment-16713314 ] Lukas Majercak commented on HDFS-14134: --- Added HDFS-14134_retrypolicy_change_proposal.pdf to

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134_retrypolicy_change_proposal.pdf > Idempotent operations throwing

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.001.patch > Idempotent operations throwing RemoteException should not

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: HDFS-14134.001.patch > Idempotent operations throwing RemoteException should not

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Attachment: (was: HDFS-14134.001.patch) > Idempotent operations throwing RemoteException

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Description: Currently, some operations that throw IOException on the NameNode are evaluated

[jira] [Commented] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713233#comment-16713233 ] Lukas Majercak commented on HDFS-14134: --- Added a patch to demonstrate the issue. > Idempotent

[jira] [Updated] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14134: -- Description: Currently, some operations that throw IOException on the NameNode are evaluated

[jira] [Created] (HDFS-14134) Idempotent operations throwing RemoteException should not be retried by the client

2018-12-07 Thread Lukas Majercak (JIRA)
Lukas Majercak created HDFS-14134: - Summary: Idempotent operations throwing RemoteException should not be retried by the client Key: HDFS-14134 URL: https://issues.apache.org/jira/browse/HDFS-14134

[jira] [Commented] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-05 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675934#comment-16675934 ] Lukas Majercak commented on HDFS-14043: --- Yes, I ran the TestSaveNamespace in my local trunk version

[jira] [Commented] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-05 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675766#comment-16675766 ] Lukas Majercak commented on HDFS-14043: --- Added patch003 to fix checkstyle errors. > Tolerate

[jira] [Updated] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-05 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14043: -- Attachment: HDFS-14043.003.patch > Tolerate corrupted seen_txid file >

[jira] [Updated] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-05 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14043: -- Attachment: HDFS-14043.002.patch > Tolerate corrupted seen_txid file >

[jira] [Commented] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-05 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675677#comment-16675677 ] Lukas Majercak commented on HDFS-14043: --- Added patch002 that should apply to trunk. > Tolerate

[jira] [Comment Edited] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-01 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672077#comment-16672077 ] Lukas Majercak edited comment on HDFS-14043 at 11/1/18 7:38 PM:

[jira] [Commented] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-01 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672077#comment-16672077 ] Lukas Majercak commented on HDFS-14043: --- [~cmccabe] could you review this, as this seems to be

[jira] [Updated] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-01 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14043: -- Description: We already tolerate IOExceptions when reading seen_txid file from namenode's

[jira] [Updated] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-01 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14043: -- Attachment: HDFS-14043.001.patch > Tolerate corrupted seen_txid file >

[jira] [Created] (HDFS-14043) Tolerate corrupted seen_txid file

2018-11-01 Thread Lukas Majercak (JIRA)
Lukas Majercak created HDFS-14043: - Summary: Tolerate corrupted seen_txid file Key: HDFS-14043 URL: https://issues.apache.org/jira/browse/HDFS-14043 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-12284) RBF: Support for Kerberos authentication

2018-10-23 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661362#comment-16661362 ] Lukas Majercak commented on HDFS-12284: --- [~daryn], I feel like we should distinguish between

[jira] [Commented] (HDFS-14010) Pass correct DF usage to ReservedSpaceCalculator builder

2018-10-19 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657347#comment-16657347 ] Lukas Majercak commented on HDFS-14010: --- 002.patch LGTM. Maybe some warn log when usage==null, and

[jira] [Updated] (HDFS-14010) Pass correct DF usage to ReservedSpaceCalculator builder

2018-10-18 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14010: -- Affects Version/s: 2.9.2 > Pass correct DF usage to ReservedSpaceCalculator builder >

[jira] [Updated] (HDFS-14010) Pass correct DF usage to ReservedSpaceCalculator builder

2018-10-18 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14010: -- Attachment: HDFS-14010.001.patch > Pass correct DF usage to ReservedSpaceCalculator builder >

[jira] [Updated] (HDFS-14010) Pass correct DF usage to ReservedSpaceCalculator builder

2018-10-18 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-14010: -- Description: In FsVolumeImpl's constructor, we currently pass the DF usage that was passed to

[jira] [Created] (HDFS-14010) Pass correct DF usage to ReservedSpaceCalculator builder

2018-10-18 Thread Lukas Majercak (JIRA)
Lukas Majercak created HDFS-14010: - Summary: Pass correct DF usage to ReservedSpaceCalculator builder Key: HDFS-14010 URL: https://issues.apache.org/jira/browse/HDFS-14010 Project: Hadoop HDFS

[jira] [Commented] (HDFS-12284) RBF: Support for Kerberos authentication

2018-10-18 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655772#comment-16655772 ] Lukas Majercak commented on HDFS-12284: --- There are star imports in every test class import static

[jira] [Updated] (HDFS-13976) Backport HDFS-12813 to branch-2.9

2018-10-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-13976: -- Attachment: TestRequestHedgingProxyProvider.png > Backport HDFS-12813 to branch-2.9 >

[jira] [Commented] (HDFS-13976) Backport HDFS-12813 to branch-2.9

2018-10-10 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645401#comment-16645401 ] Lukas Majercak commented on HDFS-13976: --- Ran the tests on branch-2.9 and it seems fine,

[jira] [Updated] (HDFS-13976) Backport HDFS-12813 to branch-2.9

2018-10-08 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-13976: -- Attachment: HDFS-12813.branch-2.9.001.patch > Backport HDFS-12813 to branch-2.9 >

[jira] [Updated] (HDFS-13976) Backport HDFS-12813 to branch-2.9

2018-10-08 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-13976: -- Attachment: HDFS-12813.branch-2.001.patch > Backport HDFS-12813 to branch-2.9 >

[jira] [Updated] (HDFS-13976) Backport HDFS-12813 to branch-2.9

2018-10-08 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-13976: -- Fix Version/s: 2.9.2 > Backport HDFS-12813 to branch-2.9 > - >

[jira] [Created] (HDFS-13976) Backport HDFS-12813 to branch-2.9

2018-10-08 Thread Lukas Majercak (JIRA)
Lukas Majercak created HDFS-13976: - Summary: Backport HDFS-12813 to branch-2.9 Key: HDFS-13976 URL: https://issues.apache.org/jira/browse/HDFS-13976 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-13792) Fix FSN read/write lock metrics name

2018-08-06 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570761#comment-16570761 ] Lukas Majercak commented on HDFS-13792: --- Thanks for this [~csun], patch001 LGTM. > Fix FSN

[jira] [Commented] (HDFS-13757) After HDFS-12886, close() can throw AssertionError "Negative replicas!"

2018-07-20 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551420#comment-16551420 ] Lukas Majercak commented on HDFS-13757: --- Surely the test should fail if you disable IBR? > After

[jira] [Commented] (HDFS-13757) After HDFS-12886, close() can throw AssertionError "Negative replicas!"

2018-07-20 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551418#comment-16551418 ] Lukas Majercak commented on HDFS-13757: --- Hi [~jojochuang]. Unfortunately, I haven't been able to

[jira] [Comment Edited] (HDFS-13714) Fix TestNameNodePrunesMissingStorages test failures on Windows

2018-07-02 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530423#comment-16530423 ] Lukas Majercak edited comment on HDFS-13714 at 7/2/18 8:44 PM: --- The rename

[jira] [Commented] (HDFS-13714) Fix TestNameNodePrunesMissingStorages test failures on Windows

2018-07-02 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530423#comment-16530423 ] Lukas Majercak commented on HDFS-13714: --- The rename used is standard Java. From docs: {code:java}

[jira] [Updated] (HDFS-13714) Fix TestNameNodePrunesMissingStorages test failures on Windows

2018-07-02 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-13714: -- Affects Version/s: 3.2.0 > Fix TestNameNodePrunesMissingStorages test failures on Windows >

[jira] [Work started] (HDFS-13714) Fix TestNameNodePrunesMissingStorages test failures on Windows

2018-07-02 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-13714 started by Lukas Majercak. - > Fix TestNameNodePrunesMissingStorages test failures on Windows >

[jira] [Updated] (HDFS-13714) Fix TestNameNodePrunesMissingStorages test failures on Windows

2018-07-02 Thread Lukas Majercak (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-13714: -- Attachment: HDFS-13714.000.patch > Fix TestNameNodePrunesMissingStorages test failures on

  1   2   3   4   >