[
https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980659#comment-15980659
]
Rui Li commented on HDFS-11687:
---
Thanks [~ngangam]. Besides isHDFSEncryptionEnabled, we also need APIs
[
https://issues.apache.org/jira/browse/HDFS-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935670#comment-15935670
]
Rui Li commented on HDFS-9962:
--
[~tasanuma0829], feel free to take the JIRA. Thanks.
> Erasure Coding: need a
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345826#comment-15345826
]
Rui Li commented on HDFS-9603:
--
Hi [~drankye], per our discussion, Wei Zhou would help with this task. But I
[
https://issues.apache.org/jira/browse/HDFS-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194788#comment-15194788
]
Rui Li commented on HDFS-9962:
--
As I proposed in HDFS-7866, one possible solution is to randomly choose a
Rui Li created HDFS-9962:
Summary: Erasure Coding: need a way to test multiple EC policies
Key: HDFS-9962
URL: https://issues.apache.org/jira/browse/HDFS-9962
Project: Hadoop HDFS
Issue Type:
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186623#comment-15186623
]
Rui Li commented on HDFS-7866:
--
Thanks guys for the review!
> Erasure coding: NameNode manages multiple
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186314#comment-15186314
]
Rui Li commented on HDFS-7866:
--
Thanks [~jingzhao] and [~zhz] for the great discussions. So we're good to
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.13.patch
I realized that {{TestEditLog}} fails on trunk as well. So it's not related to
the
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184300#comment-15184300
]
Rui Li commented on HDFS-7866:
--
Thanks Walter for the suggestions.
bq. separating the logic of manipulation of
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.12.patch
Thanks Zhe for the clarifications!
Patch updated to address comments.
> Erasure
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179541#comment-15179541
]
Rui Li commented on HDFS-7866:
--
Thanks Zhe for your comments.
1. We're using 11 bits to store the policy ID,
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177063#comment-15177063
]
Rui Li commented on HDFS-7866:
--
Test failures cannot be reproduced. I'll fix the checkstyle in next update.
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.11.patch
Thanks Zhe for the explanations!
Update patch accordingly.
> Erasure coding:
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174861#comment-15174861
]
Rui Li commented on HDFS-7866:
--
Thanks Zhe for the review and comments!
Forgive my ignorance but what do you
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171824#comment-15171824
]
Rui Li commented on HDFS-7866:
--
The latest failures are not related.
> Erasure coding: NameNode manages
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.10.patch
Thanks Kai for the explanations!
Rebase the patch and fix the checkstyle issue.
>
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.9.patch
Fix test. I don't quite understand the checkstyle issue. The methods in
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.8.patch
Update the patch as all the changes to tests are committed.
The v8 patch uses hard
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160029#comment-15160029
]
Rui Li commented on HDFS-9816:
--
Thanks Zhe and Walter for the review!
> Erasure Coding: allow to use multiple
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156816#comment-15156816
]
Rui Li commented on HDFS-9816:
--
The failures are not related.
> Erasure Coding: allow to use multiple EC
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9816:
-
Attachment: HDFS-9816.3.patch
The v3 patch reverts changes to {{TestStripedINodeFile}} and
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156362#comment-15156362
]
Rui Li commented on HDFS-9816:
--
Yeah that makes sense. Thanks Zhe. I'll update the patch.
> Erasure Coding:
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15154129#comment-15154129
]
Rui Li commented on HDFS-9816:
--
Test failures cannot be reproduced locally.
> Erasure Coding: allow to use
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9816:
-
Attachment: HDFS-9816.2.patch
Update patch to address comments.
The new static method is added to
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153579#comment-15153579
]
Rui Li commented on HDFS-9816:
--
Thanks [~zhz] for the clarifications. How about putting the static method in
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151803#comment-15151803
]
Rui Li commented on HDFS-9816:
--
Thanks Walter for your inputs.
As to reusing production code,
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151653#comment-15151653
]
Rui Li commented on HDFS-9816:
--
[~zhz], I think you mean {{TestBlockRecovery#testSafeLength}} right? If we
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151609#comment-15151609
]
Rui Li commented on HDFS-9816:
--
Thanks guys for the discussion. I added
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149691#comment-15149691
]
Rui Li commented on HDFS-9816:
--
Thanks Zhe for the review.
{{TestLeaseRecoveryStriped}} failed because of
[
https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9816:
-
Attachment: HDFS-9816.1.patch
{{TestStripedINodeFile}} and {{TestFSImage}} don't have all the changes in
Rui Li created HDFS-9816:
Summary: Erasure Coding: allow to use multiple EC policies in
striping related tests [Part 3]
Key: HDFS-9816
URL: https://issues.apache.org/jira/browse/HDFS-9816
Project: Hadoop
[
https://issues.apache.org/jira/browse/HDFS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148010#comment-15148010
]
Rui Li commented on HDFS-9755:
--
Thanks Zhe for reviewing and committing the patch.
> Erasure Coding: allow to
[
https://issues.apache.org/jira/browse/HDFS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15142113#comment-15142113
]
Rui Li commented on HDFS-9755:
--
Thanks for clarifying.
> Erasure Coding: allow to use multiple EC policies in
[
https://issues.apache.org/jira/browse/HDFS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9755:
-
Attachment: HDFS-9755.2.patch
HDFS-9755.2.patch
Thanks Zhe for the review! I added the assert in
[
https://issues.apache.org/jira/browse/HDFS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132172#comment-15132172
]
Rui Li commented on HDFS-9755:
--
{{TestDatanodeStartupFixesLegacyStorageIDs}} fails on trunk as well. Other
Rui Li created HDFS-9755:
Summary: Erasure Coding: allow to use multiple EC policies in
striping related tests [Part 2]
Key: HDFS-9755
URL: https://issues.apache.org/jira/browse/HDFS-9755
Project: Hadoop
[
https://issues.apache.org/jira/browse/HDFS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9755:
-
Status: Patch Available (was: Open)
> Erasure Coding: allow to use multiple EC policies in striping related tests
[
https://issues.apache.org/jira/browse/HDFS-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9755:
-
Attachment: HDFS-9755.1.patch
> Erasure Coding: allow to use multiple EC policies in striping related tests
>
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131520#comment-15131520
]
Rui Li commented on HDFS-9658:
--
Thanks Zhe for the review!
> Erasure Coding: allow to use multiple EC
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9658:
-
Attachment: HDFS-9658.2.patch
Thanks Zhe. Update patch to increase {{maxPerLevel}}.
> Erasure Coding: allow to
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127885#comment-15127885
]
Rui Li commented on HDFS-7866:
--
Thanks [~zhz] for pointing me to the discussion and
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129957#comment-15129957
]
Rui Li commented on HDFS-9658:
--
Failures are not related.
> Erasure Coding: allow to use multiple EC policies
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127428#comment-15127428
]
Rui Li commented on HDFS-9658:
--
Hi [~zhz], thanks for the review!
I kept {{dnIndexSuite}} so that the new
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9658:
-
Attachment: HDFS-9658.1.patch
> Erasure Coding: make tests compatible with multiple EC policies [Part 1]
>
Rui Li created HDFS-9658:
Summary: Erasure Coding: make tests compatible with multiple EC
policies [Part 1]
Key: HDFS-9658
URL: https://issues.apache.org/jira/browse/HDFS-9658
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105209#comment-15105209
]
Rui Li commented on HDFS-7866:
--
Thanks Kai for the suggestions. I just filed HDFS-9658 to break this task into
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9658:
-
Status: Patch Available (was: Open)
> Erasure Coding: make tests compatible with multiple EC policies [Part 1]
>
[
https://issues.apache.org/jira/browse/HDFS-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106105#comment-15106105
]
Rui Li commented on HDFS-9658:
--
Test failures don't seem related and cannot be reproduced locally.
> Erasure
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9603:
-
Attachment: HDFS-9603.2.patch
Update patch to address Kai's comments.
> Erasure Coding: Use ErasureCoder to
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9603:
-
Attachment: HDFS-9603.3.patch
Fix checkstyle.
> Erasure Coding: Use ErasureCoder to encode/decode a block group
>
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9603:
-
Attachment: HDFS-9603.1.patch
Upload a patch for review. It only changes how we encode data.
[~drankye] please let
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9603:
-
Status: Patch Available (was: Open)
> Erasure Coding: Use ErasureCoder to encode/decode a block group
>
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082210#comment-15082210
]
Rui Li commented on HDFS-9603:
--
Thanks Kai for the review.
I used the concrete class because we already have
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082217#comment-15082217
]
Rui Li commented on HDFS-9603:
--
Hi Kai, I got one more question about {{ECBlock}}. I remember you said we're
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082268#comment-15082268
]
Rui Li commented on HDFS-9603:
--
Thanks Kai for clarifying!
> Erasure Coding: Use ErasureCoder to
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15075676#comment-15075676
]
Rui Li commented on HDFS-7866:
--
Thanks [~drankye] for the review!
1. Sure, will do.
2. Since the system
Rui Li created HDFS-9603:
Summary: Erasure Coding: Use ErasureCoder to encode/decode a block
group
Key: HDFS-9603
URL: https://issues.apache.org/jira/browse/HDFS-9603
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-9603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072522#comment-15072522
]
Rui Li commented on HDFS-9603:
--
cc [~drankye]
> Erasure Coding: Use ErasureCoder to encode/decode a block
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.7.patch
Rebase patch with latest trunk.
Update to {{TestUnderReplicatedBlockQueues}} is
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15066031#comment-15066031
]
Rui Li commented on HDFS-7866:
--
Thanks Kai for your comments.
bp. However, please don't set it as default
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.6.patch
The v6 patch adds {{TestErasureCodingPolicies::testMultiplePoliciesCoExist}} to
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063885#comment-15063885
]
Rui Li commented on HDFS-7866:
--
The latest failures cannot be reproduced locally.
> Erasure coding: NameNode
[
https://issues.apache.org/jira/browse/HDFS-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065194#comment-15065194
]
Rui Li commented on HDFS-8461:
--
Thanks [~zhz] for your input. In
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.4.patch
The v4 patch is mainly for review and testing. It basically includes:
1. Add a new
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Status: Patch Available (was: In Progress)
> Erasure coding: NameNode manages EC schemas
>
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-7866:
-
Attachment: HDFS-7866.5.patch
Update patch to fix more tests.
> Erasure coding: NameNode manages EC schemas
>
[
https://issues.apache.org/jira/browse/HDFS-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15063496#comment-15063496
]
Rui Li commented on HDFS-8461:
--
Hi, I got a question about the following code of the patch:
{code}
} else
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048347#comment-15048347
]
Rui Li commented on HDFS-7866:
--
I tried to add another system policy (with different schema) and hit the
[
https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046636#comment-15046636
]
Rui Li commented on HDFS-7866:
--
Hi [~drankye], it seems HDFS-9345 depends on this JIRA. I can help with it if
[
https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037062#comment-15037062
]
Rui Li commented on HDFS-9496:
--
[~huizane] and [~zhz] - We already have HADOOP-11588 to implement a benchmark
Rui Li created HDFS-9424:
Summary: Erasure Coding: enhance the
ErasureCodeBenchmarkThroughput tool
Key: HDFS-9424
URL: https://issues.apache.org/jira/browse/HDFS-9424
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-9424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-9424:
-
Priority: Minor (was: Major)
> Erasure Coding: enhance the ErasureCodeBenchmarkThroughput tool
>
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003886#comment-15003886
]
Rui Li commented on HDFS-8968:
--
Thanks Zhe for the review and good suggestions. I just filed HDFS-9424 as the
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8968:
-
Attachment: HDFS-8968.5.patch
Update patch to avoid generating random buffers each time before writing to a
file,
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8968:
-
Attachment: HDFS-8968.4.patch
Address Rakesh's comments.
A new test is added to test the new benchmark tool, in
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991596#comment-14991596
]
Rui Li commented on HDFS-8968:
--
Hi [~rakeshr],
One more question. What do you mean by running a real cluster
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992866#comment-14992866
]
Rui Li commented on HDFS-8968:
--
I was planning to run ErasureCodeBenchmarkThroughput with MiniDFSCluster in
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990964#comment-14990964
]
Rui Li commented on HDFS-8968:
--
Thanks [~rakeshr] for the detailed and helpful comments. I'll update the patch
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8968:
-
Attachment: HDFS-8968.3.patch
Rebase my patch with latest trunk code.
> New benchmark throughput tool for
[
https://issues.apache.org/jira/browse/HDFS-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986864#comment-14986864
]
Rui Li commented on HDFS-9345:
--
[~drankye] told me that dummy schema depends on some other JIRAs.
So I just
Rui Li created HDFS-9345:
Summary: Erasure Coding: create dummy coder and schema
Key: HDFS-9345
URL: https://issues.apache.org/jira/browse/HDFS-9345
Project: Hadoop HDFS
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902070#comment-14902070
]
Rui Li commented on HDFS-8920:
--
Thanks guys for the review.
> Erasure Coding: when recovering lost blocks,
[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8920:
-
Attachment: HDFS-8920-HDFS-7285.2.patch
Address Kai's comments offline.
> Erasure Coding: when recovering lost
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804972#comment-14804972
]
Rui Li commented on HDFS-8968:
--
[~andrew.wang] - It's good idea to put it in hadoop-tools. Problem is that I
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791444#comment-14791444
]
Rui Li commented on HDFS-8968:
--
Hi [~zhz] and [~andrew.wang], the tool is intended to benchmark end to end
[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791532#comment-14791532
]
Rui Li commented on HDFS-8920:
--
The latest failures still can't be reproduced locally.
> Erasure Coding: when
[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746906#comment-14746906
]
Rui Li commented on HDFS-8920:
--
I tried several failures locally and cannot reproduce them. Also both
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746771#comment-14746771
]
Rui Li commented on HDFS-8968:
--
Hi [~drankye], the tool is intended to test and compare throughput of EC and
[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8920:
-
Attachment: HDFS-8920-HDFS-7285.1.patch
Sorry for taking some time to get back to this. Based on my discussion
[
https://issues.apache.org/jira/browse/HDFS-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8920:
-
Status: Patch Available (was: Open)
> Erasure Coding: when recovering lost blocks, logs can be too verbose and
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742837#comment-14742837
]
Rui Li commented on HDFS-8968:
--
Findbugs seems to have 4 new warnings but Findbugs Report is just empty. Any
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8968:
-
Attachment: HDFS-8968-HDFS-7285.2.patch
Rebase my patch and address comments. Thanks [~drankye] for your inputs.
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734521#comment-14734521
]
Rui Li commented on HDFS-8968:
--
Failed tests don't seem related. The patch here doesn't modify any existing
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8968:
-
Attachment: HDFS-8968-HDFS-7285.1.patch
> New benchmark throughput tool for striping erasure coding
>
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8968:
-
Status: Patch Available (was: In Progress)
I didn't add new test as it's an end to end throughput benchmark tool.
[
https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733224#comment-14733224
]
Rui Li commented on HDFS-8833:
--
Hey [~zhz], we'd like to try out your patch. But it seems v5 patch doesn't
[
https://issues.apache.org/jira/browse/HDFS-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HDFS-8968 started by Rui Li.
> New benchmark throughput tool for striping erasure coding
>
[
https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723022#comment-14723022
]
Rui Li commented on HDFS-7285:
--
[~brahmareddy] - Yeah I'm working on the two issues to support parallel
[
https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723077#comment-14723077
]
Rui Li commented on HDFS-7285:
--
Yep. We intend to create a throughput benchmark tool in HDFS-8968.
[
https://issues.apache.org/jira/browse/HDFS-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HDFS-8987:
-
Description:
Test progress is as follows
* For a new cluster, I format the namenode and then start HDFS service.
1 - 100 of 105 matches
Mail list logo
