This is an automated email from the ASF dual-hosted git repository.
sammichen pushed a change to branch HDDS-1564
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
omit 40da915 HDDS-1564. Rebase fix.
omit 417c958 Revert "Rebase Fix"
omit 51b8cd1 HDDS-2786. ITestOzoneContractSeek zero byte file failures
(#384)
omit d5034a0 HDDS-2777. Add bytes read statistics to Ozone FileSystem
implementation (#382)
omit a96ff61 HDDS-2704. Source tar file is not created during the relase
build. (#380)
omit 7a6b6a1 HDDS-2685. Fix Rename API in BasicOzoneFileSystem (#321)
omit f99848d HDDS-1989. Fix ApplyTransaction error handling in
OzoneManagerStateMachine. (#351)
omit 57d3fe5 HDDS-2755. Compare transactionID and updateID of Volume
operations to avoid replaying transactions (#369)
omit 0d91171 HDDS-2762. Sonar warnings fixed (#373)
omit f472e8d HDDS-2776. NPE when stop recon while start recon failed (#376)
omit 79ea2d9 HDDS-2774. Hadoop 3.1 acceptance test fails with apk command
not found (#374)
omit d318ee2 HDDS-2735. Let GitHub Actions run acceptance check in
parallel (#358)
omit 21281c6 HDDS-2709. Maven property skipShade should not skip ozonefs
compilation (#342)
omit a359ab6 HDDS-2652. Remove unnecessary log statement. (#371)
omit 1331aa1 HDDS-2739. No need to try install awscli before each test
(#359)
omit 3a6fa66 HDDS-2703. OzoneFSInputStream to support ByteBufferReadable
(#345)
omit 27a77e9 HDDS-2722. Let ChunkManager read/write ChunkBuffer instead of
ByteBuffer (#352)
omit bd14928 HDDS-2070. Create insight point to debug one specific
pipeline (#299)
omit 552f0b3 HDDS-2623. Expose SCMDatanodeProtocolServer RPC endpoint
through Recon. (#353)
omit 8158b12 Revert "HDDS-2704. Source tar file is not created during the
relase build (#334)" (#368)
omit c7caff1 HDDS-2630. NullPointerException in S3g. (#336)
omit 3ecb968 HDDS-2752. Unnecessary calls to isNoneEmpty and isAllEmpty
(#364)
omit 4753d70 HDDS-2669. Sonar : fix issues in scm Pipeline (#360)
omit bdb0839 HDDS-2724. Sonar : Fix sonar issues in package
'org.apache.hadoop.ozo… (#354)
omit eb341bd HDDS-2723. Wrap InputStream with try-with-resource in
TestOzoneFSInputStream. (#361)
omit 7f9368b HDDS-2751. Wrong number of placeholders in log message (#362)
omit 3b6eb17 HDDS-2729. Enable multilingual Hugo features in ozone docs
(#357)
omit 88727f7 HDDS-2539. Sonar: Fix sonar warnings in SCMContainerManager
class. (#332)
omit 7b2355a HDDS-2578. Handle InterruptedException in RandomKeyGenerator
(#305)
omit c311c99 HDDS-2675. Sonar : fix issues in PipelineStateManager (#347)
omit 341785d HDDS-2680. Fix updating lastAppliedIndex in
OzoneManagerStateMachine. (#316)
omit f94f8f7 HDDS-2414. Simplify robot tests with removing output greps
omit c5da69e HDDS-2423. Add the recover-trash command client side code
omit 71021e4 HDDS-2553. Sonar: Iterator.next() methods should throw
NoSuchElementException
omit 551a585 HDDS-2242. Avoid unnecessary rpc needed to discover the
pipeline leader. (#313)
omit 5707737 HDDS-2684. Refactor common test utilities to
hadoop-hdds/common (#322)
omit 3799c3e HDDS-2588. Consolidate compose environments
omit db7efc7 HDDS-2659. KeyValueHandler#handleCreateContainer should log
the exception on container creation failure (#296).
omit 54deaa0 HDDS-2688. Refactor container response builders to
hadoop-hdds/common (#324)
omit 3da6188 HDDS-2711. OM does not report JVM metrics (#346)
omit 86af50f HDDS-2704. Source tar file is not created during the relase
build (#334)
omit db20ab5 HDDS-2561. Handle InterruptedException in LeaseManager (#338)
omit 5a507b4 HDDS-2532. Sonar : fix issues in OzoneQuota (#311)
omit 14f13c9 HDDS-2389. add toStateMachineLogEntryString provider in
Ozone's ContainerStateMachine. (#298)
omit 554c311 HDDS-2105. Merge OzoneClientFactory#getRpcClient functions
(#148)
omit d0ec961 HDDS-2693. HddsVolume mixes ChunkLayOutVersion and
DataNodeLayoutVersion (#330)
omit 91cab42 HDDS-1993. Merge OzoneManagerRequestHandler and
OzoneManagerHARequest… (#304)
omit 1b43a92 HDDS-2683. Ratis MaxBuffer should be the same size as the
segment size. (#320)
omit b324d4a HDDS-2645. Refactor MiniOzoneChaosCluster to a different
package to add filesystem tests. (#292)
omit 6f743cc HDDS-2668. Sonar : fix issues reported in BlockManagerImpl
(#317)
omit 1ecd057 HDDS-2687. Fix sonar issues in package
org.apache.hadoop.ozone.recon.api. (#325)
omit b33b64d HDDS-2689. OMException NOT_A_FILE missing space in the
exception message. (#327)
omit 421337b HDDS-2692. Seek to file end throws EOF Exception. (#326)
omit 0f8239f HDDS-1991. Remove RatisClient in OM HA. (#319)
omit e306d51 HDDS-2681. Add leak detection memory flags to
MiniOzoneChaosCluster
omit 7657fbd HDDS-2662. Update gRPC and datanode protobuf version in Ozone
omit 6c5d0fb HDDS-2678. Add thread name to log pattern (#315)
omit 4c046d4 HDDS-2636. Refresh pipeline information in OzoneManager
lookupFile call. (#301)
omit 7478168 HDDS-1987. Fix listStatus API (#108)
omit 80975cc HDDS-2570. Handle InterruptedException in ProfileServlet.
(#308)
omit 93abc66 HDDS-2677. Acceptance test may fail despite success status
(#312)
omit d89f6d4 HDDS-2629. Ozone CLI: CreationTime/modifyTime of
volume/bucket/key in… (#272)
omit 30b9d58 HDDS-2530. Sonar : refactor verifyResourceName in
HddsClientUtils to fix Sonar errors (#293)
omit f2965ac HDDS-2666. Suppress loader constraint violation message in
TestOzoneFileSystemWithMocks (#303)
omit 74bab02 HDDS-1873. Add API to get last completed times for every
Recon task. (#233)
omit 0319d42 HDDS-2600. Move chaos test to org.apache.hadoop.ozone.chaos
package. (#307)
omit bf36ab0 HDDS-2565. Handle InterruptedException in VolumeSet
omit 0b5d319 HDDS-2506. Remove keyAllocationInfo and replication info from
the auditLog (#288)
omit 495700c Addendum HDDS-2646. Start acceptance tests only if at least
one THREE pipeline is available (#282)
omit d7a4ad4 HDDS-2667. Prometheus reports invalid metric type
omit d903120 HDDS-2622. Add config parameter for setting limit on total
bytes of pending requests in Ratis. (#264)
omit 3d1b44d HDDS-2646. Start acceptance tests only if at least one THREE
pipeline is available (#282)
omit 8b7d7ef HDDS-2608. Provide command to wait until SCM is out from the
safe-mode (#250)
omit f0ea1a4 HDDS-2653. Improve executor memory usage in new Freon tests
(#284)
omit 5c21608 HDDS-2475. Unregister ContainerMetadataScrubberMetrics on
thread exit (#286)
omit 0543da7 HDDS-2656. Prefer execute() over submit() if the returned
Future is ignored (#302)
omit 13c1db0 HDDS-2571. Handle InterruptedException in SCMPipelineManager
(#295)
omit bf9cbf1 HDDS-1722. Use the bindings in ReconSchemaGenerationModule to
create Recon SQL tables on startup. (#294)
omit 4efd8aa HDDS-2620. Fix listMultipartupload API. (#263)
omit 8c07266 HDDS-2386. Implement incremental ChunkBuffer. (#205)
omit cdb9e15 HDDS-2637. Handle LeaderNot ready exception in OzoneManager
StateMachine and upgrade ratis to latest version. (#276)
omit 27c384f HDDS-2573. Handle InterruptedException in KeyOutputStream.
(#257)
omit bb22b3c HDDS-2455. Implement MiniOzoneHAClusterImpl#getOMLeader (#137)
omit ab740dc HDDS-2657. Key get command creates the output file even in
case of KEY_NOT_FOUND (#290)
omit f831950 HDDS-2639. TestTableCacheImpl is flaky (#278)
omit a84a9f4 HDDS-2654. Conditionally enable profiling at the kernel level
omit 367846d HDDS-2658. Insight log level reset does not work
omit 2defafa HDDS-2655. Use pre-compiled Pattern in NetUtils#normalize
omit 28dc284 HDDS-2619. Fix Sonar issues in ReconTaskControllerImpl
omit 431c85d HDDS-2444. Remove server side dependencies from ozonefs jar
files
omit 9ebcb54 HDDS-2505. Fix logic related to SCM address calculation in
HddsUtils
omit 1938862 HDDS-1576. Support configure more than one raft log storage
to host m… (#117)
omit ad1617d HDDS-1574 Average out pipeline allocation on datanodes and
add metrcs/test (#291)
omit 01eb8cd HDDS-2650 Fix createPipeline CLI and make it message based.
(#370)
omit 7f05458 Revert "HDDS-2650 Fix createPipeline CLI. (#340)"
omit e7315e9 HDDS-2035 Implement datanode level CLI to reveal pipeline
relation. (#348)
omit 7c71710 HDDS-2650 Fix createPipeline CLI. (#340)
omit 03c93ec Rebase Fix
omit 2a3638d HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN
pipeline. (#237)
omit 260018f HDDS-1569 Support creating multiple pipelines with same
datanode. Contributed by Li Cheng.
omit 555f923 HDDS-2089: Add createPipeline CLI. (#1418)
omit 4bd9c78 HDDS-1571. Create an interface for pipeline placement policy
to support network topologies. (#1395)
omit 44c783f HDDS-1577. Add default pipeline placement policy
implementation. (#1366)
add 5e49d7a HDDS-2505. Fix logic related to SCM address calculation in
HddsUtils
add 46fd24b HDDS-2444. Remove server side dependencies from ozonefs jar
files
add 968dec4 HDDS-2619. Fix Sonar issues in ReconTaskControllerImpl
add afda1c4 HDDS-2655. Use pre-compiled Pattern in NetUtils#normalize
add 7524ee4 HDDS-2658. Insight log level reset does not work
add 3480c89 HDDS-2654. Conditionally enable profiling at the kernel level
add a25b2b4 HDDS-2639. TestTableCacheImpl is flaky (#278)
add f141097 HDDS-2657. Key get command creates the output file even in
case of KEY_NOT_FOUND (#290)
add 4d42714 HDDS-2455. Implement MiniOzoneHAClusterImpl#getOMLeader (#137)
add 87e02a3 HDDS-2573. Handle InterruptedException in KeyOutputStream.
(#257)
add 9381dd0 HDDS-2637. Handle LeaderNot ready exception in OzoneManager
StateMachine and upgrade ratis to latest version. (#276)
add 2fb8747 HDDS-2386. Implement incremental ChunkBuffer. (#205)
add dbd05b0 HDDS-2620. Fix listMultipartupload API. (#263)
add ea6c2c9 HDDS-1722. Use the bindings in ReconSchemaGenerationModule to
create Recon SQL tables on startup. (#294)
add a3a2bfd HDDS-2571. Handle InterruptedException in SCMPipelineManager
(#295)
add c9f2b02 HDDS-2656. Prefer execute() over submit() if the returned
Future is ignored (#302)
add 7b71678 HDDS-2475. Unregister ContainerMetadataScrubberMetrics on
thread exit (#286)
add 5aa7383 HDDS-2653. Improve executor memory usage in new Freon tests
(#284)
add f686239 HDDS-2608. Provide command to wait until SCM is out from the
safe-mode (#250)
add 0ff53ef HDDS-2646. Start acceptance tests only if at least one THREE
pipeline is available (#282)
add a9990f6 HDDS-2622. Add config parameter for setting limit on total
bytes of pending requests in Ratis. (#264)
add 8e1e225 HDDS-2667. Prometheus reports invalid metric type
add 6c1a9ff Addendum HDDS-2646. Start acceptance tests only if at least
one THREE pipeline is available (#282)
add b35ad98 HDDS-2506. Remove keyAllocationInfo and replication info from
the auditLog (#288)
add 4f1869c HDDS-2565. Handle InterruptedException in VolumeSet
add 1998983 HDDS-2600. Move chaos test to org.apache.hadoop.ozone.chaos
package. (#307)
add 2f72319 HDDS-1873. Add API to get last completed times for every
Recon task. (#233)
add e359e66 HDDS-2666. Suppress loader constraint violation message in
TestOzoneFileSystemWithMocks (#303)
add 1af4f48 HDDS-2530. Sonar : refactor verifyResourceName in
HddsClientUtils to fix Sonar errors (#293)
add d18aa99 HDDS-2629. Ozone CLI: CreationTime/modifyTime of
volume/bucket/key in… (#272)
add 9b07c34 HDDS-2677. Acceptance test may fail despite success status
(#312)
add 1e18347 HDDS-2570. Handle InterruptedException in ProfileServlet.
(#308)
add 9593b25 HDDS-1987. Fix listStatus API (#108)
add f69cb1e HDDS-2636. Refresh pipeline information in OzoneManager
lookupFile call. (#301)
add 76ad638 HDDS-2678. Add thread name to log pattern (#315)
add e54b9be HDDS-2662. Update gRPC and datanode protobuf version in Ozone
add db4a6e6 HDDS-2681. Add leak detection memory flags to
MiniOzoneChaosCluster
add 5eef651 HDDS-1991. Remove RatisClient in OM HA. (#319)
add 1503ab8 HDDS-2692. Seek to file end throws EOF Exception. (#326)
add 588ff1e HDDS-2689. OMException NOT_A_FILE missing space in the
exception message. (#327)
add 91a6477 HDDS-2687. Fix sonar issues in package
org.apache.hadoop.ozone.recon.api. (#325)
add f3abb4c HDDS-2668. Sonar : fix issues reported in BlockManagerImpl
(#317)
add 1d56bc2 HDDS-2645. Refactor MiniOzoneChaosCluster to a different
package to add filesystem tests. (#292)
add 5d9ae67 HDDS-2683. Ratis MaxBuffer should be the same size as the
segment size. (#320)
add 8bfc726 HDDS-1993. Merge OzoneManagerRequestHandler and
OzoneManagerHARequest… (#304)
add 022ceb5 HDDS-2693. HddsVolume mixes ChunkLayOutVersion and
DataNodeLayoutVersion (#330)
add 42a30da HDDS-2105. Merge OzoneClientFactory#getRpcClient functions
(#148)
add 6b78cc5 HDDS-2389. add toStateMachineLogEntryString provider in
Ozone's ContainerStateMachine. (#298)
add 51c5e42 HDDS-2532. Sonar : fix issues in OzoneQuota (#311)
add 050d58b HDDS-2561. Handle InterruptedException in LeaseManager (#338)
add 15f4ed4 HDDS-2704. Source tar file is not created during the relase
build (#334)
add 10bff97 HDDS-2711. OM does not report JVM metrics (#346)
add 6506789 HDDS-2688. Refactor container response builders to
hadoop-hdds/common (#324)
add 0b1ea25 HDDS-2659. KeyValueHandler#handleCreateContainer should log
the exception on container creation failure (#296).
add e14f709 HDDS-2588. Consolidate compose environments
add 4732a19 HDDS-2684. Refactor common test utilities to
hadoop-hdds/common (#322)
add eeb4ac7 HDDS-2242. Avoid unnecessary rpc needed to discover the
pipeline leader. (#313)
add d3782c5 HDDS-2553. Sonar: Iterator.next() methods should throw
NoSuchElementException
add d03fd1a HDDS-2423. Add the recover-trash command client side code
add 114ff41 HDDS-2414. Simplify robot tests with removing output greps
add 9f2818c HDDS-2680. Fix updating lastAppliedIndex in
OzoneManagerStateMachine. (#316)
add a14a6bc HDDS-2675. Sonar : fix issues in PipelineStateManager (#347)
add 7095ded HDDS-2578. Handle InterruptedException in RandomKeyGenerator
(#305)
add 6ee5fd1 HDDS-2539. Sonar: Fix sonar warnings in SCMContainerManager
class. (#332)
add 45b80c5 HDDS-2729. Enable multilingual Hugo features in ozone docs
(#357)
add 7f2acaa HDDS-2751. Wrong number of placeholders in log message (#362)
add 9d24f23 HDDS-2723. Wrap InputStream with try-with-resource in
TestOzoneFSInputStream. (#361)
add 6c8b837 HDDS-2724. Sonar : Fix sonar issues in package
'org.apache.hadoop.ozo… (#354)
add 5f4202b HDDS-2669. Sonar : fix issues in scm Pipeline (#360)
add 2a91bb6 HDDS-2752. Unnecessary calls to isNoneEmpty and isAllEmpty
(#364)
add 9b72208 HDDS-2630. NullPointerException in S3g. (#336)
add 4d0e264 Revert "HDDS-2704. Source tar file is not created during the
relase build (#334)" (#368)
add ce6858c HDDS-2623. Expose SCMDatanodeProtocolServer RPC endpoint
through Recon. (#353)
add bf23a2c HDDS-2070. Create insight point to debug one specific
pipeline (#299)
add a98eec0 HDDS-2722. Let ChunkManager read/write ChunkBuffer instead of
ByteBuffer (#352)
add dfd5d8e HDDS-2703. OzoneFSInputStream to support ByteBufferReadable
(#345)
add fb45b20 HDDS-2739. No need to try install awscli before each test
(#359)
add 39f3e68 HDDS-2652. Remove unnecessary log statement. (#371)
add e48eee6 HDDS-2709. Maven property skipShade should not skip ozonefs
compilation (#342)
add 86fbec6 HDDS-2735. Let GitHub Actions run acceptance check in
parallel (#358)
add d778472 HDDS-2774. Hadoop 3.1 acceptance test fails with apk command
not found (#374)
add d57b868 HDDS-2776. NPE when stop recon while start recon failed (#376)
add 0cee30e HDDS-2762. Sonar warnings fixed (#373)
add b834fa4 HDDS-2755. Compare transactionID and updateID of Volume
operations to avoid replaying transactions (#369)
add 20b908d HDDS-1989. Fix ApplyTransaction error handling in
OzoneManagerStateMachine. (#351)
add a71c9bf HDDS-2685. Fix Rename API in BasicOzoneFileSystem (#321)
add c1dfdea HDDS-2704. Source tar file is not created during the relase
build. (#380)
add b5008d0 HDDS-2777. Add bytes read statistics to Ozone FileSystem
implementation (#382)
add a457616 HDDS-2786. ITestOzoneContractSeek zero byte file failures
(#384)
new 7c752f4 HDDS-1577. Add default pipeline placement policy
implementation. (#1366)
new 3d2c2e4 HDDS-1571. Create an interface for pipeline placement policy
to support network topologies. (#1395)
new c3f450e HDDS-2089: Add createPipeline CLI. (#1418)
new da464bc HDDS-1569 Support creating multiple pipelines with same
datanode. Contributed by Li Cheng.
new 5525b98 HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN
pipeline. (#237)
new f27605c Rebase Fix
new baf58d5 HDDS-2650 Fix createPipeline CLI. (#340)
new 8c07782 HDDS-2035 Implement datanode level CLI to reveal pipeline
relation. (#348)
new 7604c3c Revert "HDDS-2650 Fix createPipeline CLI. (#340)"
new f842691 HDDS-2650 Fix createPipeline CLI and make it message based.
(#370)
new 09bb190 HDDS-1574 Average out pipeline allocation on datanodes and
add metrcs/test (#291)
new 3e6f02d Resolve rebase conflict.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (40da915)
\
N -- N -- N refs/heads/HDDS-1564 (3e6f02d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../transport/server/ratis/XceiverServerRatis.java | 19 -----
.../hdds/scm/server/StorageContainerManager.java | 3 +-
.../hadoop/hdds/scm/container/MockNodeManager.java | 2 +-
.../scm/pipeline/TestRatisPipelineProvider.java | 12 ++--
.../TestRatisPipelineCreateAndDestroy.java | 81 +---------------------
.../scm/pipeline/TestRatisPipelineProvider.java | 10 ---
.../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 27 +-------
7 files changed, 13 insertions(+), 141 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
