[hadoop-ozone] branch master updated (c47aaa1 -> e91cc32)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from c47aaa1 HDDS-2398. Remove usage of LogUtils class from ratis-common. (#113) add e91cc32 HDDS-2272. Avoid buffer copying in GrpcReplicationClient No new revisions were added by this update. Summary of changes: .../replication/GrpcReplicationClient.java | 39 +++--- 1 file changed, 20 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (e91cc32 -> 9857b0c)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
from e91cc32 HDDS-2272. Avoid buffer copying in GrpcReplicationClient
add 9857b0c HDDS-2291. Acceptance tests for OM HA
No new revisions were added by this update.
Summary of changes:
.../.ssh/authorized_keys | 0
.../{ozonescripts => ozone-om-ha}/.ssh/config | 0
.../{ozonescripts => ozone-om-ha}/.ssh/environment | 0
.../{ozonescripts => ozone-om-ha}/.ssh/id_rsa | 0
.../{ozonescripts => ozone-om-ha}/.ssh/id_rsa.pub | 0
.../{ozonescripts => ozone-om-ha}/Dockerfile | 15 +-
.../main/compose/ozone-om-ha/docker-compose.yaml | 20 +--
.../src/main/compose/ozone-om-ha/docker-config | 58 ++-
.../{ozonescripts/ps.sh => ozone-om-ha/startOM.sh} | 13 +-
.../stop.sh => ozone-om-ha/stopOM.sh} | 5 +-
.../main/compose/{ozones3 => ozone-om-ha}/test.sh | 14 +-
.../dist/src/main/smoketest/omha/testOMHA.robot| 176 +
12 files changed, 278 insertions(+), 23 deletions(-)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts =>
ozone-om-ha}/.ssh/authorized_keys (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts =>
ozone-om-ha}/.ssh/config (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts =>
ozone-om-ha}/.ssh/environment (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts =>
ozone-om-ha}/.ssh/id_rsa (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts =>
ozone-om-ha}/.ssh/id_rsa.pub (100%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts =>
ozone-om-ha}/Dockerfile (81%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts/ps.sh =>
ozone-om-ha/startOM.sh} (79%)
copy hadoop-ozone/dist/src/main/compose/{ozonescripts/stop.sh =>
ozone-om-ha/stopOM.sh} (93%)
copy hadoop-ozone/dist/src/main/compose/{ozones3 => ozone-om-ha}/test.sh (73%)
mode change 100755 => 100644
create mode 100644 hadoop-ozone/dist/src/main/smoketest/omha/testOMHA.robot
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (9857b0c -> 6b2cda1)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
from 9857b0c HDDS-2291. Acceptance tests for OM HA
add 6b2cda1 HDDS-1515. Create ozone dev-support script to check hadolint
violiations
No new revisions were added by this update.
Summary of changes:
.../checks/{isolation.sh => hadolint.sh} | 28 --
1 file changed, 21 insertions(+), 7 deletions(-)
copy hadoop-ozone/dev-support/checks/{isolation.sh => hadolint.sh} (70%)
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (06c70f7 -> 384fee0)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 06c70f7 HDDS-2377. Speed up TestOzoneManagerHA#testOMRetryProxy and #testTwoOMNodesDown (#99) add 384fee0 HDDS-2369. Fix typo in param description. No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/hadoop/ozone/client/ObjectStore.java | 2 +- .../java/org/apache/hadoop/ozone/client/protocol/ClientProtocol.java| 2 +- .../src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java | 2 +- .../java/org/apache/hadoop/ozone/om/protocol/OzoneManagerProtocol.java | 2 +- .../ozone/om/protocolPB/OzoneManagerProtocolClientSideTranslatorPB.java | 2 +- .../src/main/java/org/apache/hadoop/ozone/om/BucketManagerImpl.java | 2 +- .../src/main/java/org/apache/hadoop/ozone/om/IOzoneAcl.java | 2 +- .../src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java| 2 +- .../src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java | 2 +- .../src/main/java/org/apache/hadoop/ozone/om/PrefixManagerImpl.java | 2 +- .../src/main/java/org/apache/hadoop/ozone/om/VolumeManagerImpl.java | 2 +- 11 files changed, 11 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (384fee0 -> dc3a3c3)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 384fee0 HDDS-2369. Fix typo in param description. add dc3a3c3 HDDS-2327. Provide new Freon test to test Ratis pipeline with pure XceiverClientRatis No new revisions were added by this update. Summary of changes: .../hadoop/ozone/freon/BaseFreonGenerator.java | 28 +++ .../hadoop/ozone/freon/DatanodeChunkGenerator.java | 190 + .../java/org/apache/hadoop/ozone/freon/Freon.java | 3 +- 3 files changed, 220 insertions(+), 1 deletion(-) create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/DatanodeChunkGenerator.java - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (dc3a3c3 -> d893cd3)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from dc3a3c3 HDDS-2327. Provide new Freon test to test Ratis pipeline with pure XceiverClientRatis add d893cd3 HDDS-2370. Support Ozone HddsDatanodeService run as plugin with HDFS Datanode No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/hadoop/hdds/HddsUtils.java | 11 --- .../java/org/apache/hadoop/ozone/HddsDatanodeService.java | 3 +++ hadoop-hdds/docs/content/beyond/RunningWithHDFS.md| 4 ++-- .../dist/src/main/compose/ozone-hdfs/docker-compose.yaml | 2 +- hadoop-ozone/dist/src/main/compose/ozone-hdfs/docker-config | 1 - 5 files changed, 14 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-docker-ozone] branch runner-latest updated (7619b7b -> 90a1a66)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch runner-latest in repository https://gitbox.apache.org/repos/asf/hadoop-docker-ozone.git. from 7619b7b HDDS-1799. Add goofyfs to the ozone-runner docker image add 90a1a66 HDDS-1701. Move dockerbin script to libexec No new revisions were added by this update. Summary of changes: Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-docker-ozone] tag runner-20191107-1 created (now 90a1a66)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to tag runner-20191107-1 in repository https://gitbox.apache.org/repos/asf/hadoop-docker-ozone.git. at 90a1a66 (commit) No new revisions were added by this update. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated: HDDS-2427. Exclude webapps from hadoop-ozone-filesystem-lib-current uber jar
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/master by this push: new 6a450fb HDDS-2427. Exclude webapps from hadoop-ozone-filesystem-lib-current uber jar 6a450fb is described below commit 6a450fbf59ef8f9c5005eb856ed9a4c38b5bcc10 Author: Bharat Viswanadham AuthorDate: Fri Nov 8 10:24:40 2019 +0100 HDDS-2427. Exclude webapps from hadoop-ozone-filesystem-lib-current uber jar Closes #129 --- hadoop-ozone/ozonefs-lib-current/pom.xml | 15 +++ 1 file changed, 15 insertions(+) diff --git a/hadoop-ozone/ozonefs-lib-current/pom.xml b/hadoop-ozone/ozonefs-lib-current/pom.xml index 1645ccc..9b5e601 100644 --- a/hadoop-ozone/ozonefs-lib-current/pom.xml +++ b/hadoop-ozone/ozonefs-lib-current/pom.xml @@ -74,6 +74,21 @@ shade + + + org.apache.hadoop:hadoop-hdds-server-framework + + webapps/datanode/** + webapps/static/** + + + + org.apache.hadoop:hadoop-hdds-container-service + +webapps/hddsDatanode/** + + + - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2400 created (now 7c54960)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2400 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at 7c54960 trying to trigger actions build with renaming workflow definition No new revisions were added by this update. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (6a450fb -> 9ce2dd7)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
from 6a450fb HDDS-2427. Exclude webapps from
hadoop-ozone-filesystem-lib-current uber jar
add 9ce2dd7 HDDS-1701. Move dockerbin script to libexec
No new revisions were added by this update.
Summary of changes:
hadoop-ozone/dist/dev-support/bin/dist-layout-stitching | 4 +++-
hadoop-ozone/dist/pom.xml | 2 +-
hadoop-ozone/dist/src/main/docker/Dockerfile | 2 ++
hadoop-ozone/dist/src/main/{dockerbin => dockerlibexec}/entrypoint.sh | 0
hadoop-ozone/dist/src/main/{dockerbin => dockerlibexec}/envtoconf.py | 0
.../dist/src/main/{dockerbin => dockerlibexec}/transformation.py | 0
6 files changed, 6 insertions(+), 2 deletions(-)
rename hadoop-ozone/dist/src/main/{dockerbin => dockerlibexec}/entrypoint.sh
(100%)
rename hadoop-ozone/dist/src/main/{dockerbin => dockerlibexec}/envtoconf.py
(100%)
rename hadoop-ozone/dist/src/main/{dockerbin =>
dockerlibexec}/transformation.py (100%)
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated: HDDS-2325. BenchMarkDatanodeDispatcher genesis test is failing with NPE
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/master by this push:
new a6f80c0 HDDS-2325. BenchMarkDatanodeDispatcher genesis test is
failing with NPE
a6f80c0 is described below
commit a6f80c096b5320f50b6e9e9b4ba5f7c7e3544385
Author: Márton Elek
AuthorDate: Mon Nov 11 19:09:55 2019 +0100
HDDS-2325. BenchMarkDatanodeDispatcher genesis test is failing with NPE
Closes #60
---
.../container/common/impl/HddsDispatcher.java | 5 ++
.../ozone/container/common/interfaces/Handler.java | 60
.../ozone/container/keyvalue/KeyValueHandler.java | 12 ++--
.../ozone/container/ozoneimpl/OzoneContainer.java | 59 ++--
.../container/common/impl/TestHddsDispatcher.java | 16 +++--
.../container/common/interfaces/TestHandler.java | 8 ++-
.../container/keyvalue/TestKeyValueHandler.java| 10 ++-
.../TestKeyValueHandlerWithUnhealthyContainer.java | 7 +-
.../container/metrics/TestContainerMetrics.java| 9 ++-
.../container/server/TestContainerServer.java | 7 +-
.../server/TestSecureContainerServer.java | 7 +-
.../ozone/genesis/BenchMarkDatanodeDispatcher.java | 82 --
.../org/apache/hadoop/ozone/genesis/Genesis.java | 3 +-
13 files changed, 169 insertions(+), 116 deletions(-)
diff --git
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java
index 7f46707..1feb500 100644
---
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java
+++
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java
@@ -321,6 +321,11 @@ public class HddsDispatcher implements
ContainerDispatcher, Auditor {
// Once container is marked unhealthy, all the subsequent write
// transactions will fail with UNHEALTHY_CONTAINER exception.
+if (container == null) {
+ throw new NullPointerException(
+ "Error on creating containers " + result + " " + responseProto
+ .getMessage());
+}
// For container to be moved to unhealthy state here, the container can
// only be in open or closing state.
State containerState = container.getContainerData().getState();
diff --git
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java
index 8c3b981..c623e14 100644
---
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java
+++
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java
@@ -18,25 +18,19 @@
package org.apache.hadoop.ozone.container.common.interfaces;
-
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
+import java.util.function.Consumer;
import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.hdds.protocol.DatanodeDetails;
-import org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos
-.ContainerCommandRequestProto;
-import org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos
-.ContainerCommandResponseProto;
-import org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos
-.ContainerType;
-import org.apache.hadoop.hdds.protocol.proto
-.StorageContainerDatanodeProtocolProtos.IncrementalContainerReportProto;
+import
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerCommandRequestProto;
+import
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerCommandResponseProto;
+import
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerType;
+import
org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.ContainerReplicaProto;
import
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException;
import org.apache.hadoop.ozone.container.common.helpers.ContainerMetrics;
import org.apache.hadoop.ozone.container.common.impl.ContainerSet;
-import org.apache.hadoop.ozone.container.common.statemachine.StateContext;
import
org.apache.hadoop.ozone.container.common.transport.server.ratis.DispatcherContext;
import org.apache.hadoop.ozone.container.common.volume.VolumeSet;
import org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler;
@@ -54,41 +48,46 @@ public abstract class Handler {
protected final VolumeSet volumeSet;
protected String scmID;
protected final ContainerMetrics metrics;
+ protect
[hadoop-ozone] branch HDDS-2400 updated (7c54960 -> 8025665)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2400 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 7c54960 trying to trigger actions build with renaming workflow definition add 8025665 fix shebang and remove docker daemon initialization No new revisions were added by this update. Summary of changes: .github/buildenv/entrypoint.sh | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (a6f80c0 -> 3b0a49e)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from a6f80c0 HDDS-2325. BenchMarkDatanodeDispatcher genesis test is failing with NPE add 3b0a49e HDDS-2415. Completely disable tracer if hdds.tracing.enabled=false No new revisions were added by this update. Summary of changes: .../apache/hadoop/hdds/tracing/StringCodec.java| 17 +++--- .../apache/hadoop/hdds/tracing/TracingUtil.java| 64 ++ .../apache/hadoop/ozone/HddsDatanodeService.java | 2 +- .../scm/server/StorageContainerManagerStarter.java | 5 +- .../hadoop/ozone/om/OzoneManagerStarter.java | 3 +- .../hadoop/ozone/web/ozShell/OzoneShell.java | 2 +- .../hadoop/ozone/web/ozShell/s3/S3Shell.java | 5 +- .../java/org/apache/hadoop/ozone/freon/Freon.java | 9 ++- 8 files changed, 51 insertions(+), 56 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (ee8b89a -> 1c1aecf)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from ee8b89a HDDS-1868. Ozone pipelines should be marked as ready only after the leader election is complete. (#23) add 1c1aecf HDDS-2456. Add explicit base image version for images derived from ozone-runner No new revisions were added by this update. Summary of changes: .../dist/src/main/compose/ozone-om-ha/Dockerfile | 4 ++- .../main/compose/ozone-om-ha/docker-compose.yaml | 30 ++ .../dist/src/main/compose/ozonescripts/Dockerfile | 6 - .../dist/src/main/compose/ozonescripts/README.md | 6 ++--- .../main/compose/ozonescripts/docker-compose.yaml | 18 ++--- 5 files changed, 51 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (1e71b63 -> 2211d63)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 1e71b63 HDDS-2462. Add jq dependency in Contribution guideline (#145) add 2211d63 HDDS-2460. Default checksum type is wrong in description No new revisions were added by this update. Summary of changes: hadoop-hdds/common/src/main/resources/ozone-default.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (2211d63 -> c199a29)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 2211d63 HDDS-2460. Default checksum type is wrong in description add c199a29 HDDS-2451. Use lazy string evaluation in preconditions No new revisions were added by this update. Summary of changes: .../common/src/main/java/org/apache/hadoop/hdds/HddsUtils.java| 2 +- .../commandhandler/ReplicateContainerCommandHandler.java | 4 ++-- .../common/transport/server/ratis/ContainerStateMachine.java | 8 .../protocolPB/OzoneManagerProtocolServerSideTranslatorPB.java| 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (c199a29 -> 4337bb0)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from c199a29 HDDS-2451. Use lazy string evaluation in preconditions add 4337bb0 HDDS-2403. Remove leftover reference to OUTPUT_FILE from shellcheck.sh No new revisions were added by this update. Summary of changes: hadoop-ozone/dev-support/checks/shellcheck.sh | 1 - 1 file changed, 1 deletion(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (4337bb0 -> ef2a9f4)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 4337bb0 HDDS-2403. Remove leftover reference to OUTPUT_FILE from shellcheck.sh add ef2a9f4 HDDS-2445. Replace ToStringBuilder in BlockData No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hdds/client/BlockID.java | 12 +- .../hadoop/hdds/client/ContainerBlockID.java | 13 +- .../ozone/container/common/helpers/BlockData.java | 32 ++-- .../container/common/helpers/TestBlockData.java| 12 ++ .../ozone/genesis/BenchmarkBlockDataToString.java | 166 + .../org/apache/hadoop/ozone/genesis/Genesis.java | 2 + 6 files changed, 219 insertions(+), 18 deletions(-) create mode 100644 hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchmarkBlockDataToString.java - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2461 created (now a1743ea)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2461 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at a1743ea fix logging name of ChunkManagerDummyImpl This branch includes the following new commits: new a1743ea fix logging name of ChunkManagerDummyImpl The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/01: fix logging name of ChunkManagerDummyImpl
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2461
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit a1743ea48824e77bd8ef92286c05fd47a8b557d6
Author: Márton Elek
AuthorDate: Thu Nov 14 12:06:24 2019 +0100
fix logging name of ChunkManagerDummyImpl
---
.../hadoop/ozone/container/keyvalue/impl/ChunkManagerDummyImpl.java | 5 +
.../hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java | 5 -
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerDummyImpl.java
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerDummyImpl.java
index a79829c..2927c22 100644
---
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerDummyImpl.java
+++
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerDummyImpl.java
@@ -34,6 +34,8 @@ import java.io.IOException;
import java.nio.ByteBuffer;
import static
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.Result.*;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* Implementation of ChunkManager built for running performance tests.
@@ -41,6 +43,9 @@ import static
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.Res
*/
public class ChunkManagerDummyImpl extends ChunkManagerImpl {
+ private static final Logger LOG =
+ LoggerFactory.getLogger(ChunkManagerDummyImpl.class);
+
public ChunkManagerDummyImpl(boolean sync) {
super(sync);
}
diff --git
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java
index e22841e..8a3575b 100644
---
a/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java
+++
b/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/ChunkManagerImpl.java
@@ -53,7 +53,10 @@ import static
org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.Res
* This class is for performing chunk related operations.
*/
public class ChunkManagerImpl implements ChunkManager {
- static final Logger LOG = LoggerFactory.getLogger(ChunkManagerImpl.class);
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(ChunkManagerImpl.class);
+
private final boolean doSyncWrite;
public ChunkManagerImpl(boolean sync) {
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (3ca37b1 -> a1ea8a4)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 3ca37b1 HDDS-2492 Fix test clean up issue in TestSCMPipelineManager. (#179) add a1ea8a4 HDDS-2468. scmcli close pipeline command not working No new revisions were added by this update. Summary of changes: ...eContainerLocationProtocolServerSideTranslatorPB.java | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (a1ea8a4 -> d05ff4e)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from a1ea8a4 HDDS-2468. scmcli close pipeline command not working add d05ff4e HDDS-2471. Improve exception message for CompleteMultipartUpload No new revisions were added by this update. Summary of changes: .../om/request/s3/multipart/S3MultipartUploadCompleteRequest.java | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (d05ff4e -> aa828b1)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from d05ff4e HDDS-2471. Improve exception message for CompleteMultipartUpload add aa828b1 HDDS-2470. Add partName, partNumber for CommitMultipartUpload No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/ozone/OzoneConsts.java | 2 ++ .../S3MultipartUploadCommitPartRequest.java| 23 ++ 2 files changed, 21 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (aa828b1 -> 5264882)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from aa828b1 HDDS-2470. Add partName, partNumber for CommitMultipartUpload add 5264882 HDDS-2495. Sonar - "notify" may not wake up the appropriate thread No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-docker-ozone] tag ozone-0.4.1 created (now 149f27e)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to tag ozone-0.4.1 in repository https://gitbox.apache.org/repos/asf/hadoop-docker-ozone.git. at 149f27e (commit) No new revisions were added by this update. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-docker-ozone] branch ozone-0.4.1 created (now 149f27e)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch ozone-0.4.1 in repository https://gitbox.apache.org/repos/asf/hadoop-docker-ozone.git. at 149f27e HDDS-2025. Update the base image of the apache/ozone and use latest 0.4.1 release. (#7) No new revisions were added by this update. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (8af5ab8 -> 89bdb6a)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 8af5ab8 HDDS-2525. Sonar : replace lambda with method reference in SCM BufferPool. (#210) add 89bdb6a HDDS-2521. Multipart upload failing with NPE No new revisions were added by this update. Summary of changes: .../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 141 ++--- .../hadoop/ozone/client/OzoneOutputStreamStub.java | 8 +- 2 files changed, 74 insertions(+), 75 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (e5a3b0c -> dcfe5f3)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from e5a3b0c HDDS-2509. Code cleanup in replication package (#185) add dcfe5f3 Revert "HDDS-2034. Async RATIS pipeline creation and destroy through heartbeat commands (#29)" No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hdds/HddsConfigKeys.java | 12 +- .../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 5 - .../org/apache/hadoop/hdds/utils/Scheduler.java| 7 +- .../common/src/main/resources/ozone-default.xml| 30 +-- .../common/statemachine/DatanodeStateMachine.java | 6 - .../CloseContainerCommandHandler.java | 11 +- .../ClosePipelineCommandHandler.java | 120 .../commandhandler/CommandHandler.java | 2 +- .../CreatePipelineCommandHandler.java | 135 - .../states/endpoint/HeartbeatEndpointTask.java | 22 --- .../common/transport/server/XceiverServerSpi.java | 18 -- .../transport/server/ratis/XceiverServerRatis.java | 36 .../protocol/commands/ClosePipelineCommand.java| 73 --- .../protocol/commands/CreatePipelineCommand.java | 100 -- .../proto/StorageContainerDatanodeProtocol.proto | 23 --- .../hadoop/hdds/scm/block/BlockManagerImpl.java| 3 - .../hdds/scm/container/ContainerStateManager.java | 1 - .../apache/hadoop/hdds/scm/events/SCMEvents.java | 12 +- .../scm/pipeline/BackgroundPipelineCreator.java| 9 +- .../hadoop/hdds/scm/pipeline/PipelineFactory.java | 13 +- .../hadoop/hdds/scm/pipeline/PipelineManager.java | 15 +- .../hadoop/hdds/scm/pipeline/PipelineProvider.java | 2 - .../hdds/scm/pipeline/PipelineReportHandler.java | 54 +++--- .../hdds/scm/pipeline/PipelineStateManager.java| 4 +- .../hdds/scm/pipeline/RatisPipelineProvider.java | 137 +- .../hdds/scm/pipeline/RatisPipelineUtils.java | 103 ++ .../hdds/scm/pipeline/SCMPipelineManager.java | 82 ++-- .../hdds/scm/pipeline/SCMPipelineMetrics.java | 10 - .../hdds/scm/pipeline/SimplePipelineProvider.java | 5 - .../scm/safemode/HealthyPipelineSafeModeRule.java | 83 .../safemode/OneReplicaPipelineSafeModeRule.java | 65 --- .../hdds/scm/safemode/SCMSafeModeManager.java | 16 +- .../hadoop/hdds/scm/safemode/SafeModeHandler.java | 5 +- .../scm/server/SCMDatanodeHeartbeatDispatcher.java | 1 - .../hdds/scm/server/SCMDatanodeProtocolServer.java | 20 -- .../hdds/scm/server/StorageContainerManager.java | 3 +- .../java/org/apache/hadoop/hdds/scm/TestUtils.java | 12 -- .../hadoop/hdds/scm/block/TestBlockManager.java| 44 + .../container/TestCloseContainerEventHandler.java | 11 +- .../scm/container/TestSCMContainerManager.java | 4 +- .../hdds/scm/node/TestContainerPlacement.java | 2 +- .../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 11 +- .../hadoop/hdds/scm/node/TestSCMNodeManager.java | 9 +- .../scm/pipeline/MockRatisPipelineProvider.java| 10 +- .../scm/pipeline/TestRatisPipelineProvider.java| 26 +-- .../safemode/TestHealthyPipelineSafeModeRule.java | 44 +++-- .../TestOneReplicaPipelineSafeModeRule.java| 36 +++- .../hdds/scm/safemode/TestSCMSafeModeManager.java | 50 +++-- hadoop-ozone/dist/src/main/compose/testlib.sh | 35 .../TestContainerStateManagerIntegration.java | 6 +- .../metrics/TestSCMContainerManagerMetrics.java| 3 - .../hdds/scm/pipeline/TestPipelineClose.java | 4 +- .../scm/pipeline/TestRatisPipelineProvider.java| 210 - .../hdds/scm/pipeline/TestSCMPipelineManager.java | 93 ++--- .../org/apache/hadoop/ozone/MiniOzoneCluster.java | 2 +- .../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 43 +++-- .../hadoop/ozone/TestContainerOperations.java | 2 +- .../TestContainerStateMachineIdempotency.java | 2 +- .../apache/hadoop/ozone/TestMiniOzoneCluster.java | 6 +- .../hadoop/ozone/TestStorageContainerManager.java | 18 +- .../apache/hadoop/ozone/client/rpc/TestBCSID.java | 3 - .../client/rpc/TestContainerStateMachine.java | 3 - .../rpc/TestContainerStateMachineFailures.java | 9 +- .../ozone/container/TestContainerReplication.java | 2 +- .../commandhandler/TestBlockDeletion.java | 6 +- .../commandhandler/TestCloseContainerHandler.java | 2 - .../commandhandler/TestDeleteContainerHandler.java | 2 - .../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 2 - .../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +- .../apache/hadoop/ozone/om/TestScmSafeMode.java| 2 +- .../hadoop/ozone/scm/TestContainerSmallFile.java | 2 +- .../scm/TestGetCommittedBlockLengthAndPutKey.java | 2 +- .../org/apache/hadoop/ozone/scm/TestSCMMXBean.
[hadoop-ozone] branch master updated (dcfe5f3 -> 87d5a5a)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
from dcfe5f3 Revert "HDDS-2034. Async RATIS pipeline creation and destroy
through heartbeat commands (#29)"
add 87d5a5a HDDS-2533. Disable failing acceptance and unit tests
No new revisions were added by this update.
Summary of changes:
.../test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java | 2 ++
hadoop-ozone/dist/src/main/compose/ozone-om-ha/{test.sh => run.sh} | 0
.../ozone/om/ratis/TestOzoneManagerDoubleBufferWithOMResponse.java | 2 ++
3 files changed, 4 insertions(+)
rename hadoop-ozone/dist/src/main/compose/ozone-om-ha/{test.sh => run.sh}
(100%)
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2531 created (now 9d21c88)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2531 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at 9d21c88 shell script fix from @adoroszlai This branch includes the following new commits: new 3e7e212 Revert "Revert "HDDS-2034. Async RATIS pipeline creation and destroy through heartbeat commands (#29)"" new 9d21c88 shell script fix from @adoroszlai The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/02: shell script fix from @adoroszlai
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2531
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 9d21c88ee596d5e87a8cfeaf664b4cbf3c8976e2
Author: Márton Elek
AuthorDate: Wed Nov 20 10:01:39 2019 +0100
shell script fix from @adoroszlai
---
hadoop-ozone/dist/src/main/compose/testlib.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-ozone/dist/src/main/compose/testlib.sh
b/hadoop-ozone/dist/src/main/compose/testlib.sh
index 684f2f5..49274c0 100755
--- a/hadoop-ozone/dist/src/main/compose/testlib.sh
+++ b/hadoop-ozone/dist/src/main/compose/testlib.sh
@@ -94,7 +94,7 @@ wait_for_safemode_exit(){
#This line checks the safemode status in scm
local command="ozone scmcli safemode status"
if [[ "${SECURITY_ENABLED}" == 'true' ]]; then
- status=`docker-compose -f "${compose_file}" exec -T scm bash -c
"kinit -k HTTP/s...@example.com -t /etc/security/keytabs/HTTP.keytab &&
$command'"`
+ status=`docker-compose -f "${compose_file}" exec -T scm bash -c
"kinit -k HTTP/s...@example.com -t /etc/security/keytabs/HTTP.keytab &&
$command"`
else
status=`docker-compose -f "${compose_file}" exec -T scm bash -c
"$command"`
fi
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 created (now 03baab0)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at 03baab0 enable sonar cloud integration (other tests are disabled temporary) This branch includes the following new commits: new 03baab0 enable sonar cloud integration (other tests are disabled temporary) The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/01: enable sonar cloud integration (other tests are disabled temporary)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 03baab038282c8896c3dd89fe08396c406ab7490
Author: Márton Elek
AuthorDate: Wed Nov 20 11:33:56 2019 +0100
enable sonar cloud integration (other tests are disabled temporary)
---
.github/workflows/main.yml | 171 ---
hadoop-ozone/dev-support/checks/sonar.sh | 23 +
pom.xml | 5 +
3 files changed, 116 insertions(+), 83 deletions(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index c64ac69..d82e32d 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -17,40 +17,40 @@ on:
- push
- pull_request
jobs:
- build:
-name: compile
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/build.sh
- rat:
-name: rat
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/rat.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: rat
-path: target/rat
- author:
-name: author
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/author.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: author
-path: target/author
+# build:
+#name: compile
+#runs-on: ubuntu-18.04
+#steps:
+#- uses: actions/checkout@master
+#- uses: ./.github/buildenv
+# with:
+# args: ./hadoop-ozone/dev-support/checks/build.sh
+# rat:
+#name: rat
+#runs-on: ubuntu-18.04
+#steps:
+#- uses: actions/checkout@master
+#- uses: ./.github/buildenv
+# with:
+# args: ./hadoop-ozone/dev-support/checks/rat.sh
+#- uses: actions/upload-artifact@master
+# if: always()
+# with:
+#name: rat
+#path: target/rat
+# author:
+#name: author
+#runs-on: ubuntu-18.04
+#steps:
+#- uses: actions/checkout@master
+#- uses: ./.github/buildenv
+# with:
+# args: ./hadoop-ozone/dev-support/checks/author.sh
+#- uses: actions/upload-artifact@master
+# if: always()
+# with:
+#name: author
+#path: target/author
unit:
name: unit
runs-on: ubuntu-18.04
@@ -59,57 +59,62 @@ jobs:
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/unit.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: unit
-path: target/unit
- checkstyle:
-name: checkstyle
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
- uses: ./.github/buildenv
with:
- args: ./hadoop-ozone/dev-support/checks/checkstyle.sh
+args: ./hadoop-ozone/dev-support/checks/sonar.sh
+ env:
+SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
- uses: actions/upload-artifact@master
if: always()
with:
-name: checkstyle
-path: target/checkstyle
- findbugs:
-name: findbugs
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/findbugs.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: findbugs
-path: target/findbugs
- acceptance:
-name: acceptance
-runs-on: ubuntu-18.04
-needs:
- - build
- - rat
- - checkstyle
- - unit
- - findbugs
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
-args: ./hadoop-ozone/dev-support/checks/build.sh
-- run: sudo pip install robotframework
-- run: sudo chown runner -R .
-- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
-- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: acceptance
-path: target/acceptance
+name: unit
+path: target/unit
+# checkstyle:
+#name: checkstyle
+#runs-on: ubuntu-18.04
+#steps:
+#- uses: actions/checkout@master
+#- uses: ./.github/buildenv
[hadoop-ozone] branch HDDS-2587 updated: fix the name of the sonarcloud secret
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
View the commit online:
https://github.com/apache/hadoop-ozone/commit/9569d9b88a89b9d7dc01fe1736c2d5968463c804
The following commit(s) were added to refs/heads/HDDS-2587 by this push:
new 9569d9b fix the name of the sonarcloud secret
9569d9b is described below
commit 9569d9b88a89b9d7dc01fe1736c2d5968463c804
Author: Márton Elek
AuthorDate: Wed Nov 20 14:30:38 2019 +0100
fix the name of the sonarcloud secret
---
.github/workflows/main.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index d82e32d..ed2d406 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -63,7 +63,7 @@ jobs:
with:
args: ./hadoop-ozone/dev-support/checks/sonar.sh
env:
-SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
- uses: actions/upload-artifact@master
if: always()
with:
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (e68b113 -> 2134dec)
This is an automated email from the ASF dual-hosted git repository.
elek pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git.
from e68b113 HDDS-2493. Sonar: Locking on a parameter in
NetUtils.removeOutscope. (#174)
add 2134dec HDDS-2485. Disable XML external entity processing
No new revisions were added by this update.
Summary of changes:
.../hadoop/hdds/scm/net/NodeSchemaLoader.java | 27 ++--
.../hadoop/hdds/scm/net/TestNodeSchemaLoader.java | 151 +
.../src/test/resources/log4j.properties| 0
.../{good.xml => external-entity.xml} | 7 +-
.../hadoop/hdds/conf/ConfigFileAppender.java | 21 +--
.../endpoint/MultiDeleteRequestUnmarshaller.java | 8 +-
6 files changed, 125 insertions(+), 89 deletions(-)
copy hadoop-hdds/{container-service =>
common}/src/test/resources/log4j.properties (100%)
copy hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/{good.xml
=> external-entity.xml} (93%)
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated: github token is required by sonar cloud
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
View the commit online:
https://github.com/apache/hadoop-ozone/commit/df04096b7be58f76363e789e51af15e47d89b443
The following commit(s) were added to refs/heads/HDDS-2587 by this push:
new df04096 github token is required by sonar cloud
df04096 is described below
commit df04096b7be58f76363e789e51af15e47d89b443
Author: Márton Elek
AuthorDate: Wed Nov 20 15:11:50 2019 +0100
github token is required by sonar cloud
---
.github/workflows/main.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index ed2d406..c6f2ee5 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -64,6 +64,7 @@ jobs:
args: ./hadoop-ozone/dev-support/checks/sonar.sh
env:
SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
+GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/upload-artifact@master
if: always()
with:
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2034 created (now 9d21c88)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2034 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at 9d21c88 shell script fix from @adoroszlai No new revisions were added by this update. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2034 updated: fix TestDeadNodeHandler
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2034
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
View the commit online:
https://github.com/apache/hadoop-ozone/commit/724e868805944c5c2a99cf7a50adcaccda99e563
The following commit(s) were added to refs/heads/HDDS-2034 by this push:
new 724e868 fix TestDeadNodeHandler
724e868 is described below
commit 724e868805944c5c2a99cf7a50adcaccda99e563
Author: Márton Elek
AuthorDate: Wed Nov 20 16:25:12 2019 +0100
fix TestDeadNodeHandler
---
.../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 48 --
pom.xml| 2 +-
2 files changed, 36 insertions(+), 14 deletions(-)
diff --git
a/hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
b/hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
index 1676af1..24e3870 100644
---
a/hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
+++
b/hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java
@@ -30,6 +30,8 @@ import org.apache.hadoop.hdds.HddsConfigKeys;
import org.apache.hadoop.hdds.conf.OzoneConfiguration;
import org.apache.hadoop.hdds.protocol.DatanodeDetails;
import org.apache.hadoop.hdds.protocol.proto.HddsProtos;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos.ReplicationFactor;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos.ReplicationType;
import org.apache.hadoop.hdds.protocol.proto
.StorageContainerDatanodeProtocolProtos.ContainerReplicaProto;
import org.apache.hadoop.hdds.protocol.proto
@@ -43,7 +45,10 @@ import org.apache.hadoop.hdds.scm.container.ContainerManager;
import org.apache.hadoop.hdds.scm.container.ContainerNotFoundException;
import org.apache.hadoop.hdds.scm.container.ContainerReplica;
import org.apache.hadoop.hdds.scm.container.ContainerInfo;
+import org.apache.hadoop.hdds.scm.container.SCMContainerManager;
import org.apache.hadoop.hdds.scm.events.SCMEvents;
+import org.apache.hadoop.hdds.scm.net.NetworkTopology;
+import org.apache.hadoop.hdds.scm.net.NetworkTopologyImpl;
import org.apache.hadoop.hdds.scm.node.states.NodeNotFoundException;
import org.apache.hadoop.hdds.scm.pipeline.PipelineManager;
import org.apache.hadoop.hdds.scm.pipeline.PipelineProvider;
@@ -51,6 +56,7 @@ import org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager;
import org.apache.hadoop.hdds.scm.pipeline.MockRatisPipelineProvider;
import org.apache.hadoop.hdds.scm.server.SCMDatanodeHeartbeatDispatcher
.NodeReportFromDatanode;
+import org.apache.hadoop.hdds.scm.server.SCMStorageConfig;
import org.apache.hadoop.hdds.scm.server.StorageContainerManager;
import org.apache.hadoop.hdds.server.events.EventPublisher;
@@ -58,22 +64,23 @@ import org.apache.hadoop.hdds.server.events.EventQueue;
import org.apache.hadoop.security.authentication.client
.AuthenticationException;
import org.apache.hadoop.test.GenericTestUtils;
+
+import org.apache.commons.io.IOUtils;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import org.mockito.Mockito;
+import org.slf4j.LoggerFactory;
/**
* Test DeadNodeHandler.
*/
public class TestDeadNodeHandler {
- private StorageContainerManager scm;
private SCMNodeManager nodeManager;
private ContainerManager containerManager;
- private NodeReportHandler nodeReportHandler;
private SCMPipelineManager pipelineManager;
private DeadNodeHandler deadNodeHandler;
private EventPublisher publisher;
@@ -87,32 +94,40 @@ public class TestDeadNodeHandler {
TestDeadNodeHandler.class.getSimpleName() + UUID.randomUUID());
conf.set(HddsConfigKeys.OZONE_METADATA_DIRS, storageDir);
eventQueue = new EventQueue();
-scm = HddsTestUtils.getScm(conf);
-nodeManager = (SCMNodeManager) scm.getScmNodeManager();
-pipelineManager =
-(SCMPipelineManager)scm.getPipelineManager();
+
+NetworkTopology topology = Mockito.mock(NetworkTopology.class);
+SCMStorageConfig storageConfig = Mockito.mock(SCMStorageConfig.class);
+Mockito.when(storageConfig.getScmId()).thenReturn("SCM-ID");
+Mockito.when(storageConfig.getClusterID()).thenReturn("CLUSTER_ID");
+
+nodeManager = new SCMNodeManager(conf, storageConfig, eventQueue,
topology);
+
+pipelineManager = new SCMPipelineManager(conf, nodeManager, eventQueue);
+
PipelineProvider mockRatisProvider =
new MockRatisPipelineProvider(nodeManager,
pipelineManager.getStateManager(), conf);
+
pipelineManager.setPipelineProvider(HddsProtos.ReplicationType.RATIS,
mockRatisProvider);
-containerManager = scm.getContainerManager();
+
+containerManager =
+new SCMContainerManager(conf, nodeManager, pipeline
[hadoop-ozone] 02/02: remove properties from pom.xml
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git View the commit online: https://github.com/apache/hadoop-ozone/commit/dfba77e9bfcbfa5a4cd5ad2e7118fab939d26c7e commit dfba77e9bfcbfa5a4cd5ad2e7118fab939d26c7e Author: Márton Elek AuthorDate: Wed Nov 20 16:52:08 2019 +0100 remove properties from pom.xml --- pom.xml | 5 - 1 file changed, 5 deletions(-) diff --git a/pom.xml b/pom.xml index a818cb9..4c03e2b 100644 --- a/pom.xml +++ b/pom.xml @@ -229,11 +229,6 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xs 1.16 1.2.6 2.0.0-beta-1 - - -https://sonarcloud.io -apache - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated (df04096 -> dfba77e)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from df04096 github token is required by sonar cloud new 3fa199b add properties to the shell script new dfba77e remove properties from pom.xml The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: hadoop-ozone/dev-support/checks/sonar.sh | 2 +- pom.xml | 5 - 2 files changed, 1 insertion(+), 6 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/02: add properties to the shell script
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git View the commit online: https://github.com/apache/hadoop-ozone/commit/3fa199baf224966e2bbaa410d7f336ad82b204c1 commit 3fa199baf224966e2bbaa410d7f336ad82b204c1 Author: Márton Elek AuthorDate: Wed Nov 20 16:51:38 2019 +0100 add properties to the shell script --- hadoop-ozone/dev-support/checks/sonar.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hadoop-ozone/dev-support/checks/sonar.sh b/hadoop-ozone/dev-support/checks/sonar.sh index 74d909c..135a47e 100755 --- a/hadoop-ozone/dev-support/checks/sonar.sh +++ b/hadoop-ozone/dev-support/checks/sonar.sh @@ -20,4 +20,4 @@ if [ ! "$SONAR_TOKEN" ]; then echo "SONAR_TOKEN environment variable should be set" exit 1 fi -mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar +mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated: set the location of the classes
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git View the commit online: https://github.com/apache/hadoop-ozone/commit/dd792a3e699b8e21e131cc8d1ece18970a47261c The following commit(s) were added to refs/heads/HDDS-2587 by this push: new dd792a3 set the location of the classes dd792a3 is described below commit dd792a3e699b8e21e131cc8d1ece18970a47261c Author: Márton Elek AuthorDate: Wed Nov 20 20:00:39 2019 +0100 set the location of the classes --- hadoop-ozone/dev-support/checks/sonar.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hadoop-ozone/dev-support/checks/sonar.sh b/hadoop-ozone/dev-support/checks/sonar.sh index 135a47e..719c381 100755 --- a/hadoop-ozone/dev-support/checks/sonar.sh +++ b/hadoop-ozone/dev-support/checks/sonar.sh @@ -20,4 +20,4 @@ if [ ! "$SONAR_TOKEN" ]; then echo "SONAR_TOKEN environment variable should be set" exit 1 fi -mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone +mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone -Dsonar.java.binaries=target/classes - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated: install before sonar:sonar
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/HDDS-2587 by this push: new 376856d install before sonar:sonar 376856d is described below commit 376856d9f430a3a4c5d13cc5ddfc325f1ceab139 Author: Márton Elek AuthorDate: Thu Nov 21 10:37:54 2019 +0100 install before sonar:sonar --- .github/workflows/main.yml | 6 +++--- hadoop-ozone/dev-support/checks/sonar.sh | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 0a1608a..2b52480 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -56,9 +56,9 @@ jobs: runs-on: ubuntu-18.04 steps: - uses: actions/checkout@master -- uses: ./.github/buildenv - with: - args: ./hadoop-ozone/dev-support/checks/build.sh +#- uses: ./.github/buildenv +# with: +# args: ./hadoop-ozone/dev-support/checks/build.sh - uses: ./.github/buildenv with: args: ./hadoop-ozone/dev-support/checks/sonar.sh diff --git a/hadoop-ozone/dev-support/checks/sonar.sh b/hadoop-ozone/dev-support/checks/sonar.sh index 135a47e..68edb7e 100755 --- a/hadoop-ozone/dev-support/checks/sonar.sh +++ b/hadoop-ozone/dev-support/checks/sonar.sh @@ -20,4 +20,4 @@ if [ ! "$SONAR_TOKEN" ]; then echo "SONAR_TOKEN environment variable should be set" exit 1 fi -mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone +mvn install org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated: skip tests
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/HDDS-2587 by this push: new c0e1710 skip tests c0e1710 is described below commit c0e17103d80690b46cac402ea8325ef202986f8b Author: Márton Elek AuthorDate: Thu Nov 21 10:43:48 2019 +0100 skip tests --- hadoop-ozone/dev-support/checks/sonar.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hadoop-ozone/dev-support/checks/sonar.sh b/hadoop-ozone/dev-support/checks/sonar.sh index 68edb7e..670a11d 100755 --- a/hadoop-ozone/dev-support/checks/sonar.sh +++ b/hadoop-ozone/dev-support/checks/sonar.sh @@ -20,4 +20,4 @@ if [ ! "$SONAR_TOKEN" ]; then echo "SONAR_TOKEN environment variable should be set" exit 1 fi -mvn install org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone +mvn install -DskipTests org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/HDDS-2587 by this push: new 2f27d93 retrigger build 2f27d93 is described below commit 2f27d93b87dc73c229115c90cb14008c5df11e3b Author: Márton Elek AuthorDate: Thu Nov 21 11:03:10 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/02: separate PR and main branch build definition
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit b42d9c26080649596857941ed5372e1e5f4294aa
Author: Márton Elek
AuthorDate: Thu Nov 21 15:35:00 2019 +0100
separate PR and main branch build definition
---
.github/workflows/main.yml | 121 ---
.github/workflows/post-commit.yml| 112
.github/workflows/pr.yml | 106 +++
hadoop-ozone/dev-support/checks/sonar.sh | 2 +-
4 files changed, 219 insertions(+), 122 deletions(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
deleted file mode 100644
index 2b52480..000
--- a/.github/workflows/main.yml
+++ /dev/null
@@ -1,121 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-name: Build
-on:
- - push
- - pull_request
-jobs:
-# build:
-#name: compile
-#runs-on: ubuntu-18.04
-#steps:
-#- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-# args: ./hadoop-ozone/dev-support/checks/build.sh
-# rat:
-#name: rat
-#runs-on: ubuntu-18.04
-#steps:
-#- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-# args: ./hadoop-ozone/dev-support/checks/rat.sh
-#- uses: actions/upload-artifact@master
-# if: always()
-# with:
-#name: rat
-#path: target/rat
-# author:
-#name: author
-#runs-on: ubuntu-18.04
-#steps:
-#- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-# args: ./hadoop-ozone/dev-support/checks/author.sh
-#- uses: actions/upload-artifact@master
-# if: always()
-# with:
-#name: author
-#path: target/author
- sonar:
-name: sonar
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-# args: ./hadoop-ozone/dev-support/checks/build.sh
-- uses: ./.github/buildenv
- with:
-args: ./hadoop-ozone/dev-support/checks/sonar.sh
- env:
-SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
-GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: sonar
-path: target/sonar
-# checkstyle:
-#name: checkstyle
-#runs-on: ubuntu-18.04
-#steps:
-#- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-# args: ./hadoop-ozone/dev-support/checks/checkstyle.sh
-#- uses: actions/upload-artifact@master
-# if: always()
-# with:
-#name: checkstyle
-#path: target/checkstyle
-# findbugs:
-#name: findbugs
-#runs-on: ubuntu-18.04
-#steps:
-#- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-# args: ./hadoop-ozone/dev-support/checks/findbugs.sh
-#- uses: actions/upload-artifact@master
-# if: always()
-# with:
-#name: findbugs
-#path: target/findbugs
-# acceptance:
-#name: acceptance
-#runs-on: ubuntu-18.04
-#needs:
-# - build
-# - rat
-# - checkstyle
-# - unit
-# - findbugs
-#steps:
-#- uses: actions/checkout@master
-#- uses: ./.github/buildenv
-# with:
-#args: ./hadoop-ozone/dev-support/checks/build.sh
-#- run: sudo pip install robotframework
-#- run: sudo chown runner -R .
-#- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
-#- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
-#- uses: actions/upload-artifact@master
-# if: always()
-# with:
-#name: acceptance
-#path: target/acceptance
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-c
[hadoop-ozone] 01/02: revert files added by a merge to this branch
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit d1dfbc4cb3102b6687e97a527e9d67a13d0bda8c
Author: Márton Elek
AuthorDate: Thu Nov 21 15:23:37 2019 +0100
revert files added by a merge to this branch
---
.../hdds/scm/client/ContainerOperationClient.java | 2 +-
.../hadoop/hdds/scm/client/HddsClientUtils.java| 5 +
.../apache/hadoop/hdds/scm/storage/BufferPool.java | 2 +-
.../hadoop/hdds/scm/storage/CommitWatcher.java | 4 +-
.../hadoop/hdds/scm/net/NodeSchemaLoader.java | 27 +-
.../apache/hadoop/hdds/tracing/TraceAllMethod.java | 5 -
.../common/src/main/resources/ozone-default.xml| 11 +-
.../hadoop/hdds/scm/net/TestNodeSchemaLoader.java | 151
.../common/src/test/resources/log4j.properties | 23 --
.../networkTopologyTestFiles/external-entity.xml | 52 ---
.../hadoop/hdds/conf/ConfigFileAppender.java | 21 +-
.../hdds/scm/server/StorageContainerManager.java | 74 ++--
.../org/apache/hadoop/ozone/om/OMConfigKeys.java | 3 -
.../hadoop/ozone/om/exceptions/OMException.java| 6 +-
.../src/main/proto/OzoneManagerProtocol.proto | 1 -
.../hadoop/ozone/MiniOzoneHAClusterImpl.java | 6 +-
.../hadoop/ozone/TestSecureOzoneCluster.java | 399 -
.../hadoop/ozone/client/rpc/TestCommitWatcher.java | 9 +-
.../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 133 +--
.../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 66 ++--
.../apache/hadoop/ozone/om/ha/OMHANodeDetails.java | 21 +-
.../ozone/web/ozShell/keys/GetKeyHandler.java | 44 +--
.../ozone/web/ozShell/keys/PutKeyHandler.java | 35 +-
.../endpoint/MultiDeleteRequestUnmarshaller.java | 8 +-
.../ozone/s3/util/RangeHeaderParserUtil.java | 4 +-
.../ozone/s3/util/TestRangeHeaderParserUtil.java | 6 +-
.../ozone/audit/parser/common/DatabaseHelper.java | 157
.../audit/parser/handler/LoadCommandHandler.java | 11 +-
.../audit/parser/handler/QueryCommandHandler.java | 11 +-
.../parser/handler/TemplateCommandHandler.java | 11 +-
30 files changed, 554 insertions(+), 754 deletions(-)
diff --git
a/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/ContainerOperationClient.java
b/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/ContainerOperationClient.java
index 6e2fd59..b1d4538 100644
---
a/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/ContainerOperationClient.java
+++
b/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/ContainerOperationClient.java
@@ -221,7 +221,7 @@ public class ContainerOperationClient implements ScmClient {
// That makes sense, but it is not needed for the client to work.
if (LOG.isDebugEnabled()) {
LOG.debug("Pipeline creation successful. Pipeline: {}",
- pipeline);
+ pipeline.toString());
}
}
diff --git
a/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java
b/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java
index d7e3f4d..8c9a11a 100644
---
a/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java
+++
b/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java
@@ -49,6 +49,8 @@ import org.apache.ratis.protocol.AlreadyClosedException;
import org.apache.ratis.protocol.GroupMismatchException;
import org.apache.ratis.protocol.NotReplicatedException;
import org.apache.ratis.protocol.RaftRetryFailureException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.InetSocketAddress;
@@ -74,6 +76,9 @@ import java.util.concurrent.TimeoutException;
@InterfaceStability.Unstable
public final class HddsClientUtils {
+ private static final Logger LOG = LoggerFactory.getLogger(
+ HddsClientUtils.class);
+
private HddsClientUtils() {
}
diff --git
a/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java
b/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java
index f57700e..f986fe3 100644
---
a/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java
+++
b/hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BufferPool.java
@@ -94,7 +94,7 @@ public class BufferPool {
// always remove from head of the list and append at last
final ChunkBuffer buffer = bufferList.remove(0);
// Ensure the buffer to be removed is always at the head of the list.
-Preconditions.checkArgument(buffer == chunkBuffer);
+Preconditions.checkArgument(buffer.equals(chunkBuffer));
buffer.clear();
bufferList.add(buffer);
Preconditions.checkArgument(currentBufferIndex >= 0);
diff --git
a/hadoop-hdds/cl
[hadoop-ozone] branch HDDS-2587 updated (2f27d93 -> b42d9c2)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 2f27d93 retrigger build new d1dfbc4 revert files added by a merge to this branch new b42d9c2 separate PR and main branch build definition The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/main.yml | 121 --- .github/workflows/post-commit.yml | 112 ++ .github/workflows/pr.yml | 106 ++ .../hdds/scm/client/ContainerOperationClient.java | 2 +- .../hadoop/hdds/scm/client/HddsClientUtils.java| 5 + .../apache/hadoop/hdds/scm/storage/BufferPool.java | 2 +- .../hadoop/hdds/scm/storage/CommitWatcher.java | 4 +- .../hadoop/hdds/scm/net/NodeSchemaLoader.java | 27 +- .../apache/hadoop/hdds/tracing/TraceAllMethod.java | 5 - .../common/src/main/resources/ozone-default.xml| 11 +- .../hadoop/hdds/scm/net/TestNodeSchemaLoader.java | 151 .../common/src/test/resources/log4j.properties | 23 -- .../networkTopologyTestFiles/external-entity.xml | 52 --- .../hadoop/hdds/conf/ConfigFileAppender.java | 21 +- .../hdds/scm/server/StorageContainerManager.java | 74 ++-- .../org/apache/hadoop/ozone/om/OMConfigKeys.java | 3 - .../hadoop/ozone/om/exceptions/OMException.java| 6 +- .../src/main/proto/OzoneManagerProtocol.proto | 1 - hadoop-ozone/dev-support/checks/sonar.sh | 2 +- .../hadoop/ozone/MiniOzoneHAClusterImpl.java | 6 +- .../hadoop/ozone/TestSecureOzoneCluster.java | 399 - .../hadoop/ozone/client/rpc/TestCommitWatcher.java | 9 +- .../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 133 +-- .../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 66 ++-- .../apache/hadoop/ozone/om/ha/OMHANodeDetails.java | 21 +- .../ozone/web/ozShell/keys/GetKeyHandler.java | 44 +-- .../ozone/web/ozShell/keys/PutKeyHandler.java | 35 +- .../endpoint/MultiDeleteRequestUnmarshaller.java | 8 +- .../ozone/s3/util/RangeHeaderParserUtil.java | 4 +- .../ozone/s3/util/TestRangeHeaderParserUtil.java | 6 +- .../ozone/audit/parser/common/DatabaseHelper.java | 157 .../audit/parser/handler/LoadCommandHandler.java | 11 +- .../audit/parser/handler/QueryCommandHandler.java | 11 +- .../parser/handler/TemplateCommandHandler.java | 11 +- 34 files changed, 773 insertions(+), 876 deletions(-) delete mode 100644 .github/workflows/main.yml create mode 100644 .github/workflows/post-commit.yml create mode 100644 .github/workflows/pr.yml delete mode 100644 hadoop-hdds/common/src/test/resources/log4j.properties delete mode 100644 hadoop-hdds/common/src/test/resources/networkTopologyTestFiles/external-entity.xml - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/03: Sonar is executed with maven batch mode
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 0d9a83087a00cf89c350178bb5a7c3154f4f1130 Author: Márton Elek AuthorDate: Thu Nov 21 15:43:51 2019 +0100 Sonar is executed with maven batch mode --- hadoop-ozone/dev-support/checks/sonar.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hadoop-ozone/dev-support/checks/sonar.sh b/hadoop-ozone/dev-support/checks/sonar.sh index 0282b15..55f219b 100755 --- a/hadoop-ozone/dev-support/checks/sonar.sh +++ b/hadoop-ozone/dev-support/checks/sonar.sh @@ -20,4 +20,4 @@ if [ ! "$SONAR_TOKEN" ]; then echo "SONAR_TOKEN environment variable should be set" exit 1 fi -mvn verify -DskipShade -DskipTests -Dskip.yarn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone +mvn -B verify -DskipShade -DskipTests -Dskip.yarn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -Dsonar.host.url=https://sonarcloud.io -Dsonar.organization=apache -Dsonar.projectKey=hadoop-ozone - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 03/03: fix unit test
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 36bc4df6a9719c650a68a25b17feeec9ccde19f7
Author: Márton Elek
AuthorDate: Thu Nov 21 17:15:16 2019 +0100
fix unit test
---
hadoop-ozone/dev-support/checks/unit.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-ozone/dev-support/checks/unit.sh
b/hadoop-ozone/dev-support/checks/unit.sh
index 585375e..adfef53 100755
--- a/hadoop-ozone/dev-support/checks/unit.sh
+++ b/hadoop-ozone/dev-support/checks/unit.sh
@@ -17,7 +17,7 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1
&& pwd )"
cd "$DIR/../../.." || exit 1
export MAVEN_OPTS="-Xmx4096m"
-mvn -B -DskipShade -Dskip.yarn -fn test -pl
\!:hadoop-ozone-integration-test,\!:hadoop-ozone-filesystem,\!:hadoop-ozone-tools
"$@"
+mvn -B -DskipShade -Dskip.yarn -fn test -pl
\!:hadoop-ozone-integration-test,\!:hadoop-ozone-tools "$@"
REPORT_DIR=${OUTPUT_DIR:-"$DIR/../../../target/unit"}
mkdir -p "$REPORT_DIR"
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2587 updated (b42d9c2 -> 36bc4df)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2587 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from b42d9c2 separate PR and main branch build definition new 9d3a845 coverage report is turned on new 0d9a830 Sonar is executed with maven batch mode new 36bc4df fix unit test The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: hadoop-hdds/pom.xml | 3 ++- hadoop-ozone/dev-support/checks/sonar.sh | 2 +- hadoop-ozone/dev-support/checks/unit.sh | 2 +- hadoop-ozone/pom.xml | 3 ++- pom.xml | 20 +++- 5 files changed, 25 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/03: coverage report is turned on
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-2587
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 9d3a84518a3e74dcd9598a7672e13ee20062313b
Author: Márton Elek
AuthorDate: Thu Nov 21 15:43:15 2019 +0100
coverage report is turned on
---
hadoop-hdds/pom.xml | 3 ++-
hadoop-ozone/dev-support/checks/unit.sh | 2 +-
hadoop-ozone/pom.xml| 3 ++-
pom.xml | 20 +++-
4 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/hadoop-hdds/pom.xml b/hadoop-hdds/pom.xml
index 9cd5db3..882755a 100644
--- a/hadoop-hdds/pom.xml
+++ b/hadoop-hdds/pom.xml
@@ -369,7 +369,8 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd";>
${testsThreadCount}
false
- ${maven-surefire-plugin.argLine}
-DminiClusterDedicatedDirs=true
+
+ ${maven-surefire-plugin.argLine} @{argLine}
-DminiClusterDedicatedDirs=true
${testsThreadCount}
${test.build.data}/${surefire.forkNumber}
diff --git a/hadoop-ozone/dev-support/checks/unit.sh
b/hadoop-ozone/dev-support/checks/unit.sh
index 44b6d0b..585375e 100755
--- a/hadoop-ozone/dev-support/checks/unit.sh
+++ b/hadoop-ozone/dev-support/checks/unit.sh
@@ -17,7 +17,7 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1
&& pwd )"
cd "$DIR/../../.." || exit 1
export MAVEN_OPTS="-Xmx4096m"
-mvn -B -fn test -pl
\!:hadoop-ozone-integration-test,\!:hadoop-ozone-filesystem,\!:hadoop-ozone-tools
"$@"
+mvn -B -DskipShade -Dskip.yarn -fn test -pl
\!:hadoop-ozone-integration-test,\!:hadoop-ozone-filesystem,\!:hadoop-ozone-tools
"$@"
REPORT_DIR=${OUTPUT_DIR:-"$DIR/../../../target/unit"}
mkdir -p "$REPORT_DIR"
diff --git a/hadoop-ozone/pom.xml b/hadoop-ozone/pom.xml
index 6d8b465..8a19298 100644
--- a/hadoop-ozone/pom.xml
+++ b/hadoop-ozone/pom.xml
@@ -397,7 +397,8 @@
${testsThreadCount}
false
- ${maven-surefire-plugin.argLine}
-DminiClusterDedicatedDirs=true
+
+ ${maven-surefire-plugin.argLine} @{argLine}
-DminiClusterDedicatedDirs=true
${testsThreadCount}
${test.build.data}/${surefire.forkNumber}
diff --git a/pom.xml b/pom.xml
index d6e2da6..c8b16c1 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1670,6 +1670,11 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xs
apache-rat-plugin
${apache-rat-plugin.version}
+
+ org.jacoco
+ jacoco-maven-plugin
+ 0.8.3
+
@@ -1760,12 +1765,25 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xs
+org.jacoco
+jacoco-maven-plugin
+
+
+default-prepare-agent
+
+ prepare-agent
+
+
+
+
+
org.apache.maven.plugins
maven-surefire-plugin
false
${surefire.fork.timeout}
- ${maven-surefire-plugin.argLine}
+
+ ${maven-surefire-plugin.argLine} @{argLine}
${hadoop.common.build.dir}
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/01: HDDS-2627. Skip sonar check in forks
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch HDDS-2627 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 90f7f060d29ebace7e3154fa852a303c0cd4c205 Author: Doroszlai, Attila AuthorDate: Mon Nov 25 13:56:13 2019 +0100 HDDS-2627. Skip sonar check in forks Closes #266 --- .github/workflows/post-commit.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index b10d270..5097be4 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -59,6 +59,7 @@ jobs: with: args: ./hadoop-ozone/dev-support/checks/unit.sh - uses: ./.github/buildenv + if: github.repository == 'apache/hadoop-ozone' with: args: ./hadoop-ozone/dev-support/checks/sonar.sh env: - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-2627 created (now 90f7f06)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch HDDS-2627 in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at 90f7f06 HDDS-2627. Skip sonar check in forks This branch includes the following new commits: new 90f7f06 HDDS-2627. Skip sonar check in forks The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout created (now e72357c)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at e72357c debug-acceptace-timeout This branch includes the following new commits: new e72357c debug-acceptace-timeout The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/01: debug-acceptace-timeout
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit e72357c5e33841298b3a7d1242b55b015f6d8bd6
Author: Márton Elek
AuthorDate: Wed Nov 27 14:24:25 2019 +0100
debug-acceptace-timeout
---
.github/workflows/post-commit.yml | 83 +--
.github/workflows/tmate.sh| 7
2 files changed, 9 insertions(+), 81 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 5097be4..0d0adf6 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -16,86 +16,6 @@ name: build-branch
on:
- push
jobs:
- build:
-name: compile
-runs-on: ubuntu-18.04
-steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
-with:
- args: ./hadoop-ozone/dev-support/checks/build.sh
- rat:
-name: rat
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/rat.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: rat
-path: target/rat
- author:
-name: author
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/author.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: author
-path: target/author
- unit:
-name: unit
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/unit.sh
-- uses: ./.github/buildenv
- if: github.repository == 'apache/hadoop-ozone'
- with:
-args: ./hadoop-ozone/dev-support/checks/sonar.sh
- env:
-SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
-GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: unit
-path: target/unit
- checkstyle:
-name: checkstyle
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/checkstyle.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: checkstyle
-path: target/checkstyle
- findbugs:
-name: findbugs
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/findbugs.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: findbugs
-path: target/findbugs
acceptance:
name: acceptance
runs-on: ubuntu-18.04
@@ -112,7 +32,8 @@ jobs:
args: ./hadoop-ozone/dev-support/checks/build.sh
- run: sudo pip install robotframework
- run: sudo chown runner -R .
-- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
+- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
+- run: ./.github/workflows/tmate.sh
- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
- uses: actions/upload-artifact@master
if: always()
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
new file mode 100755
index 000..da1149c
--- /dev/null
+++ b/.github/workflows/tmate.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+sudo apt-get install tmate
+tmate -S /tmp/tmate.sock new-session -d
+tmate -S /tmp/tmate.sock wait tmate-ready
+tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
+tmate -S /tmp/tmate.sock display -p '#{tmate_web}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated: fix build dependency
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/debug-acceptace-timeout by this push: new 50044a0 fix build dependency 50044a0 is described below commit 50044a0ab5c6f1d2188370b919923f30644b4d17 Author: Márton Elek AuthorDate: Wed Nov 27 15:23:25 2019 +0100 fix build dependency --- .github/workflows/post-commit.yml | 6 -- 1 file changed, 6 deletions(-) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index 0d0adf6..2b71142 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -19,12 +19,6 @@ jobs: acceptance: name: acceptance runs-on: ubuntu-18.04 -needs: - - build - - rat - - checkstyle - - unit - - findbugs steps: - uses: actions/checkout@master - uses: ./.github/buildenv - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated: fix install script
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/debug-acceptace-timeout by
this push:
new c2e02ae fix install script
c2e02ae is described below
commit c2e02aee58b72f81102f4c4e6a18bf5c20bbbc6b
Author: Márton Elek
AuthorDate: Wed Nov 27 15:45:01 2019 +0100
fix install script
---
.github/workflows/post-commit.yml | 2 +-
.github/workflows/tmate.sh| 3 +--
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 2b71142..988e8eb 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,13 +21,13 @@ jobs:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@master
+- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
- run: sudo pip install robotframework
- run: sudo chown runner -R .
- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
-- run: ./.github/workflows/tmate.sh
- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
- uses: actions/upload-artifact@master
if: always()
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index da1149c..fbe9166 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,6 +1,5 @@
#!/usr/bin/env bash
-
-sudo apt-get install tmate
+apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
tmate -S /tmp/tmate.sock wait tmate-ready
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated: sudo with power
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/debug-acceptace-timeout by
this push:
new 2cc38b8 sudo with power
2cc38b8 is described below
commit 2cc38b870db2d3a1ab6f9faf95bf4f9b8bf63e50
Author: Márton Elek
AuthorDate: Wed Nov 27 18:04:10 2019 +0100
sudo with power
---
.github/workflows/tmate.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index fbe9166..55b8d50 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
-apt-get -y install tmate
+sudo apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
tmate -S /tmp/tmate.sock wait tmate-ready
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated: debug without tmate
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/debug-acceptace-timeout by
this push:
new c81c1f5 debug without tmate
c81c1f5 is described below
commit c81c1f5e22fbc773cb188e3ee5c28ee3ee0fd417
Author: Márton Elek
AuthorDate: Wed Nov 27 18:34:34 2019 +0100
debug without tmate
---
.github/workflows/post-commit.yml | 2 +-
hadoop-ozone/dev-support/checks/acceptance.sh | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 988e8eb..b411bbd 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@master
-- run: ./.github/workflows/tmate.sh
+#- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh
b/hadoop-ozone/dev-support/checks/acceptance.sh
index c3c1aa0..71481ca 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -15,7 +15,7 @@
# limitations under the License.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
-
+set -x
REPORT_DIR=${OUTPUT_DIR:-"$DIR/../../../target/acceptance"}
mkdir -p "$REPORT_DIR"
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated (c81c1f5 -> cc2a29b)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from c81c1f5 debug without tmate new 6eabf49 add timeout new cc2a29b try to fix tmate The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/post-commit.yml | 3 ++- .github/workflows/tmate.sh| 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/02: try to fix tmate
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit cc2a29be25d59ec7f86323fc75cac9cd0c1be981
Author: Márton Elek
AuthorDate: Wed Nov 27 20:01:58 2019 +0100
try to fix tmate
---
.github/workflows/post-commit.yml | 2 +-
.github/workflows/tmate.sh| 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 1a65599..16877de 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -22,7 +22,7 @@ jobs:
timeout-minutes: 90
steps:
- uses: actions/checkout@master
-#- run: ./.github/workflows/tmate.sh
+- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index 55b8d50..c10a557 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
+set -ex
sudo apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
-tmate -S /tmp/tmate.sock wait tmate-ready
+sleep 2
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
tmate -S /tmp/tmate.sock display -p '#{tmate_web}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/02: add timeout
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 6eabf49e6725f5c18fffc260ec12d1bbc1bb10cf Author: Márton Elek AuthorDate: Wed Nov 27 19:56:37 2019 +0100 add timeout --- .github/workflows/post-commit.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index b411bbd..1a65599 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -19,6 +19,7 @@ jobs: acceptance: name: acceptance runs-on: ubuntu-18.04 +timeout-minutes: 90 steps: - uses: actions/checkout@master #- run: ./.github/workflows/tmate.sh - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated (cc2a29b -> 883dcfc)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from cc2a29b try to fix tmate new 8b01da5 debug secure execution new 883dcfc more debug The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh | 2 +- hadoop-ozone/dist/src/main/compose/testlib.sh | 7 +-- 2 files changed, 6 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/02: debug secure execution
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 8b01da578a733c9da44348f587738343be76c42a
Author: Márton Elek
AuthorDate: Wed Nov 27 20:08:21 2019 +0100
debug secure execution
---
hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
index f328463..a925f36 100755
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
+++ b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
@@ -14,7 +14,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-
+set -x
COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
export COMPOSE_DIR
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/02: more debug
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptace-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 883dcfc701f62e8f1b35658d47589101dcdcca6c
Author: Márton Elek
AuthorDate: Wed Nov 27 21:05:28 2019 +0100
more debug
---
hadoop-ozone/dist/src/main/compose/testlib.sh | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/hadoop-ozone/dist/src/main/compose/testlib.sh
b/hadoop-ozone/dist/src/main/compose/testlib.sh
index b20dca8..9311ed9 100755
--- a/hadoop-ozone/dist/src/main/compose/testlib.sh
+++ b/hadoop-ozone/dist/src/main/compose/testlib.sh
@@ -103,6 +103,7 @@ start_docker_env(){
## @paramName of the container in the docker-compose file
## @paramrobot test file or directory relative to the smoketest dir
execute_robot_test(){
+ set -x
CONTAINER="$1"
shift 1 #Remove first argument which was the container name
# shellcheck disable=SC2206
@@ -115,13 +116,15 @@ execute_robot_test(){
OUTPUT_NAME="$COMPOSE_ENV_NAME-$TEST_NAME-$CONTAINER"
OUTPUT_PATH="$RESULT_DIR_INSIDE/robot-$OUTPUT_NAME.xml"
docker-compose -f "$COMPOSE_FILE" exec -T "$CONTAINER" mkdir -p
"$RESULT_DIR_INSIDE"
+
+ docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" ln -f -s /dev/stdout
/tmp/out.txt
# shellcheck disable=SC2068
- docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" python -m robot
${ARGUMENTS[@]} --log NONE -N "$TEST_NAME" --report NONE
"${OZONE_ROBOT_OPTS[@]}" --output "$OUTPUT_PATH" "$SMOKETEST_DIR_INSIDE/$TEST"
+ docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" python -m robot
${ARGUMENTS[@]} --log NONE -N "$TEST_NAME" --debugfile /tmp/out --report NONE
"${OZONE_ROBOT_OPTS[@]}" --output "$OUTPUT_PATH" "$SMOKETEST_DIR_INSIDE/$TEST"
FULL_CONTAINER_NAME=$(docker-compose -f "$COMPOSE_FILE" ps | grep
"_${CONTAINER}_" | head -n 1 | awk '{print $1}')
docker cp "$FULL_CONTAINER_NAME:$OUTPUT_PATH" "$RESULT_DIR/"
set -e
-
+ set +x
}
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/debug-acceptace-timeout by this push: new 972c5a2 retrigger build 972c5a2 is described below commit 972c5a2456564b552e2d799e7ea70b1f844a459d Author: Márton Elek AuthorDate: Wed Nov 27 22:58:48 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated (883dcfc -> e13bfea)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 883dcfc more debug new 315e06b retrigger build new e13bfea retrigger build The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/02: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit e13bfea4599212066142b5ad84b2a9173b2a55cb Author: Márton Elek AuthorDate: Wed Nov 27 22:58:41 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/02: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 315e06b493a7fad571c4c0ba2df9b75795f8d4e6 Author: Márton Elek AuthorDate: Wed Nov 27 22:58:26 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptace-timeout updated: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptace-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git The following commit(s) were added to refs/heads/debug-acceptace-timeout by this push: new 775f82d retrigger build 775f82d is described below commit 775f82d7420ca2d75e13d4372a0cce1b21f3dbac Author: Márton Elek AuthorDate: Thu Nov 28 08:36:11 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch HDDS-1880-Decom updated: HDDS-2459. Refactor ReplicationManager to consider maintenance states
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch HDDS-1880-Decom
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
The following commit(s) were added to refs/heads/HDDS-1880-Decom by this push:
new 920b8c5 HDDS-2459. Refactor ReplicationManager to consider
maintenance states
920b8c5 is described below
commit 920b8c5eddc6f25138692636a66270ee22c8dfa4
Author: S O'Donnell
AuthorDate: Thu Nov 28 09:32:13 2019 +0100
HDDS-2459. Refactor ReplicationManager to consider maintenance states
Closes #262
---
.../proto/StorageContainerDatanodeProtocol.proto | 2 +
.../hdds/scm/container/ContainerReplicaCount.java | 241
.../hdds/scm/container/ReplicationManager.java | 224 +++
.../apache/hadoop/hdds/scm/node/NodeStatus.java| 69
.../hdds/scm/server/StorageContainerManager.java | 3 +-
.../hdds/scm/container/TestReplicationManager.java | 425 -
.../TestSCMContainerPlacementRackAware.java| 6 -
.../states/TestContainerReplicaCount.java | 333
.../hdds/scm/node/TestDatanodeAdminMonitor.java| 3 +
.../hdds/scm/safemode/TestSafeModeHandler.java | 6 +-
.../scm/node/TestDecommissionAndMaintenance.java | 1 -
11 files changed, 1229 insertions(+), 84 deletions(-)
diff --git
a/hadoop-hdds/container-service/src/main/proto/StorageContainerDatanodeProtocol.proto
b/hadoop-hdds/container-service/src/main/proto/StorageContainerDatanodeProtocol.proto
index 45a1db6..6c30ba1 100644
---
a/hadoop-hdds/container-service/src/main/proto/StorageContainerDatanodeProtocol.proto
+++
b/hadoop-hdds/container-service/src/main/proto/StorageContainerDatanodeProtocol.proto
@@ -185,6 +185,8 @@ message ContainerReplicaProto {
CLOSED = 4;
UNHEALTHY = 5;
INVALID = 6;
+DECOMMISSIONED = 7;
+MAINTENANCE = 8;
}
required int64 containerID = 1;
required State state = 2;
diff --git
a/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaCount.java
b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaCount.java
new file mode 100644
index 000..a7ea56d
--- /dev/null
+++
b/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplicaCount.java
@@ -0,0 +1,241 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdds.scm.container;
+
+import org.apache.hadoop.hdds.protocol.proto
+.StorageContainerDatanodeProtocolProtos.ContainerReplicaProto;
+import java.util.Set;
+
+/**
+ * Immutable object that is created with a set of ContainerReplica objects and
+ * the number of in flight replica add and deletes, the container replication
+ * factor and the min count which must be available for maintenance. This
+ * information can be used to determine if the container is over or under
+ * replicated and also how many additional replicas need created or removed.
+ */
+public class ContainerReplicaCount {
+
+ private int healthyCount = 0;
+ private int decommissionCount = 0;
+ private int maintenanceCount = 0;
+ private int inFlightAdd = 0;
+ private int inFlightDel = 0;
+ private int repFactor;
+ private int minHealthyForMaintenance;
+ private Set replica;
+
+ public ContainerReplicaCount(Set replica, int inFlightAdd,
+ int inFlightDelete, int replicationFactor,
+ int minHealthyForMaintenance) {
+this.healthyCount = 0;
+this.decommissionCount = 0;
+this.maintenanceCount = 0;
+this.inFlightAdd = inFlightAdd;
+this.inFlightDel = inFlightDelete;
+this.repFactor = replicationFactor;
+this.replica = replica;
+this.minHealthyForMaintenance
+= Math.min(this.repFactor, minHealthyForMaintenance);
+
+for (ContainerReplica cr : this.replica) {
+ ContainerReplicaProto.State state = cr.getState();
+ if (state == ContainerReplicaProto.State.DECOMMISSIONED) {
+decommissionCount++;
+ } else if (state == ContainerReplicaProto.State.MAINTENANCE) {
+maintenanceCount++;
+ } els
[hadoop-ozone] branch debug-acceptance-timeout created (now 82fecbe)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. at 82fecbe retrigger build This branch includes the following new commits: new e72357c debug-acceptace-timeout new 50044a0 fix build dependency new c2e02ae fix install script new 2cc38b8 sudo with power new c81c1f5 debug without tmate new 6eabf49 add timeout new cc2a29b try to fix tmate new 8b01da5 debug secure execution new 883dcfc more debug new 315e06b retrigger build new e13bfea retrigger build new 972c5a2 retrigger build new 775f82d retrigger build new 82fecbe retrigger build The 14 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/14: debug-acceptace-timeout
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit e72357c5e33841298b3a7d1242b55b015f6d8bd6
Author: Márton Elek
AuthorDate: Wed Nov 27 14:24:25 2019 +0100
debug-acceptace-timeout
---
.github/workflows/post-commit.yml | 83 +--
.github/workflows/tmate.sh| 7
2 files changed, 9 insertions(+), 81 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 5097be4..0d0adf6 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -16,86 +16,6 @@ name: build-branch
on:
- push
jobs:
- build:
-name: compile
-runs-on: ubuntu-18.04
-steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
-with:
- args: ./hadoop-ozone/dev-support/checks/build.sh
- rat:
-name: rat
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/rat.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: rat
-path: target/rat
- author:
-name: author
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/author.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: author
-path: target/author
- unit:
-name: unit
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/unit.sh
-- uses: ./.github/buildenv
- if: github.repository == 'apache/hadoop-ozone'
- with:
-args: ./hadoop-ozone/dev-support/checks/sonar.sh
- env:
-SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
-GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: unit
-path: target/unit
- checkstyle:
-name: checkstyle
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/checkstyle.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: checkstyle
-path: target/checkstyle
- findbugs:
-name: findbugs
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/findbugs.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: findbugs
-path: target/findbugs
acceptance:
name: acceptance
runs-on: ubuntu-18.04
@@ -112,7 +32,8 @@ jobs:
args: ./hadoop-ozone/dev-support/checks/build.sh
- run: sudo pip install robotframework
- run: sudo chown runner -R .
-- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
+- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
+- run: ./.github/workflows/tmate.sh
- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
- uses: actions/upload-artifact@master
if: always()
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
new file mode 100755
index 000..da1149c
--- /dev/null
+++ b/.github/workflows/tmate.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+sudo apt-get install tmate
+tmate -S /tmp/tmate.sock new-session -d
+tmate -S /tmp/tmate.sock wait tmate-ready
+tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
+tmate -S /tmp/tmate.sock display -p '#{tmate_web}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 07/14: try to fix tmate
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit cc2a29be25d59ec7f86323fc75cac9cd0c1be981
Author: Márton Elek
AuthorDate: Wed Nov 27 20:01:58 2019 +0100
try to fix tmate
---
.github/workflows/post-commit.yml | 2 +-
.github/workflows/tmate.sh| 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 1a65599..16877de 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -22,7 +22,7 @@ jobs:
timeout-minutes: 90
steps:
- uses: actions/checkout@master
-#- run: ./.github/workflows/tmate.sh
+- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index 55b8d50..c10a557 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
+set -ex
sudo apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
-tmate -S /tmp/tmate.sock wait tmate-ready
+sleep 2
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
tmate -S /tmp/tmate.sock display -p '#{tmate_web}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 08/14: debug secure execution
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 8b01da578a733c9da44348f587738343be76c42a
Author: Márton Elek
AuthorDate: Wed Nov 27 20:08:21 2019 +0100
debug secure execution
---
hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
index f328463..a925f36 100755
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
+++ b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
@@ -14,7 +14,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-
+set -x
COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
export COMPOSE_DIR
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 05/14: debug without tmate
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit c81c1f5e22fbc773cb188e3ee5c28ee3ee0fd417
Author: Márton Elek
AuthorDate: Wed Nov 27 18:34:34 2019 +0100
debug without tmate
---
.github/workflows/post-commit.yml | 2 +-
hadoop-ozone/dev-support/checks/acceptance.sh | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 988e8eb..b411bbd 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@master
-- run: ./.github/workflows/tmate.sh
+#- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh
b/hadoop-ozone/dev-support/checks/acceptance.sh
index c3c1aa0..71481ca 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -15,7 +15,7 @@
# limitations under the License.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
-
+set -x
REPORT_DIR=${OUTPUT_DIR:-"$DIR/../../../target/acceptance"}
mkdir -p "$REPORT_DIR"
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 11/14: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit e13bfea4599212066142b5ad84b2a9173b2a55cb Author: Márton Elek AuthorDate: Wed Nov 27 22:58:41 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 03/14: fix install script
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit c2e02aee58b72f81102f4c4e6a18bf5c20bbbc6b
Author: Márton Elek
AuthorDate: Wed Nov 27 15:45:01 2019 +0100
fix install script
---
.github/workflows/post-commit.yml | 2 +-
.github/workflows/tmate.sh| 3 +--
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 2b71142..988e8eb 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,13 +21,13 @@ jobs:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@master
+- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
- run: sudo pip install robotframework
- run: sudo chown runner -R .
- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
-- run: ./.github/workflows/tmate.sh
- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
- uses: actions/upload-artifact@master
if: always()
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index da1149c..fbe9166 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,6 +1,5 @@
#!/usr/bin/env bash
-
-sudo apt-get install tmate
+apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
tmate -S /tmp/tmate.sock wait tmate-ready
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 12/14: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 972c5a2456564b552e2d799e7ea70b1f844a459d Author: Márton Elek AuthorDate: Wed Nov 27 22:58:48 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 14/14: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 82fecbe7cb029741f7eac2356632b7490af2b58b Author: Márton Elek AuthorDate: Thu Nov 28 09:41:15 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 10/14: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 315e06b493a7fad571c4c0ba2df9b75795f8d4e6 Author: Márton Elek AuthorDate: Wed Nov 27 22:58:26 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 09/14: more debug
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 883dcfc701f62e8f1b35658d47589101dcdcca6c
Author: Márton Elek
AuthorDate: Wed Nov 27 21:05:28 2019 +0100
more debug
---
hadoop-ozone/dist/src/main/compose/testlib.sh | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/hadoop-ozone/dist/src/main/compose/testlib.sh
b/hadoop-ozone/dist/src/main/compose/testlib.sh
index b20dca8..9311ed9 100755
--- a/hadoop-ozone/dist/src/main/compose/testlib.sh
+++ b/hadoop-ozone/dist/src/main/compose/testlib.sh
@@ -103,6 +103,7 @@ start_docker_env(){
## @paramName of the container in the docker-compose file
## @paramrobot test file or directory relative to the smoketest dir
execute_robot_test(){
+ set -x
CONTAINER="$1"
shift 1 #Remove first argument which was the container name
# shellcheck disable=SC2206
@@ -115,13 +116,15 @@ execute_robot_test(){
OUTPUT_NAME="$COMPOSE_ENV_NAME-$TEST_NAME-$CONTAINER"
OUTPUT_PATH="$RESULT_DIR_INSIDE/robot-$OUTPUT_NAME.xml"
docker-compose -f "$COMPOSE_FILE" exec -T "$CONTAINER" mkdir -p
"$RESULT_DIR_INSIDE"
+
+ docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" ln -f -s /dev/stdout
/tmp/out.txt
# shellcheck disable=SC2068
- docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" python -m robot
${ARGUMENTS[@]} --log NONE -N "$TEST_NAME" --report NONE
"${OZONE_ROBOT_OPTS[@]}" --output "$OUTPUT_PATH" "$SMOKETEST_DIR_INSIDE/$TEST"
+ docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" python -m robot
${ARGUMENTS[@]} --log NONE -N "$TEST_NAME" --debugfile /tmp/out --report NONE
"${OZONE_ROBOT_OPTS[@]}" --output "$OUTPUT_PATH" "$SMOKETEST_DIR_INSIDE/$TEST"
FULL_CONTAINER_NAME=$(docker-compose -f "$COMPOSE_FILE" ps | grep
"_${CONTAINER}_" | head -n 1 | awk '{print $1}')
docker cp "$FULL_CONTAINER_NAME:$OUTPUT_PATH" "$RESULT_DIR/"
set -e
-
+ set +x
}
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 06/14: add timeout
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 6eabf49e6725f5c18fffc260ec12d1bbc1bb10cf Author: Márton Elek AuthorDate: Wed Nov 27 19:56:37 2019 +0100 add timeout --- .github/workflows/post-commit.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index b411bbd..1a65599 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -19,6 +19,7 @@ jobs: acceptance: name: acceptance runs-on: ubuntu-18.04 +timeout-minutes: 90 steps: - uses: actions/checkout@master #- run: ./.github/workflows/tmate.sh - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 02/14: fix build dependency
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 50044a0ab5c6f1d2188370b919923f30644b4d17 Author: Márton Elek AuthorDate: Wed Nov 27 15:23:25 2019 +0100 fix build dependency --- .github/workflows/post-commit.yml | 6 -- 1 file changed, 6 deletions(-) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index 0d0adf6..2b71142 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -19,12 +19,6 @@ jobs: acceptance: name: acceptance runs-on: ubuntu-18.04 -needs: - - build - - rat - - checkstyle - - unit - - findbugs steps: - uses: actions/checkout@master - uses: ./.github/buildenv - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 13/14: retrigger build
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 775f82d7420ca2d75e13d4372a0cce1b21f3dbac Author: Márton Elek AuthorDate: Thu Nov 28 08:36:11 2019 +0100 retrigger build - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 04/14: sudo with power
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 2cc38b870db2d3a1ab6f9faf95bf4f9b8bf63e50
Author: Márton Elek
AuthorDate: Wed Nov 27 18:04:10 2019 +0100
sudo with power
---
.github/workflows/tmate.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index fbe9166..55b8d50 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
-apt-get -y install tmate
+sudo apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
tmate -S /tmp/tmate.sock wait tmate-ready
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (0ef75b0 -> a73c470)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from 0ef75b0 HDDS-2528. Sonar : code smell category issues in CommitWatcher (#247) add a73c470 HDDS-2034. Async RATIS pipeline creation and destroy through heartbeat commands No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hdds/HddsConfigKeys.java | 8 +- .../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 5 + .../org/apache/hadoop/hdds/utils/Scheduler.java| 7 +- .../common/src/main/resources/ozone-default.xml| 20 +- .../common/statemachine/DatanodeStateMachine.java | 6 + .../CloseContainerCommandHandler.java | 11 +- .../ClosePipelineCommandHandler.java | 120 .../commandhandler/CommandHandler.java | 2 +- .../CreatePipelineCommandHandler.java | 135 + .../states/endpoint/HeartbeatEndpointTask.java | 22 +++ .../common/transport/server/XceiverServerSpi.java | 18 ++ .../transport/server/ratis/XceiverServerRatis.java | 36 ...ainerCommand.java => ClosePipelineCommand.java} | 53 +++--- .../protocol/commands/CreatePipelineCommand.java | 100 ++ .../proto/StorageContainerDatanodeProtocol.proto | 23 +++ .../hadoop/hdds/scm/block/BlockManagerImpl.java| 3 + .../hdds/scm/container/ContainerStateManager.java | 1 + .../apache/hadoop/hdds/scm/events/SCMEvents.java | 12 +- .../scm/pipeline/BackgroundPipelineCreator.java| 9 +- .../hadoop/hdds/scm/pipeline/PipelineFactory.java | 13 +- .../hadoop/hdds/scm/pipeline/PipelineManager.java | 15 +- .../hadoop/hdds/scm/pipeline/PipelineProvider.java | 2 + .../hdds/scm/pipeline/PipelineReportHandler.java | 54 +++--- .../hdds/scm/pipeline/PipelineStateManager.java| 4 +- .../hdds/scm/pipeline/RatisPipelineProvider.java | 137 +- .../hdds/scm/pipeline/RatisPipelineUtils.java | 103 -- .../hdds/scm/pipeline/SCMPipelineManager.java | 82 ++-- .../hdds/scm/pipeline/SCMPipelineMetrics.java | 10 + .../hdds/scm/pipeline/SimplePipelineProvider.java | 5 + .../scm/safemode/HealthyPipelineSafeModeRule.java | 78 +++- .../safemode/OneReplicaPipelineSafeModeRule.java | 65 +++ .../hdds/scm/safemode/SCMSafeModeManager.java | 16 +- .../hadoop/hdds/scm/safemode/SafeModeHandler.java | 5 +- .../scm/server/SCMDatanodeHeartbeatDispatcher.java | 1 + .../hdds/scm/server/SCMDatanodeProtocolServer.java | 20 ++ .../hdds/scm/server/StorageContainerManager.java | 3 +- .../java/org/apache/hadoop/hdds/scm/TestUtils.java | 12 ++ .../hadoop/hdds/scm/block/TestBlockManager.java| 44 - .../container/TestCloseContainerEventHandler.java | 11 +- .../scm/container/TestSCMContainerManager.java | 4 +- .../hdds/scm/node/TestContainerPlacement.java | 2 +- .../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 11 +- .../hadoop/hdds/scm/node/TestSCMNodeManager.java | 9 +- .../scm/pipeline/MockRatisPipelineProvider.java| 10 +- .../scm/pipeline/TestRatisPipelineProvider.java| 26 ++- .../hdds/scm/pipeline/TestSCMPipelineManager.java | 93 +++-- .../safemode/TestHealthyPipelineSafeModeRule.java | 44 ++--- .../TestOneReplicaPipelineSafeModeRule.java| 36 +--- .../hdds/scm/safemode/TestSCMSafeModeManager.java | 50 ++--- hadoop-ozone/dist/src/main/compose/testlib.sh | 35 .../dist/src/main/smoketest/__init__.robot | 3 +- .../TestContainerStateManagerIntegration.java | 6 +- .../metrics/TestSCMContainerManagerMetrics.java| 3 + .../hdds/scm/pipeline/TestPipelineClose.java | 4 +- .../scm/pipeline/TestRatisPipelineProvider.java| 210 + .../org/apache/hadoop/ozone/MiniOzoneCluster.java | 2 +- .../apache/hadoop/ozone/MiniOzoneClusterImpl.java | 43 ++--- .../hadoop/ozone/TestContainerOperations.java | 2 +- .../TestContainerStateMachineIdempotency.java | 2 +- .../apache/hadoop/ozone/TestMiniOzoneCluster.java | 6 +- .../hadoop/ozone/TestStorageContainerManager.java | 18 +- .../apache/hadoop/ozone/client/rpc/TestBCSID.java | 3 + .../rpc/TestContainerStateMachineFailures.java | 9 +- .../ozone/container/TestContainerReplication.java | 2 +- .../commandhandler/TestBlockDeletion.java | 6 +- .../commandhandler/TestCloseContainerHandler.java | 2 + .../commandhandler/TestDeleteContainerHandler.java | 2 + .../hadoop/ozone/dn/scrubber/TestDataScrubber.java | 2 + .../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +- .../apache/hadoop/ozone/om/TestScmSafeMode.java| 2 +- .../hadoop/ozone/scm/TestContainerSmallFile.java | 2 +- .../scm/TestGetCommittedBlockLengthAndPutKey.java | 2 +- .../org/apache/hadoop/ozone/scm/TestSCMMXBean.java |
[hadoop-ozone] 02/09: fix build dependency
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit 96da4f41a75a729315df2314244fabe8d233af40 Author: Márton Elek AuthorDate: Wed Nov 27 15:23:25 2019 +0100 fix build dependency --- .github/workflows/post-commit.yml | 6 -- 1 file changed, 6 deletions(-) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index 0d0adf6..2b71142 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -19,12 +19,6 @@ jobs: acceptance: name: acceptance runs-on: ubuntu-18.04 -needs: - - build - - rat - - checkstyle - - unit - - findbugs steps: - uses: actions/checkout@master - uses: ./.github/buildenv - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch debug-acceptance-timeout updated (82fecbe -> f5694c9)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. discard 82fecbe retrigger build discard 775f82d retrigger build discard 972c5a2 retrigger build discard e13bfea retrigger build discard 315e06b retrigger build discard 883dcfc more debug discard 8b01da5 debug secure execution discard cc2a29b try to fix tmate discard 6eabf49 add timeout discard c81c1f5 debug without tmate discard 2cc38b8 sudo with power discard c2e02ae fix install script discard 50044a0 fix build dependency discard e72357c debug-acceptace-timeout add 045e3da HDDS-2635 Minor:Fix typo in HddsConfigKeys and ozone-default. (#274) add ba9db10 HDDS-2632. Fix TestContainerPersistence#testDeleteChunk (#273) add ac93176 HDDS-2633. Let findbugs.sh skip frontend plugin for Recon (#275) add 0ef75b0 HDDS-2528. Sonar : code smell category issues in CommitWatcher (#247) add a73c470 HDDS-2034. Async RATIS pipeline creation and destroy through heartbeat commands new f36ce6d debug-acceptace-timeout new 96da4f4 fix build dependency new 4ac68e1 fix install script new 90ac2bc sudo with power new 381da33 debug without tmate new bf44556 add timeout new 0223230 try to fix tmate new cb7425e debug secure execution new f5694c9 more debug This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (82fecbe) \ N -- N -- N refs/heads/debug-acceptance-timeout (f5694c9) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 9 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../hadoop/hdds/scm/storage/CommitWatcher.java | 20 +- .../org/apache/hadoop/hdds/HddsConfigKeys.java | 10 +- .../apache/hadoop/hdds/scm/pipeline/Pipeline.java | 5 + .../org/apache/hadoop/hdds/utils/Scheduler.java| 7 +- .../common/src/main/resources/ozone-default.xml| 22 +-- .../common/statemachine/DatanodeStateMachine.java | 6 + .../CloseContainerCommandHandler.java | 11 +- .../ClosePipelineCommandHandler.java | 120 .../commandhandler/CommandHandler.java | 2 +- .../CreatePipelineCommandHandler.java | 135 + .../states/endpoint/HeartbeatEndpointTask.java | 22 +++ .../common/transport/server/XceiverServerSpi.java | 18 ++ .../transport/server/ratis/XceiverServerRatis.java | 36 ...ainerCommand.java => ClosePipelineCommand.java} | 53 +++--- .../protocol/commands/CreatePipelineCommand.java | 100 ++ .../proto/StorageContainerDatanodeProtocol.proto | 23 +++ .../common/impl/TestContainerPersistence.java | 2 +- .../hadoop/hdds/scm/block/BlockManagerImpl.java| 3 + .../hdds/scm/container/ContainerStateManager.java | 1 + .../apache/hadoop/hdds/scm/events/SCMEvents.java | 12 +- .../scm/pipeline/BackgroundPipelineCreator.java| 9 +- .../hadoop/hdds/scm/pipeline/PipelineFactory.java | 13 +- .../hadoop/hdds/scm/pipeline/PipelineManager.java | 15 +- .../hadoop/hdds/scm/pipeline/PipelineProvider.java | 2 + .../hdds/scm/pipeline/PipelineReportHandler.java | 54 +++--- .../hdds/scm/pipeline/PipelineStateManager.java| 4 +- .../hdds/scm/pipeline/RatisPipelineProvider.java | 137 +- .../hdds/scm/pipeline/RatisPipelineUtils.java | 103 -- .../hdds/scm/pipeline/SCMPipelineManager.java | 82 ++-- .../hdds/scm/pipeline/SCMPipelineMetrics.java | 10 + .../hdds/scm/pipeline/SimplePipelineProvider.java | 5 + .../scm/safemode/HealthyPipelineSafeModeRule.java | 78 +++- .../safemode/OneReplicaPipelineSafeModeRule.java | 65 +++ .../hdds/scm/safemode/SCMSafeModeManager.java | 16 +- .../hadoop/hdds/scm/safemode/SafeModeHandler.java | 5 +- .../scm/server/SCMDatanodeHeartbeatDispatcher.java | 1 + .../hdds/scm/server/SCMDatanodeProtocolServer.java | 20 ++ .../hdds/scm/server/StorageContainerManager.java | 3 +- .../java/org/apache/hadoop/hdds/scm/TestUtils.java | 12 ++ .../hadoop/hdds/scm/block/TestBlockManager.java| 44 - .../container
[hadoop-ozone] 09/09: more debug
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit f5694c993a3e887067d364218bf6db2d7a71b085
Author: Márton Elek
AuthorDate: Wed Nov 27 21:05:28 2019 +0100
more debug
---
hadoop-ozone/dist/src/main/compose/testlib.sh | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/hadoop-ozone/dist/src/main/compose/testlib.sh
b/hadoop-ozone/dist/src/main/compose/testlib.sh
index 49274c0..15eef7a 100755
--- a/hadoop-ozone/dist/src/main/compose/testlib.sh
+++ b/hadoop-ozone/dist/src/main/compose/testlib.sh
@@ -138,6 +138,7 @@ start_docker_env(){
## @paramName of the container in the docker-compose file
## @paramrobot test file or directory relative to the smoketest dir
execute_robot_test(){
+ set -x
CONTAINER="$1"
shift 1 #Remove first argument which was the container name
# shellcheck disable=SC2206
@@ -150,13 +151,15 @@ execute_robot_test(){
OUTPUT_NAME="$COMPOSE_ENV_NAME-$TEST_NAME-$CONTAINER"
OUTPUT_PATH="$RESULT_DIR_INSIDE/robot-$OUTPUT_NAME.xml"
docker-compose -f "$COMPOSE_FILE" exec -T "$CONTAINER" mkdir -p
"$RESULT_DIR_INSIDE"
+
+ docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" ln -f -s /dev/stdout
/tmp/out.txt
# shellcheck disable=SC2068
- docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" python -m robot
${ARGUMENTS[@]} --log NONE -N "$TEST_NAME" --report NONE
"${OZONE_ROBOT_OPTS[@]}" --output "$OUTPUT_PATH" "$SMOKETEST_DIR_INSIDE/$TEST"
+ docker-compose -f "$COMPOSE_FILE" exec -T -e
SECURITY_ENABLED="${SECURITY_ENABLED}" "$CONTAINER" python -m robot
${ARGUMENTS[@]} --log NONE -N "$TEST_NAME" --debugfile /tmp/out --report NONE
"${OZONE_ROBOT_OPTS[@]}" --output "$OUTPUT_PATH" "$SMOKETEST_DIR_INSIDE/$TEST"
FULL_CONTAINER_NAME=$(docker-compose -f "$COMPOSE_FILE" ps | grep
"_${CONTAINER}_" | head -n 1 | awk '{print $1}')
docker cp "$FULL_CONTAINER_NAME:$OUTPUT_PATH" "$RESULT_DIR/"
set -e
-
+ set +x
}
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 06/09: add timeout
This is an automated email from the ASF dual-hosted git repository. elek pushed a commit to branch debug-acceptance-timeout in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git commit bf44556966dede3f9d78262c8b9a1b58e645a5c7 Author: Márton Elek AuthorDate: Wed Nov 27 19:56:37 2019 +0100 add timeout --- .github/workflows/post-commit.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/post-commit.yml b/.github/workflows/post-commit.yml index b411bbd..1a65599 100644 --- a/.github/workflows/post-commit.yml +++ b/.github/workflows/post-commit.yml @@ -19,6 +19,7 @@ jobs: acceptance: name: acceptance runs-on: ubuntu-18.04 +timeout-minutes: 90 steps: - uses: actions/checkout@master #- run: ./.github/workflows/tmate.sh - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 07/09: try to fix tmate
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 0223230a1e86f4b04550b1466f5acb82523108b6
Author: Márton Elek
AuthorDate: Wed Nov 27 20:01:58 2019 +0100
try to fix tmate
---
.github/workflows/post-commit.yml | 2 +-
.github/workflows/tmate.sh| 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 1a65599..16877de 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -22,7 +22,7 @@ jobs:
timeout-minutes: 90
steps:
- uses: actions/checkout@master
-#- run: ./.github/workflows/tmate.sh
+- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index 55b8d50..c10a557 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
+set -ex
sudo apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
-tmate -S /tmp/tmate.sock wait tmate-ready
+sleep 2
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
tmate -S /tmp/tmate.sock display -p '#{tmate_web}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 03/09: fix install script
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 4ac68e15b208224734728794b502273b0cd4a264
Author: Márton Elek
AuthorDate: Wed Nov 27 15:45:01 2019 +0100
fix install script
---
.github/workflows/post-commit.yml | 2 +-
.github/workflows/tmate.sh| 3 +--
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 2b71142..988e8eb 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,13 +21,13 @@ jobs:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@master
+- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
- run: sudo pip install robotframework
- run: sudo chown runner -R .
- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
-- run: ./.github/workflows/tmate.sh
- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
- uses: actions/upload-artifact@master
if: always()
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index da1149c..fbe9166 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,6 +1,5 @@
#!/usr/bin/env bash
-
-sudo apt-get install tmate
+apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
tmate -S /tmp/tmate.sock wait tmate-ready
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 04/09: sudo with power
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 90ac2bca2474cb8e4dd0fa42683688139903c69e
Author: Márton Elek
AuthorDate: Wed Nov 27 18:04:10 2019 +0100
sudo with power
---
.github/workflows/tmate.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
index fbe9166..55b8d50 100755
--- a/.github/workflows/tmate.sh
+++ b/.github/workflows/tmate.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
-apt-get -y install tmate
+sudo apt-get -y install tmate
tmate -S /tmp/tmate.sock new-session -d
tmate -S /tmp/tmate.sock wait tmate-ready
tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 05/09: debug without tmate
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit 381da3339f4e46930f686bd6d897443a0b17a15f
Author: Márton Elek
AuthorDate: Wed Nov 27 18:34:34 2019 +0100
debug without tmate
---
.github/workflows/post-commit.yml | 2 +-
hadoop-ozone/dev-support/checks/acceptance.sh | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 988e8eb..b411bbd 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@master
-- run: ./.github/workflows/tmate.sh
+#- run: ./.github/workflows/tmate.sh
- uses: ./.github/buildenv
with:
args: ./hadoop-ozone/dev-support/checks/build.sh
diff --git a/hadoop-ozone/dev-support/checks/acceptance.sh
b/hadoop-ozone/dev-support/checks/acceptance.sh
index c3c1aa0..71481ca 100755
--- a/hadoop-ozone/dev-support/checks/acceptance.sh
+++ b/hadoop-ozone/dev-support/checks/acceptance.sh
@@ -15,7 +15,7 @@
# limitations under the License.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
cd "$DIR/../../.." || exit 1
-
+set -x
REPORT_DIR=${OUTPUT_DIR:-"$DIR/../../../target/acceptance"}
mkdir -p "$REPORT_DIR"
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 08/09: debug secure execution
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit cb7425e8c9e9a78752349fd1159759b51f30783f
Author: Márton Elek
AuthorDate: Wed Nov 27 20:08:21 2019 +0100
debug secure execution
---
hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
index f328463..a925f36 100755
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
+++ b/hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
@@ -14,7 +14,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-
+set -x
COMPOSE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
export COMPOSE_DIR
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] 01/09: debug-acceptace-timeout
This is an automated email from the ASF dual-hosted git repository.
elek pushed a commit to branch debug-acceptance-timeout
in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git
commit f36ce6d7d9901d28fe5b0b8eef979f970fd50676
Author: Márton Elek
AuthorDate: Wed Nov 27 14:24:25 2019 +0100
debug-acceptace-timeout
---
.github/workflows/post-commit.yml | 83 +--
.github/workflows/tmate.sh| 7
2 files changed, 9 insertions(+), 81 deletions(-)
diff --git a/.github/workflows/post-commit.yml
b/.github/workflows/post-commit.yml
index 5097be4..0d0adf6 100644
--- a/.github/workflows/post-commit.yml
+++ b/.github/workflows/post-commit.yml
@@ -16,86 +16,6 @@ name: build-branch
on:
- push
jobs:
- build:
-name: compile
-runs-on: ubuntu-18.04
-steps:
- - uses: actions/checkout@master
- - uses: ./.github/buildenv
-with:
- args: ./hadoop-ozone/dev-support/checks/build.sh
- rat:
-name: rat
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/rat.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: rat
-path: target/rat
- author:
-name: author
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/author.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: author
-path: target/author
- unit:
-name: unit
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/unit.sh
-- uses: ./.github/buildenv
- if: github.repository == 'apache/hadoop-ozone'
- with:
-args: ./hadoop-ozone/dev-support/checks/sonar.sh
- env:
-SONAR_TOKEN: ${{ secrets.SONARCLOUD_TOKEN }}
-GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: unit
-path: target/unit
- checkstyle:
-name: checkstyle
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/checkstyle.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: checkstyle
-path: target/checkstyle
- findbugs:
-name: findbugs
-runs-on: ubuntu-18.04
-steps:
-- uses: actions/checkout@master
-- uses: ./.github/buildenv
- with:
- args: ./hadoop-ozone/dev-support/checks/findbugs.sh
-- uses: actions/upload-artifact@master
- if: always()
- with:
-name: findbugs
-path: target/findbugs
acceptance:
name: acceptance
runs-on: ubuntu-18.04
@@ -112,7 +32,8 @@ jobs:
args: ./hadoop-ozone/dev-support/checks/build.sh
- run: sudo pip install robotframework
- run: sudo chown runner -R .
-- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
+- run: cd ./hadoop-ozone/dist/target/ozone-*-SNAPSHOT/ && mkdir .aws
&& sudo chown 1000 .aws
+- run: ./.github/workflows/tmate.sh
- run: ./hadoop-ozone/dev-support/checks/acceptance.sh
- uses: actions/upload-artifact@master
if: always()
diff --git a/.github/workflows/tmate.sh b/.github/workflows/tmate.sh
new file mode 100755
index 000..da1149c
--- /dev/null
+++ b/.github/workflows/tmate.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+sudo apt-get install tmate
+tmate -S /tmp/tmate.sock new-session -d
+tmate -S /tmp/tmate.sock wait tmate-ready
+tmate -S /tmp/tmate.sock display -p '#{tmate_ssh}'
+tmate -S /tmp/tmate.sock display -p '#{tmate_web}'
-
To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
[hadoop-ozone] branch master updated (b1ceedd -> c91916d)
This is an automated email from the ASF dual-hosted git repository. elek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hadoop-ozone.git. from b1ceedd Revert "HDDS-2640 Add leaderID information in pipeline list subcommand (#279)" (#280) add c91916d HDDS-2639. Turning off unit test until review of HDDS-2639 No new revisions were added by this update. Summary of changes: .../java/org/apache/hadoop/hdds/utils/db/cache/TestTableCacheImpl.java | 2 ++ 1 file changed, 2 insertions(+) - To unsubscribe, e-mail: ozone-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-commits-h...@hadoop.apache.org
