[jira] [Created] (HDFS-14915) Move Superuser Check Before Taking Lock For Encryption API

2019-10-20 Thread Ayush Saxena (Jira) 
Ayush Saxena created HDFS-14915:
---

 Summary: Move Superuser Check Before Taking Lock For Encryption API
 Key: HDFS-14915
 URL: https://issues.apache.org/jira/browse/HDFS-14915
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Ayush Saxena
Assignee: Ayush Saxena






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #12: HDDS-2206. Separate handling for OMException and IOException in the Ozone Manager. Contributed by Supratim Deka

2019-10-20 Thread GitBox 
bharatviswa504 merged pull request #12: HDDS-2206. Separate handling for 
OMException and IOException in the Ozone Manager. Contributed by Supratim Deka
URL: https://github.com/apache/hadoop-ozone/pull/12
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #50: HDDS-2131. Optimize replication type and creation time calculation in S3 MPU list call.

2019-10-20 Thread GitBox 
bharatviswa504 commented on a change in pull request #50: HDDS-2131. Optimize 
replication type and creation time calculation in S3 MPU list call.
URL: https://github.com/apache/hadoop-ozone/pull/50#discussion_r336837495
 
 

 ##
 File path: 
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
 ##
 @@ -1323,29 +1323,36 @@ public OmMultipartUploadList 
listMultipartUploads(String volumeName,
 
   List collect = multipartUploadKeys.stream()
   .map(OmMultipartUpload::from)
-  .map(upload -> {
+  .peek(upload -> {
 String dbKey = metadataManager
 .getOzoneKey(upload.getVolumeName(),
 upload.getBucketName(),
 upload.getKeyName());
 try {
-  Table openKeyTable =
-  metadataManager.getOpenKeyTable();
+  Table keyInfoTable =
+  metadataManager.getMultipartInfoTable();
 
-  OmKeyInfo omKeyInfo =
-  openKeyTable.get(upload.getDbKey());
+  OmMultipartKeyInfo multipartKeyInfo =
+  keyInfoTable.get(upload.getDbKey());
 
   upload.setCreationTime(
-  Instant.ofEpochMilli(omKeyInfo.getCreationTime()));
-
-  upload.setReplicationType(omKeyInfo.getType());
-  upload.setReplicationFactor(omKeyInfo.getFactor());
+  Instant.ofEpochMilli(multipartKeyInfo.getCreationTime()));
+
+  TreeMap partKeyInfoMap =
+  multipartKeyInfo.getPartKeyInfoMap();
+  if (!partKeyInfoMap.isEmpty()) {
+PartKeyInfo partKeyInfo =
 
 Review comment:
   If the MPU uploaded does not have any parts then I don't see that logic here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #50: HDDS-2131. Optimize replication type and creation time calculation in S3 MPU list call.

2019-10-20 Thread GitBox 
bharatviswa504 commented on issue #50: HDDS-2131. Optimize replication type and 
creation time calculation in S3 MPU list call.
URL: https://github.com/apache/hadoop-ozone/pull/50#issuecomment-544348468
 
 
   > > And also this is missing S3 end change to use the replication type and 
replication factor and show them in S3StorageType format.
   > > Need to use this class for conversion. 
https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3StorageType.java
   > 
   > Thanks for pointing this one out, where does this change go?
   > Do you mean we should add S3StorageType to 
org.apache.hadoop.ozone.om.response.s3.multipart.S3InitiateMultipartUploadResponse#S3InitiateMultipartUploadResponse
   
   This was already taken care of. Sorry I thought this is missing.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDDS-2326) Http server of Freon is not started for new Freon tests

2019-10-20 Thread Bharat Viswanadham (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDDS-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bharat Viswanadham resolved HDDS-2326.
--
Fix Version/s: 0.5.0
   Resolution: Fixed

> Http server of Freon is not started for new Freon tests
> ---
>
> Key: HDDS-2326
> URL: https://issues.apache.org/jira/browse/HDDS-2326
> Project: Hadoop Distributed Data Store
>  Issue Type: New Feature
>  Components: freon
>Reporter: Marton Elek
>Assignee: Marton Elek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> HDDS-2022 introduced new Freon tests but the Freon http server is not started 
> for the new tests.
> Freon includes a http server which can be turned on with the '–server' flag. 
> It helps to monitor and profile the freon as the http server contains by 
> default the prometheus and profiler servlets.
> The server should be started if's requested.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 merged pull request #52: HDDS-2326. Http server of Freon is not started for new Freon tests

2019-10-20 Thread GitBox 
bharatviswa504 merged pull request #52: HDDS-2326. Http server of Freon is not 
started for new Freon tests
URL: https://github.com/apache/hadoop-ozone/pull/52
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #52: HDDS-2326. Http server of Freon is not started for new Freon tests

2019-10-20 Thread GitBox 
bharatviswa504 commented on issue #52: HDDS-2326. Http server of Freon is not 
started for new Freon tests
URL: https://github.com/apache/hadoop-ozone/pull/52#issuecomment-544347044
 
 
   Thank You @elek for the contribution and @adoroszlai for the review.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
bharatviswa504 commented on a change in pull request #62: HDDS-2335. Params not 
included in AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62#discussion_r336836042
 
 

 ##
 File path: 
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditMessage.java
 ##
 @@ -120,7 +120,7 @@ public Builder withException(Throwable ex){
 public AuditMessage build(){
   AuditMessage auditMessage = new AuditMessage();
   auditMessage.message = "user=" + this.user + " | ip=" + this.ip + " | " +
-  "op=" + this.op + " | " + "ret=" + this.ret;
+  "op=" + this.op + " " + this.params + " | " + "ret=" + this.ret;
 
 Review comment:
   My question was can we add like params = <>. Like how for others we 
have.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 merged pull request #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
mukul1987 merged pull request #62: HDDS-2335. Params not included in 
AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines should be marked as ready only after the leader election is complete.

2019-10-20 Thread GitBox 
mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines 
should be marked as ready only after the leader election is complete.
URL: https://github.com/apache/hadoop-ozone/pull/23#discussion_r336831346
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/SCMPipelineManager.java
 ##
 @@ -288,6 +294,28 @@ public int getNumberOfContainers(PipelineID pipelineID) 
throws IOException {
 return stateManager.getNumberOfContainers(pipelineID);
   }
 
+  @Override
+  public void processReportedLeaderInfo(PipelineID pipelineID,
+  DatanodeDetails dn, ByteString leaderID) throws IOException {
+Pipeline pipeline = stateManager.getPipeline(pipelineID);
+Map ids =
+reportedLeadersForPipeline.computeIfAbsent(pipelineID,
+k -> new HashMap<>());
+
+ids.put(dn.getUuid(), leaderID);
 
 Review comment:
   It will be better to maintain node->leaderId map inside the Pipeline.java 
class. The information is pipeline specific and hence should be kept there.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [Discuss] Hadoop-Ozone repository mailing list configurations

2019-10-20 Thread Rohith Sharma K S 
+ common/yarn and mapreduce/submarine

Looks like same issue in submarine repository also !


On Mon, 21 Oct 2019 at 09:30, Rohith Sharma K S 
wrote:

> Folks,
>
> In Hadoop world, any mailing list has its own purposes as below
> 1. hdfs/common/yarn/mapreduce-*dev *mailing list is meant for developer
> discussion purpose.
> 2. hdfs/common/yarn/mapreduce*-issues* mailing list used for comments
> made in the issues.
>
>  It appears Hadoop-Ozone repository configured *hdfs-dev *mailing list
> for *hdfs-issues* list also. As a result hdfs-dev mailing list is
> bombarded with every comment made in hadoop-ozone repository.
>
>
> Could it be fixed?
>
> -Rohith Sharma K S
>
>
>

[GitHub] [hadoop-ozone] timmylicheng commented on issue #28: HDDS-1569 Support creating multiple pipelines with same datanode

2019-10-20 Thread GitBox 
timmylicheng commented on issue #28: HDDS-1569 Support creating multiple 
pipelines with same datanode
URL: https://github.com/apache/hadoop-ozone/pull/28#issuecomment-544338358
 
 
   /retest


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] timmylicheng commented on issue #28: HDDS-1569 Support creating multiple pipelines with same datanode

2019-10-20 Thread GitBox 
timmylicheng commented on issue #28: HDDS-1569 Support creating multiple 
pipelines with same datanode
URL: https://github.com/apache/hadoop-ozone/pull/28#issuecomment-544325211
 
 
   /retest


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] dchitlangia commented on a change in pull request #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
dchitlangia commented on a change in pull request #62: HDDS-2335. Params not 
included in AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62#discussion_r336815546
 
 

 ##
 File path: 
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditMessage.java
 ##
 @@ -120,7 +120,7 @@ public Builder withException(Throwable ex){
 public AuditMessage build(){
   AuditMessage auditMessage = new AuditMessage();
   auditMessage.message = "user=" + this.user + " | ip=" + this.ip + " | " +
-  "op=" + this.op + " | " + "ret=" + this.ret;
+  "op=" + this.op + " " + this.params + " | " + "ret=" + this.ret;
 
 Review comment:
   @bharatviswa504 `params=` is not required. the old format was something like:
   `. | op=OPERATION_NAME [params] |..`
   So changes from @adoroszlai are fine.
   
   This seems to have been broken by HDDS-2323. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 commented on issue #64: HDDS-2337. Fix checkstyle errors

2019-10-20 Thread GitBox 
mukul1987 commented on issue #64: HDDS-2337. Fix checkstyle errors
URL: https://github.com/apache/hadoop-ozone/pull/64#issuecomment-544312356
 
 
   /recheck
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 merged pull request #63: HDDS-2336. Fix TestKeyValueContainer#testRocksDBCreateUsesCachedOptions

2019-10-20 Thread GitBox 
mukul1987 merged pull request #63: HDDS-2336. Fix 
TestKeyValueContainer#testRocksDBCreateUsesCachedOptions
URL: https://github.com/apache/hadoop-ozone/pull/63
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on issue #65: HDDS-2334. Dummy chunk manager fails with length mismatch error

2019-10-20 Thread GitBox 
adoroszlai commented on issue #65: HDDS-2334. Dummy chunk manager fails with 
length mismatch error
URL: https://github.com/apache/hadoop-ozone/pull/65#issuecomment-544287990
 
 
   @bshashikant @szetszwo please review
   
   (Checkstyle, Findbugs and unit test failures are unrelated (and being fixed 
in other PRs)).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
bharatviswa504 commented on a change in pull request #62: HDDS-2335. Params not 
included in AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62#discussion_r336791506
 
 

 ##
 File path: 
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/audit/AuditMessage.java
 ##
 @@ -120,7 +120,7 @@ public Builder withException(Throwable ex){
 public AuditMessage build(){
   AuditMessage auditMessage = new AuditMessage();
   auditMessage.message = "user=" + this.user + " | ip=" + this.ip + " | " +
-  "op=" + this.op + " | " + "ret=" + this.ret;
+  "op=" + this.op + " " + this.params + " | " + "ret=" + this.ret;
 
 Review comment:
   Missing params = 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on issue #26: HDDS-2305. Update Ozone to latest ratis snapshot(0.5.0-3f446aa-SNAPSHOT). Contributed by Mukul Kumar Singh.

2019-10-20 Thread GitBox 
adoroszlai commented on issue #26: HDDS-2305. Update Ozone to latest ratis 
snapshot(0.5.0-3f446aa-SNAPSHOT). Contributed by  Mukul Kumar Singh.
URL: https://github.com/apache/hadoop-ozone/pull/26#issuecomment-544271571
 
 
   Thanks @mukul1987.  OM HA acceptance test works fine with 
https://github.com/apache/hadoop-ozone/pull/23/commits/4b03f42a28323a712f9ef6e19d203f8638651830


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #65: HDDS-2334. Dummy chunk manager fails with length mismatch error

2019-10-20 Thread GitBox 
adoroszlai opened a new pull request #65: HDDS-2334. Dummy chunk manager fails 
with length mismatch error
URL: https://github.com/apache/hadoop-ozone/pull/65
 
 
   ## What changes were proposed in this pull request?
   
   Data size validation logic was recently 
[changed](https://github.com/apache/hadoop-ozone/commit/e70ea7b66ca3326c3b00ddc3e4af7144d48ea5f5#diff-92341865368a6b82a1430bcb40bd4264R83)
 for real `ChunkManager`, but not for the dummy implementation.  This change 
extracts the validation logic and reuses it for the dummy one, too.  This 
restores the ability to skip writing data to disk (for performance testing).
   
   https://issues.apache.org/jira/browse/HDDS-2334
   
   ## How was this patch tested?
   
   Changed existing unit test to use a buffer with additional "header" at the 
beginning.  Added test cases for dummy implementation.
   
   Tested on compose cluster with the following additional configs:
   
   ```
   OZONE-SITE.XML_hdds.container.chunk.persistdata=false
   OZONE-SITE.XML_ozone.client.verify.checksum=false
   ```
   
   ```
   $ ozone sh volume create vol1
   $ ozone sh bucket create vol1/buck1;
   $ ozone sh key put vol1/buck1/key1 /etc/passwd
   $ ozone sh key get vol1/buck1/key1 asdf
   $ ls -l /etc/passwd
   -rw-r--r-- 1 root root 671 Jun 17 15:33 /etc/passwd
   $ wc asdf
 0   0 671 asdf
   ```
   
   Also tested regular, "persistent" chunk manager:
   
   ```
   $ docker-compose exec scm ozone freon rk --numOfThreads 1 --numOfVolumes 1 
--numOfBuckets 1 --replicationType RATIS --factor ONE --validateWrites 
--keySize 1024 --numOfKeys 10 --bufferSize 1024
   ...
   Status: Success
   Git Base Revision: e97acb3bd8f3befd27418996fa5d4b50bf2e17bf
   Number of Volumes created: 1
   Number of Buckets created: 1
   Number of Keys added: 10
   Ratis replication factor: ONE
   Ratis replication type: RATIS
   Average Time spent in volume creation: 00:00:00,182
   Average Time spent in bucket creation: 00:00:00,030
   Average Time spent in key creation: 00:00:00,290
   Average Time spent in key write: 00:00:02,379
   Total bytes written: 10240
   Total number of writes validated: 10
   Writes validated: 100.0 %
   Successful validation: 10
   Unsuccessful validation: 0
   Total Execution time: 00:00:09,389
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on issue #64: HDDS-2337. Fix checkstyle errors

2019-10-20 Thread GitBox 
adoroszlai commented on issue #64: HDDS-2337. Fix checkstyle errors
URL: https://github.com/apache/hadoop-ozone/pull/64#issuecomment-544264670
 
 
   Findbugs being fixed in #62, unit test in #63.
   
   @bshashikant @mukul1987 please review


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on issue #63: HDDS-2336. Fix TestKeyValueContainer#testRocksDBCreateUsesCachedOptions

2019-10-20 Thread GitBox 
adoroszlai commented on issue #63: HDDS-2336. Fix 
TestKeyValueContainer#testRocksDBCreateUsesCachedOptions
URL: https://github.com/apache/hadoop-ozone/pull/63#issuecomment-544264509
 
 
   
https://github.com/elek/ozone-ci-q4/blob/2dcb51dec5ccd866604c361d62f8b04c1188e17b/pr/pr-hdds-2336-74zpv/unit/output.log#L488-L489:
   
   ```
   [INFO] Running 
org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer
   [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
2.529 s - in org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer
   ```
   
   Findbugs being fixed in #62, checkstyle in #64.
   
   @swagle @mukul1987 please review


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on issue #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
adoroszlai commented on issue #62: HDDS-2335. Params not included in 
AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62#issuecomment-544245303
 
 
   @dineshchitlangia @mukul1987 @swagle please review


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai commented on issue #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
adoroszlai commented on issue #62: HDDS-2335. Params not included in 
AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62#issuecomment-544245260
 
 
   Unit test failure is not related, being fixed in #63.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #64: HDDS-2337. Fix checkstyle errors

2019-10-20 Thread GitBox 
adoroszlai opened a new pull request #64: HDDS-2337. Fix checkstyle errors
URL: https://github.com/apache/hadoop-ozone/pull/64
 
 
   ## What changes were proposed in this pull request?
   
   Fix current [checkstyle 
errors](https://github.com/elek/ozone-ci-q4/blob/master/pr/pr-hdds-2281-wfpgn/checkstyle/summary.txt).
   
   Also:
   
* fix some log message placeholder vs. parameter count mismatch.
* remove `NoSuchAlgorithmException` from javadoc where not declared to be 
thrown
   
   https://issues.apache.org/jira/browse/HDDS-2337
   
   ## How was this patch tested?
   
   Ran checkstyle.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDDS-2337) Fix checkstyle errors

2019-10-20 Thread Attila Doroszlai (Jira) 
Attila Doroszlai created HDDS-2337:
--

 Summary: Fix checkstyle errors
 Key: HDDS-2337
 URL: https://issues.apache.org/jira/browse/HDDS-2337
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


Checkstyle errors intoduced in HDDS-2281:

{noformat:title=https://github.com/elek/ozone-ci-q4/blob/master/pr/pr-hdds-2281-wfpgn/checkstyle/summary.txt}
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
 465: Line is longer than 80 characters (found 81).
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ContainerTestHelper.java
 244: Line is longer than 80 characters (found 84).
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineFailures.java
 30: Unused import - 
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException.
 506: ; is preceded with whitespace.
 517: ; is preceded with whitespace.
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #63: HDDS-2336. Fix TestKeyValueContainer#testRocksDBCreateUsesCachedOptions

2019-10-20 Thread GitBox 
adoroszlai opened a new pull request #63: HDDS-2336. Fix 
TestKeyValueContainer#testRocksDBCreateUsesCachedOptions
URL: https://github.com/apache/hadoop-ozone/pull/63
 
 
   ## What changes were proposed in this pull request?
   
   `testRocksDBCreateUsesCachedOptions` is 
[failing](https://github.com/elek/ozone-ci-q4/blob/master/pr/pr-hdds-2283-cnrrq/unit/hadoop-hdds/container-service/org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer.txt)
 because each test method in `TestKeyValueContainer` introduces a new entry in 
`MetadataStoreBuilder.CACHED_OPTS`, since `Configuration` does not implement 
`equals`.  Thus `testRocksDBCreateUsesCachedOptions` passes by itself, but 
fails when the whole test class is run.
   
   https://issues.apache.org/jira/browse/HDDS-2336
   
   ## How was this patch tested?
   
   Unit test.  No other code changed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDDS-2336) Fix TestKeyValueContainer#testRocksDBCreateUsesCachedOptions

2019-10-20 Thread Attila Doroszlai (Jira) 
Attila Doroszlai created HDDS-2336:
--

 Summary: Fix 
TestKeyValueContainer#testRocksDBCreateUsesCachedOptions
 Key: HDDS-2336
 URL: https://issues.apache.org/jira/browse/HDDS-2336
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: test
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


TestKeyValueContainer#testRocksDBCreateUsesCachedOptions, introduced in 
HDDS-2283, is failing:

{noformat:title=https://github.com/elek/ozone-ci-q4/blob/master/pr/pr-hdds-2283-cnrrq/unit/hadoop-hdds/container-service/org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer.txt}
testRocksDBCreateUsesCachedOptions(org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer)
  Time elapsed: 0.135 s  <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<11>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at 
org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer.testRocksDBCreateUsesCachedOptions(TestKeyValueContainer.java:406)
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDDS-2280) HddsUtils#CheckForException should not return null in case the ratis exception cause is not set

2019-10-20 Thread Mukul Kumar Singh (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDDS-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukul Kumar Singh resolved HDDS-2280.
-
Fix Version/s: 0.5.0
   Resolution: Fixed

Thanks for the contribution [~shashikant] and [~bharat] for the review. I have 
committed this.

> HddsUtils#CheckForException should not return null in case the ratis 
> exception cause is not set
> ---
>
> Key: HDDS-2280
> URL: https://issues.apache.org/jira/browse/HDDS-2280
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Client
>Affects Versions: 0.5.0
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> HddsUtils#CheckForException checks for the cause to be set properly to one of 
> the defined/expected exceptions. In case, ratis throws up any runtime 
> exception, HddsUtils#CheckForException can return null and lead to 
> NullPointerException while write.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 merged pull request #57: HDDS-2280. HddsUtils#CheckForException should not return null in case the ratis exception cause is not set

2019-10-20 Thread GitBox 
mukul1987 merged pull request #57: HDDS-2280. HddsUtils#CheckForException 
should not return null in case the ratis exception cause is not set 
URL: https://github.com/apache/hadoop-ozone/pull/57
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDDS-2281) ContainerStateMachine#handleWriteChunk should ignore close container exception

2019-10-20 Thread Mukul Kumar Singh (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDDS-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukul Kumar Singh resolved HDDS-2281.
-
Resolution: Fixed

Thanks for the contribution [~shashikant]. I have committed this.

> ContainerStateMachine#handleWriteChunk should ignore close container 
> exception 
> ---
>
> Key: HDDS-2281
> URL: https://issues.apache.org/jira/browse/HDDS-2281
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Datanode
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Currently, ContainerStateMachine#applyTrannsaction ignores close container 
> exception.Similarly,ContainerStateMachine#handleWriteChunk call also should 
> ignore close container exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 merged pull request #54: HDDS-2281. ContainerStateMachine#handleWriteChunk should ignore close container exception

2019-10-20 Thread GitBox 
mukul1987 merged pull request #54: HDDS-2281. 
ContainerStateMachine#handleWriteChunk should ignore close container exception
URL: https://github.com/apache/hadoop-ozone/pull/54
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines should be marked as ready only after the leader election is complete.

2019-10-20 Thread GitBox 
mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines 
should be marked as ready only after the leader election is complete.
URL: https://github.com/apache/hadoop-ozone/pull/23#discussion_r336773422
 
 

 ##
 File path: 
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineReportHandler.java
 ##
 @@ -50,6 +52,8 @@
   private final Configuration conf;
   private final SCMSafeModeManager scmSafeModeManager;
   private final boolean pipelineAvailabilityCheck;
+  private Map>
 
 Review comment:
   Entries from this map need to be removed on pipeline destroy. I think we 
should maintain in this pipelineStateManger. @nandakumar131 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines should be marked as ready only after the leader election is complete.

2019-10-20 Thread GitBox 
mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines 
should be marked as ready only after the leader election is complete.
URL: https://github.com/apache/hadoop-ozone/pull/23#discussion_r336773105
 
 

 ##
 File path: 
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java
 ##
 @@ -686,4 +693,17 @@ void notifyGroupRemove(RaftGroupId gid) {
   void notifyGroupAdd(RaftGroupId gid) {
 raftGids.add(gid);
 
 Review comment:
   lets add an entry to  leaderIdMap on groupAdd and remove it on groupRemove, 
else we can leak memory.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines should be marked as ready only after the leader election is complete.

2019-10-20 Thread GitBox 
mukul1987 commented on a change in pull request #23: HDDS-1868. Ozone pipelines 
should be marked as ready only after the leader election is complete.
URL: https://github.com/apache/hadoop-ozone/pull/23#discussion_r336772497
 
 

 ##
 File path: 
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java
 ##
 @@ -95,6 +98,8 @@
 
   // Timeout for the cluster to be ready
   private int waitForClusterToBeReadyTimeout = 6; // 1 min
+  // Timeout for all/any pipelines to be in open state
+  private int waitForPipelineOpenTimeout = 6;
 
 Review comment:
   This is not used anywhere, lets remove it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #62: HDDS-2335. Params not included in AuditMessage

2019-10-20 Thread GitBox 
adoroszlai opened a new pull request #62: HDDS-2335. Params not included in 
AuditMessage
URL: https://github.com/apache/hadoop-ozone/pull/62
 
 
   ## What changes were proposed in this pull request?
   
   Include operation parameters in audit messages like before 
[HDDS-2323](https://github.com/apache/hadoop-ozone/commit/b9618834c9902fc8fd9ae12872092cfb1e5c1be3).
   
   https://issues.apache.org/jira/browse/HDDS-2335
   
   ## How was this patch tested?
   
   Added unit test.
   
   Verified Findbugs violation is fixed.
   
   Tested using Freon:
   
   ```
   2019-10-20 08:54:19,437 | INFO  | OMAudit | user=hadoop | ip=192.168.144.3 | 
op=CREATE_VOLUME {admin=hadoop, owner=hadoop, volume=vol-0-52867, 
creationTime=1571561659397, quotaInBytes=1152921504606846976, objectID=1, 
updateID=1} | ret=SUCCESS |
   2019-10-20 08:54:19,497 | INFO  | OMAudit | user=hadoop | ip=192.168.144.3 | 
op=CREATE_BUCKET {volume=vol-0-52867, bucket=bucket-0-43473, gdprEnabled=null, 
acls=[user:hadoop:a[ACCESS], group:users:a[ACCESS]], isVersionEnabled=false, 
storageType=DISK, creationTime=1571561659483} | ret=SUCCESS |
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDDS-2335) Params not included in AuditMessage

2019-10-20 Thread Attila Doroszlai (Jira) 
Attila Doroszlai created HDDS-2335:
--

 Summary: Params not included in AuditMessage
 Key: HDDS-2335
 URL: https://issues.apache.org/jira/browse/HDDS-2335
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Affects Versions: 0.5.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


HDDS-2323 introduced the following Findbugs violation:

{noformat:title=https://github.com/elek/ozone-ci-q4/blob/master/trunk/trunk-nightly-20191020-r5wzl/findbugs/summary.txt}
M P UrF: Unread field: 
org.apache.hadoop.ozone.audit.AuditMessage$Builder.params  At 
AuditMessage.java:[line 106]
{noformat}

Which reveals that {{params}} is now not logged in audit messages:

{noformat}
2019-10-20 08:41:35,248 | INFO  | OMAudit | user=hadoop | ip=192.168.128.2 | 
op=CREATE_VOLUME | ret=SUCCESS |
2019-10-20 08:41:35,312 | INFO  | OMAudit | user=hadoop | ip=192.168.128.2 | 
op=CREATE_BUCKET | ret=SUCCESS |
2019-10-20 08:41:35,407 | INFO  | OMAudit | user=hadoop | ip=192.168.128.2 | 
op=ALLOCATE_KEY | ret=SUCCESS |
2019-10-20 08:41:37,355 | INFO  | OMAudit | user=hadoop | ip=192.168.128.2 | 
op=COMMIT_KEY | ret=SUCCESS |
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDDS-2334) Dummy chunk manager fails with length mismatch error

2019-10-20 Thread Attila Doroszlai (Jira) 
Attila Doroszlai created HDDS-2334:
--

 Summary: Dummy chunk manager fails with length mismatch error
 Key: HDDS-2334
 URL: https://issues.apache.org/jira/browse/HDDS-2334
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
  Components: test
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai


HDDS-1094 added a config option ({{hdds.container.chunk.persistdata=false}}) to 
drop chunks instead of writing them to disk.  Currently this option triggers 
the following error with any key size:

{noformat}
org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException: 
data array does not match the length specified. DataLen: 16777216 Byte Array: 
16777478
at 
org.apache.hadoop.ozone.container.keyvalue.impl.ChunkManagerDummyImpl.writeChunk(ChunkManagerDummyImpl.java:87)
at 
org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.handleWriteChunk(KeyValueHandler.java:695)
at 
org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.handle(KeyValueHandler.java:176)
at 
org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatchRequest(HddsDispatcher.java:277)
at 
org.apache.hadoop.ozone.container.common.impl.HddsDispatcher.dispatch(HddsDispatcher.java:150)
at 
org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.dispatchCommand(ContainerStateMachine.java:413)
at 
org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.runCommand(ContainerStateMachine.java:423)
at 
org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.lambda$handleWriteChunk$1(ContainerStateMachine.java:458)
at 
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDDS-2283) Container creation on datanodes take time because of Rocksdb option creation.

2019-10-20 Thread Mukul Kumar Singh (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDDS-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukul Kumar Singh resolved HDDS-2283.
-
Resolution: Fixed

> Container creation on datanodes take time because of Rocksdb option creation.
> -
>
> Key: HDDS-2283
> URL: https://issues.apache.org/jira/browse/HDDS-2283
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>  Components: Ozone Datanode
>Reporter: Mukul Kumar Singh
>Assignee: Siddharth Wagle
>Priority: Major
>  Labels: pull-request-available
> Attachments: HDDS-2283.00.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Container Creation on datanodes take around 300ms due to rocksdb creation. 
> Rocksdb creation is taking a considerable time and this needs to be optimized.
> Creating a rocksdb per disk should be enough and each container can be table 
> inside the rocksdb.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 merged pull request #41: HDDS-2283. Container Creation on datanodes take around 300ms due to rocksdb creation.

2019-10-20 Thread GitBox 
mukul1987 merged pull request #41: HDDS-2283. Container Creation on datanodes 
take around 300ms due to rocksdb creation.
URL: https://github.com/apache/hadoop-ozone/pull/41
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 commented on issue #26: HDDS-2305. Update Ozone to latest ratis snapshot(0.5.0-3f446aa-SNAPSHOT). Contributed by Mukul Kumar Singh.

2019-10-20 Thread GitBox 
mukul1987 commented on issue #26: HDDS-2305. Update Ozone to latest ratis 
snapshot(0.5.0-3f446aa-SNAPSHOT). Contributed by  Mukul Kumar Singh.
URL: https://github.com/apache/hadoop-ozone/pull/26#issuecomment-544225132
 
 
   @swagle is updating the ratis version with 
https://github.com/apache/hadoop-ozone/pull/23.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDDS-2286) Add a log info in ozone client and scm to print the exclusion list during allocate block

2019-10-20 Thread Mukul Kumar Singh (Jira) 


 [ 
https://issues.apache.org/jira/browse/HDDS-2286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukul Kumar Singh resolved HDDS-2286.
-
Resolution: Fixed

Thanks for the contribution [~swagle] and [~adoroszlai] for the review. I have 
committed this.

> Add a log info in ozone client and scm to print the exclusion list during 
> allocate block
> 
>
> Key: HDDS-2286
> URL: https://issues.apache.org/jira/browse/HDDS-2286
> Project: Hadoop Distributed Data Store
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: Shashikant Banerjee
>Assignee: Siddharth Wagle
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[GitHub] [hadoop-ozone] mukul1987 merged pull request #46: HDDS-2286. Add a log info in ozone client and scm to print the exclus…

2019-10-20 Thread GitBox 
mukul1987 merged pull request #46: HDDS-2286. Add a log info in ozone client 
and scm to print the exclus…
URL: https://github.com/apache/hadoop-ozone/pull/46
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org