[
https://issues.apache.org/jira/browse/HDDS-2042?focusedWorklogId=302183&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302183
]
ASF GitHub Bot logged work on HDDS-2042:
----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Aug/19 16:36
Start Date: 27/Aug/19 16:36
Worklog Time Spent: 10m
Work Description: adoroszlai commented on pull request #1357: HDDS-2042.
Avoid log on console with Ozone shell
URL: https://github.com/apache/hadoop/pull/1357
## What changes were proposed in this pull request?
Sample Docker Compose-based clusters are currently
[configured](https://github.com/apache/hadoop/blob/3329257d99d2808e66ae6c2fe87a9c4f8877026f/hadoop-ozone/dist/src/main/compose/ozone/docker-config#L32-L33)
to log almost everything to the console. This is useful because logs can be
accessed via `docker logs` or `docker-compose logs`. Further, [default logs
settings](https://github.com/apache/hadoop/blob/3329257d99d2808e66ae6c2fe87a9c4f8877026f/hadoop-ozone/dist/src/main/conf/log4j.properties#L137-L138)
for non-Docker Compose clusters also direct INFO and higher level messages to
the console.
However, it does not work well with interactive clients, eg. `ozone sh`,
since regular console output and log messages are mixed. Previously there
[were](https://issues.apache.org/jira/browse/HDDS-1489)
[attempts](https://issues.apache.org/jira/browse/HDDS-465) to work around this
by setting specific class log levels to `WARN` or `ERROR`. This approach has
two problems:
1. it applies to Ozone server processes, too
2. new INFO or higher level messages may be introduced any time (eg.
MetricsSystem startup)
This pull request proposes to collect logs for `ozone` subcommands `sh` and
`freon` to files instead of the console. It eliminates the need for per-class
log level settings (though current ones are kept).
https://issues.apache.org/jira/browse/HDDS-2042
## How was this patch tested?
Tested commands in sample docker cluster. Output is clean (except pesky
warning (omitted here) about illegal access for key operations).
```
$ cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozones3
$ docker-compose exec scm bash
bash-4.2$ ozone sh volume create vol1
bash-4.2$ ozone sh bucket create vol1/buck1
bash-4.2$ ozone sh key put vol1/buck1/key1 /etc/passwd
```
Verified that log file contains the messages previously sent to console:
```
bash-4.2$ tail /var/log/hadoop/ozone-shell.log
2019-08-27 14:45:02,695 [main] INFO RpcClient:293 - Creating Volume: vol1,
with hadoop as owner.
2019-08-27 14:45:13,099 [main] INFO RpcClient:432 - Creating Bucket:
vol1/buck1, with Versioning false and Storage Type set to DISK and Encryption
set to false
2019-08-27 14:45:24,011 [main] INFO MetricsConfig:118 - Loaded properties
from hadoop-metrics2.properties
2019-08-27 14:45:24,160 [main] INFO MetricsSystemImpl:374 - Scheduled
Metric snapshot period at 10 second(s).
2019-08-27 14:45:24,160 [main] INFO MetricsSystemImpl:191 -
XceiverClientMetrics metrics system started
```
Verbose mode still works:
```
bash-4.2$ ozone sh --verbose key put vol1/buck1/key2 /etc/passwd
Volume Name : vol1
Bucket Name : buck1
Key Name : key2
File Hash : b01f053617ddfeb782a3e757d9c08912
```
Freon:
```
bash-4.2$ ozone freon rk --numOfVolumes=1 --numOfBuckets=1 --numOfKeys=20
--numOfThreads=1
100.00%
|?????????????????????????????????????????????????????????????????????????????????????????????????????|
20/20 Time: 0:00:03
***************************************************
Status: Success
Git Base Revision: e97acb3bd8f3befd27418996fa5d4b50bf2e17bf
Number of Volumes created: 1
Number of Buckets created: 1
Number of Keys added: 20
Ratis replication factor: ONE
Ratis replication type: STAND_ALONE
Average Time spent in volume creation: 00:00:00,080
Average Time spent in bucket creation: 00:00:00,013
Average Time spent in key creation: 00:00:00,234
Average Time spent in key write: 00:00:02,310
Total bytes written: 204800
Total Execution time: 00:00:06,284
***************************************************
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 302183)
Remaining Estimate: 0h
Time Spent: 10m
> Avoid log on console with Ozone shell
> -------------------------------------
>
> Key: HDDS-2042
> URL: https://issues.apache.org/jira/browse/HDDS-2042
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: docker
> Reporter: Doroszlai, Attila
> Assignee: Doroszlai, Attila
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HDDS-1489 fixed several sample docker compose configs to avoid unnecessary
> messages on console when running eg. {{ozone sh key put}}. The goal of this
> task is to fix the remaining ones.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
