[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #259: HDDS-2619. Fix Sonar issues in ReconTaskControllerImpl.
avijayanhwx commented on a change in pull request #259: HDDS-2619. Fix Sonar
issues in ReconTaskControllerImpl.
URL: https://github.com/apache/hadoop-ozone/pull/259#discussion_r352994031
##
File path:
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java
##
@@ -179,8 +186,8 @@ public void reInitializeTasks(OMMetadataManager
omMetadataManager)
List> results = executorService.invokeAll(tasks);
for (Future f : results) {
String taskName = f.get().getLeft().toString();
-if (!(Boolean)f.get().getRight()) {
- LOG.info("Init failed for task : " + taskName);
+if (!(boolean)f.get().getRight()) {
Review comment:
Thank you @adoroszlai. Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #259: HDDS-2619. Fix Sonar issues in ReconTaskControllerImpl.
avijayanhwx commented on a change in pull request #259: HDDS-2619. Fix Sonar issues in ReconTaskControllerImpl. URL: https://github.com/apache/hadoop-ozone/pull/259#discussion_r352993871 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/ReconTaskControllerImpl.java ## @@ -134,7 +134,6 @@ public void consumeOMEvents(OMUpdateEventBatch events, } // Reprocess the failed tasks. -// TODO Move to a separate task queue since reprocess may be a heavy // operation for large OM DB instances Review comment: Thank you @adoroszlai. Removed that line. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-2657) Key get command creates the output file even in case of KEY_NOT_FOUND
[
https://issues.apache.org/jira/browse/HDDS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986274#comment-16986274
]
Attila Doroszlai commented on HDDS-2657:
Thanks [~ayushtkn] for reporting this issue. I have tested the fix, it works
fine.
> Key get command creates the output file even in case of KEY_NOT_FOUND
> -
>
> Key: HDDS-2657
> URL: https://issues.apache.org/jira/browse/HDDS-2657
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
> Attachments: HDDS-2657-01.patch
>
>
> If a key isn't found, the key get command still creates an output file of 0
> size.
> {noformat}
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:41 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ./ozone sh key get "hive/bucket/key 21" "file 21"
> KEY_NOT_FOUND Key not found
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:42 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rw-r--r-- 1 ayush ayush 0 Dec 2 22:42 'file 21'
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-2657) Key get command creates the output file even in case of KEY_NOT_FOUND
[
https://issues.apache.org/jira/browse/HDDS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986228#comment-16986228
]
Ayush Saxena commented on HDDS-2657:
HDDS-2580, Could be a reason for this,
The output stream was created before the input stream, so if there is no key,
still the output stream creates the empty file.
Uploaded fix.
Verified :
{noformat}
ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
./ozone sh key get "hive/bucket/key 21" "file 21"
KEY_NOT_FOUND Key not found
ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
ll
total 24
drwxr-xr-x 2 ayush ayush 4096 Dec 2 22:55 ./
drwxr-xr-x 13 ayush ayush 4096 Dec 2 23:05 ../
-rwxr-xr-x 1 ayush ayush 12786 Dec 2 22:55 ozone*
{noformat}
Pls Review!!!
> Key get command creates the output file even in case of KEY_NOT_FOUND
> -
>
> Key: HDDS-2657
> URL: https://issues.apache.org/jira/browse/HDDS-2657
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
> Attachments: HDDS-2657-01.patch
>
>
> If a key isn't found, the key get command still creates an output file of 0
> size.
> {noformat}
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:41 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ./ozone sh key get "hive/bucket/key 21" "file 21"
> KEY_NOT_FOUND Key not found
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:42 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rw-r--r-- 1 ayush ayush 0 Dec 2 22:42 'file 21'
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2657) Key get command creates the output file even in case of KEY_NOT_FOUND
[
https://issues.apache.org/jira/browse/HDDS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDDS-2657:
---
Status: Patch Available (was: Open)
> Key get command creates the output file even in case of KEY_NOT_FOUND
> -
>
> Key: HDDS-2657
> URL: https://issues.apache.org/jira/browse/HDDS-2657
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
> Attachments: HDDS-2657-01.patch
>
>
> If a key isn't found, the key get command still creates an output file of 0
> size.
> {noformat}
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:41 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ./ozone sh key get "hive/bucket/key 21" "file 21"
> KEY_NOT_FOUND Key not found
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:42 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rw-r--r-- 1 ayush ayush 0 Dec 2 22:42 'file 21'
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2657) Key get command creates the output file even in case of KEY_NOT_FOUND
[
https://issues.apache.org/jira/browse/HDDS-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDDS-2657:
---
Attachment: HDDS-2657-01.patch
> Key get command creates the output file even in case of KEY_NOT_FOUND
> -
>
> Key: HDDS-2657
> URL: https://issues.apache.org/jira/browse/HDDS-2657
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
> Attachments: HDDS-2657-01.patch
>
>
> If a key isn't found, the key get command still creates an output file of 0
> size.
> {noformat}
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:41 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ./ozone sh key get "hive/bucket/key 21" "file 21"
> KEY_NOT_FOUND Key not found
> ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
> ll
> total 36
> drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:42 ./
> drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
> -rw-r--r-- 1 ayush ayush 0 Dec 2 22:42 'file 21'
> -rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Created] (HDDS-2657) Key get command creates the output file even in case of KEY_NOT_FOUND
Ayush Saxena created HDDS-2657:
--
Summary: Key get command creates the output file even in case of
KEY_NOT_FOUND
Key: HDDS-2657
URL: https://issues.apache.org/jira/browse/HDDS-2657
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Ayush Saxena
If a key isn't found, the key get command still creates an output file of 0
size.
{noformat}
ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
ll
total 36
drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:41 ./
drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
-rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
./ozone sh key get "hive/bucket/key 21" "file 21"
KEY_NOT_FOUND Key not found
ayush@ayushpc:~/ozone/hadoop-ozone/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/bin$
ll
total 36
drwxr-xr-x 3 ayush ayush 4096 Dec 2 22:42 ./
drwxr-xr-x 13 ayush ayush 4096 Nov 30 19:39 ../
-rw-r--r-- 1 ayush ayush 0 Dec 2 22:42 'file 21'
-rwxr-xr-x 1 ayush ayush 12786 Nov 30 19:35 ozone*
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2475) Unregister ContainerMetadataScrubberMetrics on thread exit
[
https://issues.apache.org/jira/browse/HDDS-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-2475:
---
Status: Patch Available (was: In Progress)
> Unregister ContainerMetadataScrubberMetrics on thread exit
> --
>
> Key: HDDS-2475
> URL: https://issues.apache.org/jira/browse/HDDS-2475
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{ContainerMetadataScanner}} thread should call
> {{ContainerMetadataScrubberMetrics#unregister}} before exiting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2654) Conditionally enable profiling at the kernel level
[
https://issues.apache.org/jira/browse/HDDS-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-2654:
---
Status: Patch Available (was: In Progress)
> Conditionally enable profiling at the kernel level
> --
>
> Key: HDDS-2654
> URL: https://issues.apache.org/jira/browse/HDDS-2654
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: docker
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Extend {{entrypoint.sh}} to set the kernel parameters required for profiling
> if the {{ASYNC_PROFILER_HOME}} environment variable (used by
> {{ProfileServlet}}) is set.
> Ref:
> {code:title=https://cwiki.apache.org/confluence/display/HADOOP/Java+Profiling+of+Ozone}
> echo 1 > /proc/sys/kernel/perf_event_paranoid
> echo 0 > /proc/sys/kernel/kptr_restrict
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2654) Conditionally enable profiling at the kernel level
[
https://issues.apache.org/jira/browse/HDDS-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-2654:
-
Labels: pull-request-available (was: )
> Conditionally enable profiling at the kernel level
> --
>
> Key: HDDS-2654
> URL: https://issues.apache.org/jira/browse/HDDS-2654
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: docker
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
> Labels: pull-request-available
>
> Extend {{entrypoint.sh}} to set the kernel parameters required for profiling
> if the {{ASYNC_PROFILER_HOME}} environment variable (used by
> {{ProfileServlet}}) is set.
> Ref:
> {code:title=https://cwiki.apache.org/confluence/display/HADOOP/Java+Profiling+of+Ozone}
> echo 1 > /proc/sys/kernel/perf_event_paranoid
> echo 0 > /proc/sys/kernel/kptr_restrict
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #287: HDDS-2654. Conditionally enable profiling at the kernel level
adoroszlai opened a new pull request #287: HDDS-2654. Conditionally enable profiling at the kernel level URL: https://github.com/apache/hadoop-ozone/pull/287 ## What changes were proposed in this pull request? Extend `entrypoint.sh` to set the kernel parameters required for profiling if the `ASYNC_PROFILER_HOME` environment variable (used by `ProfileServlet`) is defined. https://issues.apache.org/jira/browse/HDDS-2654 ## How was this patch tested? ``` $ cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozone $ docker-compose up -d $ docker-compose logs scm | grep 'Enabled profiling' scm_1 | Enabled profiling in kernel # wait for OM $ open http://localhost:9874/prof ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-2654) Conditionally enable profiling at the kernel level
[
https://issues.apache.org/jira/browse/HDDS-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-2654:
---
Description:
Extend {{entrypoint.sh}} to set the kernel parameters required for profiling if
the {{ASYNC_PROFILER_HOME}} environment variable (used by {{ProfileServlet}})
is set.
Ref:
{code:title=https://cwiki.apache.org/confluence/display/HADOOP/Java+Profiling+of+Ozone}
echo 1 > /proc/sys/kernel/perf_event_paranoid
echo 0 > /proc/sys/kernel/kptr_restrict
{code}
was:
Extend {{entrypoint.sh}} to set the kernel parameters required for profiling if
the {{ASYNC_PROFILER_ENABLED}} environment variable is set.
Ref:
{code:title=https://cwiki.apache.org/confluence/display/HADOOP/Java+Profiling+of+Ozone}
echo 1 > /proc/sys/kernel/perf_event_paranoid
echo 0 > /proc/sys/kernel/kptr_restrict
{code}
> Conditionally enable profiling at the kernel level
> --
>
> Key: HDDS-2654
> URL: https://issues.apache.org/jira/browse/HDDS-2654
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: docker
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
>
> Extend {{entrypoint.sh}} to set the kernel parameters required for profiling
> if the {{ASYNC_PROFILER_HOME}} environment variable (used by
> {{ProfileServlet}}) is set.
> Ref:
> {code:title=https://cwiki.apache.org/confluence/display/HADOOP/Java+Profiling+of+Ozone}
> echo 1 > /proc/sys/kernel/perf_event_paranoid
> echo 0 > /proc/sys/kernel/kptr_restrict
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
