[jira] [Commented] (HDFS-10898) libhdfs++: Make logs more informative and consistent
[ https://issues.apache.org/jira/browse/HDFS-10898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545531#comment-15545531 ] Stephen commented on HDFS-10898: +1 The distinction between TRACE and DEBUG isn't clear to me, nor do I think it's important at this time. Perhaps another JIRA could pick that up. However, based on recent logs I've seen, the INFO to DEBUG conversions in this patch are absolutely necessary. > libhdfs++: Make logs more informative and consistent > > > Key: HDFS-10898 > URL: https://issues.apache.org/jira/browse/HDFS-10898 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Trivial > Attachments: HDFS-10898.HDFS-8707.000.patch > > > Most of the public C++ FileHandle/FileSystem operations have a LOG_TRACE > level message about parameters passed in etc. However many methods use > LOG_DEBUG and a couple use LOG_INFO. > We most likely want FS operations that happen a lot (read/open/seek/stat) to > stick to LOG_DEBUG consistently and only use LOG_INFO for things like > FileSystem::Connect or RpcConnection:: that don't get called often and are > important enough to warrant showing up in the log. LOG_TRACE can be reserved > for things happening deeper inside public methods and methods that aren't > part of the public API. > Related improvements that could be brought into this to avoid opening a ton > of small Jiras: > -Print the "this" pointer address in the log message to make it easier to > correlate objects when there's concurrent work being done. This has been > very helpful in the past but often got stripped out before patches went in. > People just need be aware that operator new may eventually place an object of > the same type at the same address sometime in the future. > -For objects owned by other objects, but created on the fly, include a > pointer back to the parent/creator object if that pointer is already being > tracked (See the nested stucts in BlockReaderImpl). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10898) libhdfs++: Make logs more informative and consistent
[
https://issues.apache.org/jira/browse/HDFS-10898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543417#comment-15543417
]
Bob Hansen commented on HDFS-10898:
---
What would you think of keeping SEEK and READ at the TRACE level since
consumers will tend to do those gazillions of times?
We should move the async FileHandleImpl::PositionRead to DEBUG also.
While we're here, can we make the INFO level message for Connect be more
human-readable ("Connecting to :"), perhaps with a DEBUG
level that includes debugging-appropriate data (this pointer, etc.). Have that
one only in the async version, so the consumer gets one INFO-level message per
connect. Not critical, but since whe're here...
> libhdfs++: Make logs more informative and consistent
>
>
> Key: HDFS-10898
> URL: https://issues.apache.org/jira/browse/HDFS-10898
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: James Clampffer
>Assignee: James Clampffer
>Priority: Trivial
> Attachments: HDFS-10898.HDFS-8707.000.patch
>
>
> Most of the public C++ FileHandle/FileSystem operations have a LOG_TRACE
> level message about parameters passed in etc. However many methods use
> LOG_DEBUG and a couple use LOG_INFO.
> We most likely want FS operations that happen a lot (read/open/seek/stat) to
> stick to LOG_DEBUG consistently and only use LOG_INFO for things like
> FileSystem::Connect or RpcConnection:: that don't get called often and are
> important enough to warrant showing up in the log. LOG_TRACE can be reserved
> for things happening deeper inside public methods and methods that aren't
> part of the public API.
> Related improvements that could be brought into this to avoid opening a ton
> of small Jiras:
> -Print the "this" pointer address in the log message to make it easier to
> correlate objects when there's concurrent work being done. This has been
> very helpful in the past but often got stripped out before patches went in.
> People just need be aware that operator new may eventually place an object of
> the same type at the same address sometime in the future.
> -For objects owned by other objects, but created on the fly, include a
> pointer back to the parent/creator object if that pointer is already being
> tracked (See the nested stucts in BlockReaderImpl).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10898) libhdfs++: Make logs more informative and consistent
[ https://issues.apache.org/jira/browse/HDFS-10898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542642#comment-15542642 ] James Clampffer commented on HDFS-10898: Looks like the CI failure is caused by the same assert that caused the first patch for HDFS-10931 to fail. Once HDFS-10931 lands with the test fix I'll resubmit this patch and see if it's cleared up, it passes in my docker environment locally. > libhdfs++: Make logs more informative and consistent > > > Key: HDFS-10898 > URL: https://issues.apache.org/jira/browse/HDFS-10898 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Trivial > Attachments: HDFS-10898.HDFS-8707.000.patch > > > Most of the public C++ FileHandle/FileSystem operations have a LOG_TRACE > level message about parameters passed in etc. However many methods use > LOG_DEBUG and a couple use LOG_INFO. > We most likely want FS operations that happen a lot (read/open/seek/stat) to > stick to LOG_DEBUG consistently and only use LOG_INFO for things like > FileSystem::Connect or RpcConnection:: that don't get called often and are > important enough to warrant showing up in the log. LOG_TRACE can be reserved > for things happening deeper inside public methods and methods that aren't > part of the public API. > Related improvements that could be brought into this to avoid opening a ton > of small Jiras: > -Print the "this" pointer address in the log message to make it easier to > correlate objects when there's concurrent work being done. This has been > very helpful in the past but often got stripped out before patches went in. > People just need be aware that operator new may eventually place an object of > the same type at the same address sometime in the future. > -For objects owned by other objects, but created on the fly, include a > pointer back to the parent/creator object if that pointer is already being > tracked (See the nested stucts in BlockReaderImpl). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10898) libhdfs++: Make logs more informative and consistent
[
https://issues.apache.org/jira/browse/HDFS-10898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15539762#comment-15539762
]
Hadoop QA commented on HDFS-10898:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
53s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
44s{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
56s{color} | {color:green} HDFS-8707 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
56s{color} | {color:green} HDFS-8707 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
53s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
55s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 11s{color}
| {color:red} hadoop-hdfs-native-client in the patch failed with JDK
v1.7.0_111. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 64m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_111 Failed CTEST tests |
test_libhdfs_mini_stress_hdfspp_test_shim_static |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:78fc6b6 |
| JIRA Issue | HDFS-10898 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831235/HDFS-10898.HDFS-8707.000.patch
|
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 4e14c0ec9455 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-8707 / feba09b |
| Default Java | 1.7.0_111 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_101
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 |
| CTEST |
https://builds.apache.org/job/PreCommit-HDFS-Build/16968/artifact/patchprocess/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.7.0_111-ctest.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/16968/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.7.0_111.txt
|
| JDK v1.7.0_111 Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/16968/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/16968/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was
