[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676325#comment-16676325 ] Sunil Govindan commented on HDFS-13403: --- Hi [~James C] pls help to check the Fixed Version. > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Fix For: 3.2.0, 3.3.0 > > Attachments: HDFS-13403.000.patch, build_fixes.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467494#comment-16467494 ] James Clampffer commented on HDFS-13403: Thanks for finding that [~bibinchundatt]! [~pifta] I posted a patch on HDFS-13534 to fix this if you'd like to review. > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch, build_fixes.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[
https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466343#comment-16466343
]
Bibin A Chundatt commented on HDFS-13403:
-
[~James C]
For version gcc 7 native compilation fails
[https://bugzilla.redhat.com/show_bug.cgi?id=1417383]
{code}
For reference, the , , and headers used to include the
whole of (thousands of lines), but now they don't. This is a Good
Thing™.
I'll make a note of this in the GCC 7 "porting to" doc.
{code}
Seems we have to explicitly include
> libhdfs++: Use hdfs::IoService object rather than asio::io_service
> --
>
> Key: HDFS-13403
> URL: https://issues.apache.org/jira/browse/HDFS-13403
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: James Clampffer
>Assignee: James Clampffer
>Priority: Critical
> Attachments: HDFS-13403.000.patch, build_fixes.patch
>
>
> At the moment the hdfs::IoService is a simple wrapper over asio's io_service
> object. I'd like to make this smarter and have it do things like track which
> tasks are queued, validate that dependencies of tasks exist, and monitor
> ioservice throughput and contention. In order to get there we need to use
> have all components in the library to go through the hdfs::IoService rather
> than directly interacting with the asio::io_service. The only time the
> asio::io_service should be used is when calling things like asio::async_write
> that need an io_service&. HDFS-11884 will be able get rid of those remaining
> instances once this work is in place.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465026#comment-16465026 ] Istvan Fajth commented on HDFS-13403: - Hi [~James C], I am wondering if the build_fixes.patch can be committed to the trunk? As I see it is still missing, I just ran into it again :) Thank you in advance! > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch, build_fixes.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446056#comment-16446056 ] Istvan Fajth commented on HDFS-13403: - Hi [~James C], yes, those are one type of the warnings, also I see warnings due to -Wunused-lambda-capture though after checking a few instance of this warning, they not seems to be related to this patch, though I just vaguely checked, but most likely they were there before as well. With the patch applied, I do not see inconsistent-missing-override warnings, and the include helped as well to get over the compile error. > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch, build_fixes.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441819#comment-16441819 ] James Clampffer commented on HDFS-13403: Hi [~pifta] Thanks for pointing this out! I'm not sure why the missing include isn't causing issues when I build on GCC and Clang in the docker container but functional should be included in ioservice.h. I do see some errors in the clang build due to -Winconsistent-missing-override that don't show up when using GCC. Are those what you're seeing? I'll get patch up with the include and virtual override warning fixes tomorrow. The post-commit jenkins build bailed out before it hit hadoop_hdfs_native_client so I don't think that's related. > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[
https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441466#comment-16441466
]
Istvan Fajth commented on HDFS-13403:
-
Hello [~James C],
I have run into an error when I tried to build the project on my Mac with `mvn
clean install -Pnative -DskipTests`. The compile failed at ioservice.h with the
following error:
{{ [exec] [ 30%] Building CXX object
main/native/libhdfspp/lib/common/CMakeFiles/common_obj.dir/ioservice_impl.cc.o}}
{{ [exec] In file included from
/Users//IdeaProjects/hadoop-apache/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/ioservice_impl.cc:19:}}
{{ [exec] In file included from
/Users//IdeaProjects/hadoop-apache/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/ioservice_impl.h:22:}}
{{ [exec]
/Users//IdeaProjects/hadoop-apache/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/include/hdfspp/ioservice.h:109:30:
error: no template named 'function' in namespace 'std'}}
{{ [exec] virtual void PostTask(std::function asyncTask) = 0;}}
{{ [exec] ~^}}
{{ }}
There are further compile errors as well, if needed I can provide a full output.
After adding #include to the ioservice.h the error goes away. I am
not sure if it is something in my environment, but the Jenkins build failed
somewhere else, though as I see it have skipped the Apache Hadoop HDFS Native
Client package.
> libhdfs++: Use hdfs::IoService object rather than asio::io_service
> --
>
> Key: HDFS-13403
> URL: https://issues.apache.org/jira/browse/HDFS-13403
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: James Clampffer
>Assignee: James Clampffer
>Priority: Critical
> Attachments: HDFS-13403.000.patch
>
>
> At the moment the hdfs::IoService is a simple wrapper over asio's io_service
> object. I'd like to make this smarter and have it do things like track which
> tasks are queued, validate that dependencies of tasks exist, and monitor
> ioservice throughput and contention. In order to get there we need to use
> have all components in the library to go through the hdfs::IoService rather
> than directly interacting with the asio::io_service. The only time the
> asio::io_service should be used is when calling things like asio::async_write
> that need an io_service&. HDFS-11884 will be able get rid of those remaining
> instances once this work is in place.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434003#comment-16434003 ] Hudson commented on HDFS-13403: --- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #13969 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13969/]) HDFS-13403: libhdfs++ Use hdfs::IoService object rather than (jhc: rev eefe2a147c83dbb62c4021b67d59d3b9f065f890) * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/reader/block_reader.h * (add) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/ioservice_impl.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/rpc_connection.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/include/hdfspp/hdfspp.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/rpc_connection_impl.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/namenode_tracker.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/rpc_engine.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/bindings/c/hdfs.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filesystem.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/bad_datanode_test.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/mock_connection.h * (delete) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/hdfs_ioservice.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filehandle.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/namenode_info.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filehandle.h * (add) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/include/hdfspp/ioservice.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/async_stream.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/namenode_operations.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/reader/block_reader.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/connection/datanodeconnection.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/reader/datatransfer.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/CMakeLists.txt * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/request.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/namenode_info.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ioservice_test.cc * (add) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/ioservice_impl.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/continuation/asio.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/rpc_connection_impl.h * (delete) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/hdfs_ioservice.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filesystem.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/rpc_engine_test.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/rpc_engine.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/logging.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/namenode_tracker.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/remote_block_reader_test.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/connection/datanodeconnection.cc > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical >
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434002#comment-16434002 ] James Clampffer commented on HDFS-13403: Thanks for the review [~mtracy]! I've committed this to trunk. > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431366#comment-16431366 ] Mitchell Tracy commented on HDFS-13403: --- +1 on HDFS-13403.000.patch > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[
https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431030#comment-16431030
]
genericqa commented on HDFS-13403:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
59m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 15m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 59s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
29s{color} | {color:green} hadoop-hdfs-native-client in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | HDFS-13403 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12918183/HDFS-13403.000.patch |
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 725a072e8a3c 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ac32b35 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/23840/testReport/ |
| Max. process+thread count | 361 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/23840/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> libhdfs++: Use hdfs::IoService object rather than asio::io_service
> --
>
> Key: HDFS-13403
> URL: https://issues.apache.org/jira/browse/HDFS-13403
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: James Clampffer
>Assignee: James Clampffer
>Priority: Critical
> Attachments: HDFS-13403.000.patch
>
>
> At the moment the hdfs::IoService is a simple wrapper over asio's io_service
> object. I'd like to make this smarter and have it do things like track which
> tasks are queued, validate that dependencies of tasks exist, and monitor
> ioservice throughput and contention. In order
[jira] [Commented] (HDFS-13403) libhdfs++: Use hdfs::IoService object rather than asio::io_service
[ https://issues.apache.org/jira/browse/HDFS-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430805#comment-16430805 ] James Clampffer commented on HDFS-13403: Attached a patch. Not a whole lot going on other than making uses of asio::io_service in arguments and member variables instances of shared_ptr. Since this already touched a lot of files I cleaned up a few things as I went: -Clear out includes in headers that weren't necessary. -Pull a few method implementations out of header files. -Rename the MutableBuffers typedef to MutableBuffer to avoid confusion with BufferSequences which aren't supported in the FileSystem API yet. > libhdfs++: Use hdfs::IoService object rather than asio::io_service > -- > > Key: HDFS-13403 > URL: https://issues.apache.org/jira/browse/HDFS-13403 > Project: Hadoop HDFS > Issue Type: Task >Reporter: James Clampffer >Assignee: James Clampffer >Priority: Critical > Attachments: HDFS-13403.000.patch > > > At the moment the hdfs::IoService is a simple wrapper over asio's io_service > object. I'd like to make this smarter and have it do things like track which > tasks are queued, validate that dependencies of tasks exist, and monitor > ioservice throughput and contention. In order to get there we need to use > have all components in the library to go through the hdfs::IoService rather > than directly interacting with the asio::io_service. The only time the > asio::io_service should be used is when calling things like asio::async_write > that need an io_service&. HDFS-11884 will be able get rid of those remaining > instances once this work is in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
