[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737891#comment-17737891
]
ASF GitHub Bot commented on HDFS-16930:
---
lfrancke commented on PR #5449:
URL: https://github.com/apache/hadoop/pull/5449#issuecomment-1610300957
If some of these things are not clear then we can maybe split out the bits
that can definitely be fixed.
e.g. the `LIBHDFS_PATH` fix.
I've stumbled across this as well tonight.
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Assignee: Chao-Heng Lee
>Priority: Minor
> Labels: pull-request-available
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697159#comment-17697159
]
ASF GitHub Bot commented on HDFS-16930:
---
jojochuang commented on code in PR #5449:
URL: https://github.com/apache/hadoop/pull/5449#discussion_r1127102374
##
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_dfs_wrapper.sh:
##
@@ -31,22 +31,13 @@ if [ "$JAVA_HOME" = "" ]; then
export JAVA_HOME=/usr/local/java
fi
-if [ "$LD_LIBRARY_PATH" = "" ]; then
-export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/$OS_ARCH/server:/usr/local/lib
-fi
-
-while IFS= read -r -d '' file
-do
- export CLASSPATH=$CLASSPATH:$file
-done < <(find "$HADOOP_HOME/hadoop-client" -name "*.jar" -print0)
-
while IFS= read -r -d '' file
do
export CLASSPATH=$CLASSPATH:$file
-done < <(find "$HADOOP_HOME/hadoop-hdfs-project" -name "*.jar" -print0)
+done < <(find "$HADOOP_HOME/hadoop-tools/hadoop-distcp" -name "*.jar" -print0)
Review Comment:
It's been a while but I don't think fuse-dfs relies on distcp jars
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Assignee: Chao-Heng Lee
>Priority: Minor
> Labels: pull-request-available
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695655#comment-17695655
]
ASF GitHub Bot commented on HDFS-16930:
---
hadoop-yetus commented on PR #5449:
URL: https://github.com/apache/hadoop/pull/5449#issuecomment-1451675091
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 12m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 37m 41s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5449/1/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 17s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 19m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 38s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 97m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5449/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5449 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets
shellcheck shelldocs |
| uname | Linux a0da5c06db93 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 685625159e24efbec4fb45f10a0cb6602b848337 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5449/1/testReport/ |
| Max. process+thread count | 558 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5449/1/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Assignee: Chao-Heng Lee
>Priority: Minor
> Labels: pull-request-available
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695575#comment-17695575
]
ASF GitHub Bot commented on HDFS-16930:
---
chaohengstudent opened a new pull request, #5449:
URL: https://github.com/apache/hadoop/pull/5449
### Description of PR
update path with LIBHDFS_PATH
add path to LD_LIBRARY_PATH with compatible to java11
edit CLASSPATH
### How was this patch tested?
manual test
```
root@6b76602de66c:/opt/hadoop# ./fuse_dfs_wrapper.sh -d
hdfs://192.168.103.44:14370 /mnt/hdfs
INFO
/home/chaoheng/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_options.c:115
Ignoring option -d
INFO
/home/chaoheng/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_options.c:164
Adding FUSE arg /mnt/hdfs
FUSE library version: 2.9.9
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 2, opcode: INIT (26), nodeid: 0, insize: 104, pid: 0
INIT: 7.36
flags=0x73fb
max_readahead=0x0002
INFO
/home/chaoheng/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_init.c:98
Mounting with options: [ protected=(NULL), nn_uri=hdfs://192.168.103.44:14370,
nn_port=0, debug=0, read_only=0, initchecks=0, no_permissions=0, usetrash=0,
entry_timeout=60, attribute_timeout=60, rdbuffer_size=10485760, direct_io=0 ]
fuseConnectInit: initialized with timer period 5, expiry period 300
INIT: 7.19
flags=0x0039
max_readahead=0x0002
max_write=0x0002
max_background=0
congestion_threshold=0
unique: 2, success, outsize: 40
```
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Assignee: Chao-Heng Lee
>Priority: Minor
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692132#comment-17692132
]
Ayush Saxena commented on HDFS-16930:
-
Added [~chaoheng] as HDFS contributor to assign the ticket.
Welcome to hadoop!!!
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Assignee: Chao-Heng Lee
>Priority: Minor
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692115#comment-17692115
]
Chao-Heng Lee commented on HDFS-16930:
--
I am newbie. I want to contribute this issue. Is there someone can assign this
issue to me?
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Priority: Minor
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16930) Update the wrapper for fuse-dfs
[
https://issues.apache.org/jira/browse/HDFS-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692107#comment-17692107
]
Chao-Heng Lee commented on HDFS-16930:
--
[~chia7712]
> Update the wrapper for fuse-dfs
> ---
>
> Key: HDFS-16930
> URL: https://issues.apache.org/jira/browse/HDFS-16930
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs
>Reporter: Chao-Heng Lee
>Priority: Minor
>
> The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
> Although the documentation mentions that the script may not work out of the
> box, it would be clearer to update any outdated paths.
> for example, from
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
> to
> {code:java}
> export
> LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
