errose28 commented on code in PR #7529: URL: https://github.com/apache/ozone/pull/7529#discussion_r1870390885
########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. Review Comment: ```suggestion By default, the client reads data over GRPC from the Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over a GRPC connection. ``` nit. There's a mix of Datanode and DataNode in this doc. We should standardize on Datanode. The new website strictly enforces this. ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. + +The paths that Java will search for libraries are specified by the "java.library.path" property. The default value of "java.library.path" depends on the OS and Java version. For example, on Linux with OpenJDK 8 it is `/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib`. + +Command "ozone checknative" can be used to detect whether libhadoop.so can be loaded successfully by Ozone service. + +## Configuration + +Short-circuit local reads need to be configured on both the DataNode and the client. By default, it is disabled. + +```XML +<property> + <name>ozone.client.read.short-circuit</name> + <value>false</value> + <description>Disable or enable the short-circuit local read feature.</description> +</property> +``` + +It makes use of a UNIX domain socket, a special path in the filesystem. You will need to set a path to this socket. + +```XML +<property> + <name>ozone.domain.socket.path</name> + <value>/var/lib/ozone_dn_socket</value> + <description>The path used to create domain socket.</description> +</property> +``` + +The DataNode needs to be able to create this path. On the other hand, it should not be possible for any user except the Ozone user(user who launches Ozone service) or root to create this path. For this reason, paths under /var/run or /var/lib are often used， just like the current default value "/var/lib/ozone_dn_socket". + +If you configure the "ozone.domain.socket.path" to other value, for example "/dir1/dir2/ozone_dn_socket", please make sure that both dir1 and dir2 are exiting directories, and there is no ozone_dn_socket under dir2. ozone_dn_socket will be created by Ozone Datanode later during Datanode start. + +### Security Consideration + +To ensure data security and integrity, Ozone will follow the rule as Hadoop ("https://cwiki.apache.org/confluence/display/HADOOP2/SocketPathSecurity";) on "ozone.domain.socket.path" path permission check. +It will fail the "ozone.domain.socket.path" verification and disable the feature if the filesystem permissions of the specified path are inadequate. +The verification failure message carries detail instruction about how to fix the problem. Following is an example, + +"The path component: '/etc/hadoop' in '/etc/hadoop/ozone_dn_socket' has permissions 0777 uid 0 and gid 0. It is not protected because it is world-writable. This might help: 'chmod o-w /etc/hadoop'. For more information: https://wiki.apache.org/hadoop/SocketPathSecurity"; Review Comment: ```suggestion The verification failure message carries instructions about how to fix the problem. Here is an example: \``` The path component: '/etc/hadoop' in '/etc/hadoop/ozone_dn_socket' has permissions 0777 uid 0 and gid 0. It is not protected because it is world-writable. This might help: 'chmod o-w /etc/hadoop'. For more information: https://wiki.apache.org/hadoop/SocketPathSecurity \``` ``` ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. + +The paths that Java will search for libraries are specified by the "java.library.path" property. The default value of "java.library.path" depends on the OS and Java version. For example, on Linux with OpenJDK 8 it is `/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib`. + +Command "ozone checknative" can be used to detect whether libhadoop.so can be loaded successfully by Ozone service. + +## Configuration + +Short-circuit local reads need to be configured on both the DataNode and the client. By default, it is disabled. + +```XML +<property> + <name>ozone.client.read.short-circuit</name> + <value>false</value> + <description>Disable or enable the short-circuit local read feature.</description> +</property> +``` + +It makes use of a UNIX domain socket, a special path in the filesystem. You will need to set a path to this socket. + +```XML +<property> + <name>ozone.domain.socket.path</name> + <value>/var/lib/ozone_dn_socket</value> + <description>The path used to create domain socket.</description> +</property> +``` + +The DataNode needs to be able to create this path. On the other hand, it should not be possible for any user except the Ozone user(user who launches Ozone service) or root to create this path. For this reason, paths under /var/run or /var/lib are often used， just like the current default value "/var/lib/ozone_dn_socket". + +If you configure the "ozone.domain.socket.path" to other value, for example "/dir1/dir2/ozone_dn_socket", please make sure that both dir1 and dir2 are exiting directories, and there is no ozone_dn_socket under dir2. ozone_dn_socket will be created by Ozone Datanode later during Datanode start. + +### Security Consideration + +To ensure data security and integrity, Ozone will follow the rule as Hadoop ("https://cwiki.apache.org/confluence/display/HADOOP2/SocketPathSecurity";) on "ozone.domain.socket.path" path permission check. Review Comment: ```suggestion To ensure data security and integrity, Ozone will follow the same rules as Hadoop to check permission on the `ozone.domain.socket.path` path as documented in [Socket Path Security](https://cwiki.apache.org/confluence/display/HADOOP2/SocketPathSecurity). ``` Is there a better resource for this somewhere on hadoop.apache.org/docs? Linking to confluence instead of official docs is not great, and this link is hardcoded to an old hadoop 2 url that will not get updated with subsequent versions. ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. + +The paths that Java will search for libraries are specified by the "java.library.path" property. The default value of "java.library.path" depends on the OS and Java version. For example, on Linux with OpenJDK 8 it is `/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib`. + +Command "ozone checknative" can be used to detect whether libhadoop.so can be loaded successfully by Ozone service. + +## Configuration + +Short-circuit local reads need to be configured on both the DataNode and the client. By default, it is disabled. Review Comment: It may seem obvious that we need to enable it to use it, but the docs don't actually show that here. Ideally copy/pasting the provided xml into a config would yield the desired result. ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. Review Comment: ```suggestion The Hadoop native library `libhadoop.so` provides support for Unix domain sockets. Please refer to Hadoop's [Native Libraries Guide](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html) for details. ``` Let's use markdown formatting as much as possible to help readability. There's other instances later in this page where we can replace "" with `` and use markdown links instead of plain ones. ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. + +The paths that Java will search for libraries are specified by the "java.library.path" property. The default value of "java.library.path" depends on the OS and Java version. For example, on Linux with OpenJDK 8 it is `/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib`. + +Command "ozone checknative" can be used to detect whether libhadoop.so can be loaded successfully by Ozone service. + +## Configuration + +Short-circuit local reads need to be configured on both the DataNode and the client. By default, it is disabled. + +```XML +<property> + <name>ozone.client.read.short-circuit</name> + <value>false</value> + <description>Disable or enable the short-circuit local read feature.</description> +</property> +``` + +It makes use of a UNIX domain socket, a special path in the filesystem. You will need to set a path to this socket. + +```XML +<property> + <name>ozone.domain.socket.path</name> + <value>/var/lib/ozone_dn_socket</value> + <description>The path used to create domain socket.</description> +</property> +``` + +The DataNode needs to be able to create this path. On the other hand, it should not be possible for any user except the Ozone user(user who launches Ozone service) or root to create this path. For this reason, paths under /var/run or /var/lib are often used， just like the current default value "/var/lib/ozone_dn_socket". + +If you configure the "ozone.domain.socket.path" to other value, for example "/dir1/dir2/ozone_dn_socket", please make sure that both dir1 and dir2 are exiting directories, and there is no ozone_dn_socket under dir2. ozone_dn_socket will be created by Ozone Datanode later during Datanode start. Review Comment: ```suggestion If you configure the "ozone.domain.socket.path" to another value, for example `/dir1/dir2/ozone_dn_socket`, please make sure that both `dir1` and `dir2` are existing directories, but the file `ozone_dn_socket` does not exist under `dir2`. `ozone_dn_socket` will be created during Datanode startup. ``` ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. + +The paths that Java will search for libraries are specified by the "java.library.path" property. The default value of "java.library.path" depends on the OS and Java version. For example, on Linux with OpenJDK 8 it is `/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib`. + +Command "ozone checknative" can be used to detect whether libhadoop.so can be loaded successfully by Ozone service. Review Comment: ```suggestion The `ozone checknative` command can be used to detect whether `libhadoop.so` can be loaded successfully by Ozone. ``` ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. Review Comment: I'm not sure what is meant by a "proper" libhadoop.so. These three mini paragraphs all address the same topic and can be grouped into one paragraph. Additionally we should have instructions for how to make java see the library before talking about loading it. ########## hadoop-hdds/docs/content/feature/Short-Circuit-Read.md: ########## @@ -0,0 +1,76 @@ +--- +title: "Short Circuit Local Read in Datanode" +weight: 2 +menu: + main: + parent: Features +summary: Introduction to Ozone Datanode Short Circuit Local Read Feature +--- +<!--- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Current in Ozone, client reads data over GRPC from Datanode. When the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over GRPC connection. + +This “short-circuit” local read feature will bypass the DataNode, allowing the client to read the file from local disk directly when the client is co-located with the data on the same server. + +Short-circuit local read can provide a substantial performance boost to many applications, by removing the overhead of network communication. + +## Prerequisite + +Short-circuit local reads make use of a UNIX domain socket. This is a special path in the filesystem that allows the client and the DataNodes to communicate. + +The Hadoop native library "libhadoop.so" provides the support to use the Unix domain socket. Please refer to Native Libraries ("https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/NativeLibraries.html";) for details of this library. + +Before enabling short-circuit local reads, you must have a proper libhadoop.so, and make sure it's under the directory where Java can find and load it through "System.loadLibrary()" call. + +The paths that Java will search for libraries are specified by the "java.library.path" property. The default value of "java.library.path" depends on the OS and Java version. For example, on Linux with OpenJDK 8 it is `/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib`. + +Command "ozone checknative" can be used to detect whether libhadoop.so can be loaded successfully by Ozone service. + +## Configuration + +Short-circuit local reads need to be configured on both the DataNode and the client. By default, it is disabled. + +```XML +<property> + <name>ozone.client.read.short-circuit</name> + <value>false</value> + <description>Disable or enable the short-circuit local read feature.</description> +</property> +``` + +It makes use of a UNIX domain socket, a special path in the filesystem. You will need to set a path to this socket. + +```XML +<property> + <name>ozone.domain.socket.path</name> + <value>/var/lib/ozone_dn_socket</value> + <description>The path used to create domain socket.</description> +</property> +``` + +The DataNode needs to be able to create this path. On the other hand, it should not be possible for any user except the Ozone user(user who launches Ozone service) or root to create this path. For this reason, paths under /var/run or /var/lib are often used， just like the current default value "/var/lib/ozone_dn_socket". + +If you configure the "ozone.domain.socket.path" to other value, for example "/dir1/dir2/ozone_dn_socket", please make sure that both dir1 and dir2 are exiting directories, and there is no ozone_dn_socket under dir2. ozone_dn_socket will be created by Ozone Datanode later during Datanode start. + +### Security Consideration + +To ensure data security and integrity, Ozone will follow the rule as Hadoop ("https://cwiki.apache.org/confluence/display/HADOOP2/SocketPathSecurity";) on "ozone.domain.socket.path" path permission check. Review Comment: Also, is our implementation using hadoop linraries to do these permission checks or did we re-implement them ourselves? If we are using hadoop then it makes since to link to hadoop docs so they update with the code. If we re-implemented them ourselves we should also "re-implement" our own documentation so they match now and in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
