Copilot commented on code in PR #4817: URL: https://github.com/apache/bookkeeper/pull/4817#discussion_r3504852966
########## site3/website/versioned_docs/version-4.18.0/overview/overview.md: ########## @@ -0,0 +1,62 @@ +--- +id: overview +title: Apache BookKeeper 4.18.0-SNAPSHOT +--- Review Comment: The page title still says "4.18.0-SNAPSHOT" even though this PR adds a released 4.18.0 doc version (versions.json + latest_release=4.18.0). This is confusing in the rendered docs and seems inconsistent with the PR goal of generating 4.18.0 website docs. ########## site3/website/versioned_docs/version-4.18.0/reference/config.md: ########## @@ -0,0 +1,367 @@ +--- +id: config +title: BookKeeper configuration +--- + +The table below lists parameters that you can set to configure bookies. All configuration takes place in the `bk_server.conf` file in the `bookkeeper-server/conf` directory of your [BookKeeper installation](../getting-started/installation). + + + +## Server parameters + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| bookiePort | The port that the bookie server listens on. | 3181 | +| allowMultipleDirsUnderSameDiskPartition | Configure the bookie to allow/disallow multiple ledger/index/journal directories in the same filesystem disk partition. | true | +| listeningInterface | The network interface that the bookie should listen on. If not set, the bookie will listen on all interfaces. | eth0 | +| advertisedAddress | Configure a specific hostname or IP address that the bookie should use to advertise itself to<br />clients. If not set, bookie will advertised its own IP address or hostname, depending on the<br />`listeningInterface` and `useHostNameAsBookieID` settings.<br /> | eth0 | +| allowLoopback | Whether the bookie is allowed to use a loopback interface as its primary<br />interface (the interface it uses to establish its identity). By default, loopback interfaces are *not* allowed as the primary interface.<br /><br />Using a loopback interface as the primary interface usually indicates a configuration error. It's fairly common in some VPS setups, for example, to not configure a hostname or to have the hostname resolve to 127.0.0.1. If this is the case, then all bookies in the cluster will establish their identities as 127.0.0.1:3181, and only one will be able to join the cluster. For VPSs configured like this, you should explicitly set the listening interface.<br /> | false | +| useHostNameAsBookieID | Whether the bookie should use its hostname to register with the ZooKeeper coordination service. When `false`, the bookie will use its IP address for the registration. | false | +| useShortHostName | Whether the bookie should use short hostname or [FQDN](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) hostname for registration and ledger metadata when `useHostNameAsBookieID` is enabled. | false | +| allowEphemeralPorts | Whether the bookie is allowed to use an ephemeral port (port 0) as its server port. By default, an ephemeral port is not allowed. Using an ephemeral port as the service port usually indicates a configuration error. However, in unit tests, using an ephemeral port will address port conflict problems and allow running tests in parallel. | false | +| enableLocalTransport | Whether allow the bookie to listen for BookKeeper clients executed on the local JVM. | false | +| disableServerSocketBind | Whether allow the bookie to disable bind on network interfaces, this bookie will be available only to BookKeeper clients executed on the local JVM. | false | +| bookieDeathWatchInterval | Interval to watch whether bookie is dead or not, in milliseconds. | 1000 | +| extraServerComponents | Configure a list of extra server components to enable and load on a bookie server. This provides a plugin mechanism to run extra server components along with a bookie server. | | +| ignoreExtraServerComponentsStartupFailures | Whether the bookie should ignore startup failures on loading server components specified by `extraServerComponents`. | false | + + +## Worker thread settings + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| numAddWorkerThreads | The number of threads that handle write requests. if zero, writes are handled by [Netty threads](//netty.io/wiki/thread-model.html) directly. | 1 | +| numReadWorkerThreads | The number of threads that handle read requests. If zero, reads are handled by [Netty threads](//netty.io/wiki/thread-model.html) directly. | 8 | +| numLongPollWorkerThreads | The number of threads that handle long poll requests. If zero, long poll requests are handled by [Netty threads](//netty.io/wiki/thread-model.html) directly. | | +| numJournalCallbackThreads | The number of threads that handle journal callbacks. If zero, journal callbacks are executed directly on force write threads. | 1 | +| numHighPriorityWorkerThreads | The number of threads that should be used for high priority requests (i.e. recovery reads and adds, and fencing). If zero, reads are handled by [Netty threads](//netty.io/wiki/thread-model.html) directly. | 8 | +| maxPendingAddRequestsPerThread | If read worker threads are enabled, limit the number of pending requests, to avoid the executor queue to grow indefinitely. If zero or negative, the number of pending requests is unlimited. | 10000 | +| maxPendingReadRequestsPerThread | If add worker threads are enabled, limit the number of pending requests, to avoid the executor queue to grow indefinitely. If zero or negative, the number of pending requests is unlimited. | 10000 | +| enableBusyWait | Option to enable busy-wait settings. Default is false.<br />WARNING: This option will enable spin-waiting on executors and IO threads in order to reduce latency during<br />context switches. The spinning will consume 100% CPU even when bookie is not doing any work. It is recommended to<br />reduce the number of threads in the main workers pool and Netty event loop to only have few CPU cores busy.<br /> | | + + +## Long poll settings + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| requestTimerTickDurationMs | The tick duration for long poll request timer, in milliseconds. See [HashedWheelTimer](//netty.io/4.1/api/io/netty/util/HashedWheelTimer.html) for more details. | 10 | +| requestTimerNumTicks | The number of ticks per wheel for long poll request timer. See [HashedWheelTimer](//netty.io/4.1/api/io/netty/util/HashedWheelTimer.html) for more details. | 1024 | + + +## Read-only mode support + +| Parameter | Description | Default +| -- | ----------- | ------- | +| readOnlyModeEnabled | If all ledger directories configured are full, then support only read requests for clients. If "readOnlyModeEnabled=true" then on all ledger disks full, bookie will be converted to read-only mode and serve only read requests. Otherwise the bookie will be shutdown. By default, this will be enabled. | true | +| forceReadOnlyBookie | Whether the bookie is force started in read only mode or not. | false | +| persistBookieStatusEnabled | Persist the bookie status locally on the disks. So the bookies can keep their status upon restarts. | false | +| readOnlyModeOnAnyDiskFullEnabled | If any ledger directories configured are full, then support only read requests for clients. If "readOnlyModeOnAnyDiskFullEnabled=true" then on any ledger disks full, bookie will be converted to read-only mode and serve only read requests. When all disks recovered, the bookie will be converted to read-write mode.Otherwise it will obey the `readOnlyModeEnabled` behavior. By default, this will be disabled. | false | + +## Netty server settings + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| serverTcpNoDelay | This settings is used to enabled/disabled Nagle's algorithm, which is a means of improving the efficiency of TCP/IP networks by reducing the number of packets that need to be sent over the network.<br /><br />If you are sending many small messages, such that more than one can fit in a single IP packet, setting server.tcpnodelay to false to enable Nagle algorithm can provide better performance.<br /> | true | +| serverSockKeepalive | This setting is used to send keep-alive messages on connection-oriented sockets. | true | +| serverTcpLinger | The socket linger timeout on close. When enabled, a close or shutdown will not return until all queued messages for the socket have been successfully sent or the linger timeout has been reached. Otherwise, the call returns immediately and the closing is done in the background. | | +| byteBufAllocatorSizeInitial | The Recv ByteBuf allocator initial buf size. | 65536 | +| byteBufAllocatorSizeMin | The Recv ByteBuf allocator min buf size. | 65536 | +| byteBufAllocatorSizeMax | The Recv ByteBuf allocator max buf size. | 1048576 | +| nettyMaxFrameSizeBytes | The maximum netty frame size in bytes. Any message received larger than this will be rejected, so when the client-side attempt to send more than the default size bytes, it should set up the corresponding parameter `setNettyMaxFrameSizeBytes(int maxSize)`, pay attention to the parameter should be less than the value of server-side. | 5242880 | + + +## Http server settings + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| httpServerEnabled | The flag enables/disables starting the admin http server. | false | +| httpServerPort | The http server port to listen on if `httpServerEnabled` is set to true. | 8080 | +| httpServerHost | The http server host to listen on if `httpServerEnabled` is set to true. | 0.0.0.0 | + + +## Security settings + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| bookieAuthProviderFactoryClass | The bookie authentication provider factory class name. If this is null, no authentication will take place. | | +| permittedStartupUsers | The list of users are permitted to run the bookie process. Any users can run the bookie process if it is not set.<br /><br />Example settings - "permittedStartupUsers=user1,user2,user3"<br /> | | + + +## TLS settings + +| Parameter | Description | Default +| --------- | ----------- | ------- | +| tlsProvider | TLS Provider (JDK or OpenSSL) | OpenSSL | +| tlsProviderFactoryClass | The path to the class that provides security. | org.apache.bookkeeper.tls.TLSContextFactory | +| tlsClientAuthentication | Type of security used by server. | true | +| tlsKeyStoreType | Bookie Keystore type. | JKS | +| tlsKeyStore | Bookie Keystore location (path). | | +| tlsKeyStore | Bookie Keystore location (path). | | +| tlsKeyStorePasswordPath | Bookie Keystore password path, if the keystore is protected by a password. | | Review Comment: The TLS settings table lists `tlsKeyStore` twice with the same description. This duplication makes the config reference harder to read and can mislead readers into thinking there are two separate settings. ########## site3/website/versioned_docs/version-4.18.0/security/zookeeper.md: ########## @@ -0,0 +1,41 @@ +--- +id: zookeeper +title: ZooKeeper Authentication +--- + +## New Clusters + +To enable `ZooKeeper` authentication on Bookies or Clients, there are two necessary steps: + +1. Create a `JAAS` login file and set the appropriate system property to point to it as described in [GSSAPI (Kerberos)](sasl#notes). +2. Set the configuration property `zkEnableSecurity` in each bookie to `true`. + +The metadata stored in `ZooKeeper` is such that only certain clients will be able to modify and read the corresponding znodes. +The rationale behind this decision is that the data stored in ZooKeeper is not sensitive, but inappropriate manipulation of znodes can cause cluster +disruption. + +## Migrating Clusters + +If you are running a version of BookKeeper that does not support security or simply with security disabled, and you want to make the cluster secure, +then you need to execute the following steps to enable ZooKeeper authentication with minimal disruption to your operations. + +1. Perform a rolling restart setting the `JAAS` login file, which enables bookie or clients to authenticate. At the end of the rolling restart, + bookies (or clients) are able to manipulate znodes with strict ACLs, but they will not create znodes with those ACLs. +2. Perform a second rolling restart of bookies, this time setting the configuration parameter `zkEnableSecurity` to true, which enables the use + of secure ACLs when creating znodes. +3. Currently we don't have provide a tool to set acls on old znodes. You are recommended to set it manually using ZooKeeper tools. + +It is also possible to turn off authentication in a secured cluster. To do it, follow these steps: + +1. Perform a rolling restart of bookies setting the `JAAS` login file, which enable bookies to authenticate, but setting `zkEnableSecurity` to `false`. Review Comment: Grammar: "which enable bookies" should be "which enables bookies" (subject/verb agreement). ########## site3/website/versioned_docs/version-4.18.0/deployment/manual.md: ########## @@ -0,0 +1,65 @@ +--- +id: manual +title: Manual deployment +--- + +A BookKeeper cluster consists of two main components: + +* A [ZooKeeper](#zookeeper-setup) cluster that is used for configuration- and coordination-related tasks +* An [ensemble](#starting-up-bookies) of bookies + +## ZooKeeper setup + +We won't provide a full guide to setting up a ZooKeeper cluster here. We recommend that you consult [this guide](https://zookeeper.apache.org/doc/current/zookeeperAdmin.html) in the official ZooKeeper documentation. + +## Cluster metadata setup + +Once your ZooKeeper cluster is up and running, there is some metadata that needs to be written to ZooKeeper, so you need to modify the bookie's configuration to make sure that it points to the right ZooKeeper cluster. + +On each bookie host, you need to [download](../getting-started/installation#download) the BookKeeper package as a tarball. Once you've done that, you need to configure the bookie by setting values in the `bookkeeper-server/conf/bk_server.conf` config file. The one parameter that you will absolutely need to change is the `metadataServiceUri` parameter, which you will need to set to the ZooKeeper connection string for your ZooKeeper cluster. Here's an example: + +```properties +metadataServiceUri=zk+hierarchical://100.0.0.1:2181;100.0.0.2:2181;100.0.0.3:2181/ledgers +``` + +> A full listing of configurable parameters available in `bookkeeper-server/conf/bk_server.conf` can be found in the [Configuration](../reference/config) reference manual. + +Once the bookie's configuration is set, you can set up cluster metadata for the cluster by running the following command from any bookie in the cluster: + +```shell +$ bookkeeper-server/bin/bookkeeper shell metaformat +``` + +You can run in the formatting Review Comment: This sentence is incomplete ("You can run in the formatting"). It reads like a truncated thought and breaks the flow of the deployment instructions. ########## site3/website/versioned_sidebars/version-4.18.0-sidebars.json: ########## @@ -0,0 +1,162 @@ +{ + "docsSidebar": [ + { + "type": "doc", + "id": "version-4.18.0/overview/overview", + "label": "Overview" + }, + { + + "type": "category", + "label": "Getting started", + "items": [ Review Comment: There is a blank line containing only whitespace inside the JSON array. This is harmless at runtime but tends to cause noisy diffs and may fail strict formatting/linting checks. ########## site3/website/versioned_docs/version-4.18.0/security/zookeeper.md: ########## @@ -0,0 +1,41 @@ +--- +id: zookeeper +title: ZooKeeper Authentication +--- + +## New Clusters + +To enable `ZooKeeper` authentication on Bookies or Clients, there are two necessary steps: + +1. Create a `JAAS` login file and set the appropriate system property to point to it as described in [GSSAPI (Kerberos)](sasl#notes). +2. Set the configuration property `zkEnableSecurity` in each bookie to `true`. + +The metadata stored in `ZooKeeper` is such that only certain clients will be able to modify and read the corresponding znodes. +The rationale behind this decision is that the data stored in ZooKeeper is not sensitive, but inappropriate manipulation of znodes can cause cluster +disruption. + +## Migrating Clusters + +If you are running a version of BookKeeper that does not support security or simply with security disabled, and you want to make the cluster secure, +then you need to execute the following steps to enable ZooKeeper authentication with minimal disruption to your operations. + +1. Perform a rolling restart setting the `JAAS` login file, which enables bookie or clients to authenticate. At the end of the rolling restart, + bookies (or clients) are able to manipulate znodes with strict ACLs, but they will not create znodes with those ACLs. +2. Perform a second rolling restart of bookies, this time setting the configuration parameter `zkEnableSecurity` to true, which enables the use + of secure ACLs when creating znodes. +3. Currently we don't have provide a tool to set acls on old znodes. You are recommended to set it manually using ZooKeeper tools. Review Comment: Grammar/terminology: "don't have provide" is ungrammatical, and "acls" should be capitalized as ACLs. This section is user-facing documentation, so it should read cleanly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
