This is an automated email from the ASF dual-hosted git repository.
martijnvisser pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-connector-pulsar.git
The following commit(s) were added to refs/heads/main by this push:
new 6205470 [hotfix] Manually copy over generated layout shortcodes
6205470 is described below
commit 6205470178729167a2d6799095e0367b6cea6c74
Author: Martijn Visser <[email protected]>
AuthorDate: Wed Dec 14 15:31:47 2022 +0100
[hotfix] Manually copy over generated layout shortcodes
---
.../generated/pulsar_admin_configuration.html | 42 ++++
.../generated/pulsar_client_configuration.html | 222 +++++++++++++++++++++
.../generated/pulsar_consumer_configuration.html | 162 +++++++++++++++
.../generated/pulsar_producer_configuration.html | 78 ++++++++
.../generated/pulsar_sink_configuration.html | 54 +++++
.../generated/pulsar_source_configuration.html | 66 ++++++
6 files changed, 624 insertions(+)
diff --git a/docs/layouts/shortcodes/generated/pulsar_admin_configuration.html
b/docs/layouts/shortcodes/generated/pulsar_admin_configuration.html
new file mode 100644
index 0000000..31c7273
--- /dev/null
+++ b/docs/layouts/shortcodes/generated/pulsar_admin_configuration.html
@@ -0,0 +1,42 @@
+<table class="configuration table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Key</th>
+ <th class="text-left" style="width: 15%">Default</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ <th class="text-left" style="width: 55%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><h5>pulsar.admin.adminUrl</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>The Pulsar service HTTP URL for the admin endpoint. For
example, <code
class="highlighter-rouge">http://my-broker.example.com:8080</code>, or <code
class="highlighter-rouge">https://my-broker.example.com:8443</code> for
TLS.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.admin.autoCertRefreshTime</h5></td>
+ <td style="word-wrap: break-word;">300000</td>
+ <td>Integer</td>
+ <td>The auto cert refresh time (in ms) if Pulsar admin supports
TLS authentication.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.admin.connectTimeout</h5></td>
+ <td style="word-wrap: break-word;">60000</td>
+ <td>Integer</td>
+ <td>The connection time out (in ms) for the PulsarAdmin
client.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.admin.readTimeout</h5></td>
+ <td style="word-wrap: break-word;">60000</td>
+ <td>Integer</td>
+ <td>The server response read timeout (in ms) for the PulsarAdmin
client for any request.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.admin.requestTimeout</h5></td>
+ <td style="word-wrap: break-word;">300000</td>
+ <td>Integer</td>
+ <td>The server request timeout (in ms) for the PulsarAdmin client
for any request.</td>
+ </tr>
+ </tbody>
+</table>
diff --git a/docs/layouts/shortcodes/generated/pulsar_client_configuration.html
b/docs/layouts/shortcodes/generated/pulsar_client_configuration.html
new file mode 100644
index 0000000..970b847
--- /dev/null
+++ b/docs/layouts/shortcodes/generated/pulsar_client_configuration.html
@@ -0,0 +1,222 @@
+<table class="configuration table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Key</th>
+ <th class="text-left" style="width: 15%">Default</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ <th class="text-left" style="width: 55%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><h5>pulsar.client.authParamMap</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>Map</td>
+ <td>Parameters for the authentication plugin.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.authParams</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Parameters for the authentication plugin.<br /><br
/>Example:<br /><code class="highlighter-rouge">key1:val1,key2:val2</code></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.authPluginClassName</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Name of the authentication plugin.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.concurrentLookupRequest</h5></td>
+ <td style="word-wrap: break-word;">5000</td>
+ <td>Integer</td>
+ <td>The number of concurrent lookup requests allowed to send on
each broker connection to prevent overload on the broker. It should be
configured with a higher value only in case of it requires to produce or
subscribe on thousands of topic using a created <code
class="highlighter-rouge">PulsarClient</code></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.connectionTimeoutMs</h5></td>
+ <td style="word-wrap: break-word;">10000</td>
+ <td>Integer</td>
+ <td>Duration (in ms) of waiting for a connection to a broker to be
established.<br />If the duration passes without a response from a broker, the
connection attempt is dropped.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.connectionsPerBroker</h5></td>
+ <td style="word-wrap: break-word;">1</td>
+ <td>Integer</td>
+ <td>The maximum number of connections that the client library will
open to a single broker.<br /> By default, the connection pool will use a
single connection for all the producers and consumers. Increasing this
parameter may improve throughput when using many producers over a high latency
connection.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.enableBusyWait</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Option to enable busy-wait settings.<br />This option will
enable spin-waiting on executors and IO threads in order to reduce latency
during context switches. The spinning will consume 100% CPU even when the
broker is not doing any work. It is recommended to reduce the number of IO
threads and BookKeeper client threads to only have fewer CPU cores busy.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.enableTransaction</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If transaction is enabled, start the <code
class="highlighter-rouge">transactionCoordinatorClient</code> with <code
class="highlighter-rouge">PulsarClient</code>.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.initialBackoffIntervalNanos</h5></td>
+ <td style="word-wrap: break-word;">100000000</td>
+ <td>Long</td>
+ <td>Default duration (in nanoseconds) for a backoff interval.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.keepAliveIntervalSeconds</h5></td>
+ <td style="word-wrap: break-word;">30</td>
+ <td>Integer</td>
+ <td>Interval (in seconds) for keeping connection between the
Pulsar client and broker alive.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.listenerName</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Configure the <code
class="highlighter-rouge">listenerName</code> that the broker will return the
corresponding <code class="highlighter-rouge">advertisedListener</code>.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.maxBackoffIntervalNanos</h5></td>
+ <td style="word-wrap: break-word;">60000000000</td>
+ <td>Long</td>
+ <td>The maximum duration (in nanoseconds) for a backoff
interval.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.maxLookupRedirects</h5></td>
+ <td style="word-wrap: break-word;">20</td>
+ <td>Integer</td>
+ <td>The maximum number of times a lookup-request redirections to a
broker.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.maxLookupRequest</h5></td>
+ <td style="word-wrap: break-word;">50000</td>
+ <td>Integer</td>
+ <td>The maximum number of lookup requests allowed on each broker
connection to prevent overload on the broker. It should be greater than <code
class="highlighter-rouge">pulsar.client.concurrentLookupRequest</code>.
Requests that inside <code
class="highlighter-rouge">pulsar.client.concurrentLookupRequest</code> are
already sent to broker, and requests beyond <code
class="highlighter-rouge">pulsar.client.concurrentLookupRequest</code> and
under <code class="highlighter-rouge"> [...]
+ </tr>
+ <tr>
+
<td><h5>pulsar.client.maxNumberOfRejectedRequestPerConnection</h5></td>
+ <td style="word-wrap: break-word;">50</td>
+ <td>Integer</td>
+ <td>The maximum number of rejected requests of a broker in a
certain period (30s) after the current connection is closed and the client
creates a new connection to connect to a different broker.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.memoryLimitBytes</h5></td>
+ <td style="word-wrap: break-word;">67108864</td>
+ <td>Long</td>
+ <td>The limit (in bytes) on the amount of direct memory that will
be allocated by this client instance.<br />Note: at this moment this is only
limiting the memory for producers. Setting this to <code
class="highlighter-rouge">0</code> will disable the limit.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.numIoThreads</h5></td>
+ <td style="word-wrap: break-word;">1</td>
+ <td>Integer</td>
+ <td>The number of threads used for handling connections to
brokers.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.numListenerThreads</h5></td>
+ <td style="word-wrap: break-word;">1</td>
+ <td>Integer</td>
+ <td>The number of threads used for handling message listeners. The
listener thread pool is shared across all the consumers and readers that are
using a <code class="highlighter-rouge">listener</code> model to get messages.
For a given consumer, the listener is always invoked from the same thread to
ensure ordering.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.operationTimeoutMs</h5></td>
+ <td style="word-wrap: break-word;">30000</td>
+ <td>Integer</td>
+ <td>Operation timeout (in ms). Operations such as creating
producers, subscribing or unsubscribing topics are retried during this
interval. If the operation is not completed during this interval, the operation
will be marked as failed.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.proxyProtocol</h5></td>
+ <td style="word-wrap: break-word;">SNI</td>
+ <td><p>Enum</p></td>
+ <td>Protocol type to determine the type of proxy routing when a
client connects to the proxy using <code
class="highlighter-rouge">pulsar.client.proxyServiceUrl</code>.<br /><br
/>Possible values:<ul><li>"SNI"</li></ul></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.proxyServiceUrl</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Proxy-service URL when a client connects to the broker via the
proxy. The client can choose the type of proxy-routing.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.requestTimeoutMs</h5></td>
+ <td style="word-wrap: break-word;">60000</td>
+ <td>Integer</td>
+ <td>Maximum duration (in ms) for completing a request. This config
option is not supported before Pulsar 2.8.1</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.serviceUrl</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Service URL provider for Pulsar service.<br />To connect to
Pulsar using client libraries, you need to specify a Pulsar protocol URL.<br
/>You can assign Pulsar protocol URLs to specific clusters and use the Pulsar
scheme.<br /><ul><li>This is an example of <code
class="highlighter-rouge">localhost</code>: <code
class="highlighter-rouge">pulsar://localhost:6650</code>.</li><li>If you have
multiple brokers, the URL is as: <code
class="highlighter-rouge">pulsar://localhost: [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.sslProvider</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>The name of the security provider used for SSL connections.
The default value is the default security provider of the JVM.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.statsIntervalSeconds</h5></td>
+ <td style="word-wrap: break-word;">60</td>
+ <td>Long</td>
+ <td>Interval between each stats info.<br /><ul><li>Stats is
activated with positive <code
class="highlighter-rouge">statsInterval</code></li><li>Set <code
class="highlighter-rouge">statsIntervalSeconds</code> to 1 second at
least.</li></ul></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsAllowInsecureConnection</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Whether the Pulsar client accepts untrusted TLS certificate
from the broker.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsCiphers</h5></td>
+ <td style="word-wrap: break-word;"></td>
+ <td>List<String></td>
+ <td>A list of cipher suites. This is a named combination of
authentication, encryption, MAC and the key exchange algorithm used to
negotiate the security settings for a network connection using the TLS or SSL
network protocol. By default all the available cipher suites are supported.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsHostnameVerificationEnable</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Whether to enable TLS hostname verification. It allows to
validate hostname verification when a client connects to the broker over TLS.
It validates incoming x509 certificate and matches provided hostname (CN/SAN)
with the expected broker's host name. It follows RFC 2818, 3.1. Server Identity
hostname verification.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsProtocols</h5></td>
+ <td style="word-wrap: break-word;"></td>
+ <td>List<String></td>
+ <td>The SSL protocol used to generate the SSLContext. By default,
it is set TLS, which is fine for most cases. Allowed values in recent JVMs are
TLS, TLSv1.3, TLSv1.2 and TLSv1.1.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsTrustCertsFilePath</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Path to the trusted TLS certificate file.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsTrustStorePassword</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>The store password for the key store file.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsTrustStorePath</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>The location of the trust store file.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.tlsTrustStoreType</h5></td>
+ <td style="word-wrap: break-word;">"JKS"</td>
+ <td>String</td>
+ <td>The file format of the trust store file.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.useKeyStoreTls</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If TLS is enabled, whether use the KeyStore type as the TLS
configuration parameter. If it is set to <code
class="highlighter-rouge">false</code>, it means to use the default pem type
configuration.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.client.useTcpNoDelay</h5></td>
+ <td style="word-wrap: break-word;">true</td>
+ <td>Boolean</td>
+ <td>Whether to use the TCP no-delay flag on the connection to
disable Nagle algorithm.<br />No-delay features ensures that packets are sent
out on the network as soon as possible, and it is critical to achieve low
latency publishes. On the other hand, sending out a huge number of small
packets might limit the overall throughput. Therefore, if latency is not a
concern, it is recommended to set this option to <code
class="highlighter-rouge">false</code>.<br />By default, it is [...]
+ </tr>
+ </tbody>
+</table>
diff --git
a/docs/layouts/shortcodes/generated/pulsar_consumer_configuration.html
b/docs/layouts/shortcodes/generated/pulsar_consumer_configuration.html
new file mode 100644
index 0000000..41f0b24
--- /dev/null
+++ b/docs/layouts/shortcodes/generated/pulsar_consumer_configuration.html
@@ -0,0 +1,162 @@
+<table class="configuration table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Key</th>
+ <th class="text-left" style="width: 15%">Default</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ <th class="text-left" style="width: 55%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><h5>pulsar.consumer.ackReceiptEnabled</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Acknowledgement will return a receipt but this does not mean
that the message will not be resent after getting the receipt.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.ackTimeoutMillis</h5></td>
+ <td style="word-wrap: break-word;">0</td>
+ <td>Long</td>
+ <td>The timeout (in ms) for unacknowledged messages, truncated to
the nearest millisecond. The timeout needs to be greater than 1 second.<br />By
default, the acknowledge timeout is disabled and that means that messages
delivered to a consumer will not be re-delivered unless the consumer
crashes.<br />When acknowledgement timeout being enabled, if a message is not
acknowledged within the specified timeout it will be re-delivered to the
consumer (possibly to a different consum [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.acknowledgementsGroupTimeMicros</h5></td>
+ <td style="word-wrap: break-word;">100000</td>
+ <td>Long</td>
+ <td>Group a consumer acknowledgment for a specified time (in μs).
By default, a consumer uses <code class="highlighter-rouge">100μs</code>
grouping time to send out acknowledgments to a broker. If the group time is set
to <code class="highlighter-rouge">0</code>, acknowledgments are sent out
immediately. A longer ack group time is more efficient at the expense of a
slight increase in message re-deliveries after a failure.</td>
+ </tr>
+ <tr>
+
<td><h5>pulsar.consumer.autoAckOldestChunkedMessageOnQueueFull</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Buffering a large number of outstanding uncompleted chunked
messages can bring memory pressure and it can be guarded by providing this
<code class="highlighter-rouge">pulsar.consumer.maxPendingChunkedMessage</code>
threshold. Once a consumer reaches this threshold, it drops the outstanding
unchunked-messages by silently acknowledging if <code
class="highlighter-rouge">pulsar.consumer.autoAckOldestChunkedMessageOnQueueFull</code>
is true. Otherwise, it marks them for redel [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.consumerName</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>The consumer name is informative and it can be used to
identify a particular consumer instance from the topic stats.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.cryptoFailureAction</h5></td>
+ <td style="word-wrap: break-word;">FAIL</td>
+ <td><p>Enum</p></td>
+ <td>The consumer should take action when it receives a message
that can not be decrypted.<ul><li><code class="highlighter-rouge">FAIL</code>:
this is the default option to fail messages until crypto
succeeds.</li><li><code class="highlighter-rouge">DISCARD</code>: silently
acknowledge but do not deliver messages to an application.</li><li><code
class="highlighter-rouge">CONSUME</code>: deliver encrypted messages to
applications. It is the application's responsibility to decry [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.deadLetterPolicy.deadLetterTopic</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Name of the dead topic where the failed messages are sent.</td>
+ </tr>
+ <tr>
+
<td><h5>pulsar.consumer.deadLetterPolicy.maxRedeliverCount</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>Integer</td>
+ <td>The maximum number of times that a message are redelivered
before being sent to the dead letter queue.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.deadLetterPolicy.retryLetterTopic</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Name of the retry topic where the failed messages are
sent.</td>
+ </tr>
+ <tr>
+
<td><h5>pulsar.consumer.expireTimeOfIncompleteChunkedMessageMillis</h5></td>
+ <td style="word-wrap: break-word;">60000</td>
+ <td>Long</td>
+ <td>If a producer fails to publish all the chunks of a message,
the consumer can expire incomplete chunks if the consumer cannot receive all
chunks in expire times (default 1 hour, in ms).</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.maxPendingChunkedMessage</h5></td>
+ <td style="word-wrap: break-word;">10</td>
+ <td>Integer</td>
+ <td>The consumer buffers chunk messages into memory until it
receives all the chunks of the original message. While consuming
chunk-messages, chunks from the same message might not be contiguous in the
stream and they might be mixed with other messages' chunks. So, consumer has to
maintain multiple buffers to manage chunks coming from different messages. This
mainly happens when multiple publishers are publishing messages on the topic
concurrently or publishers failed to publ [...]
+ </tr>
+ <tr>
+
<td><h5>pulsar.consumer.maxTotalReceiverQueueSizeAcrossPartitions</h5></td>
+ <td style="word-wrap: break-word;">50000</td>
+ <td>Integer</td>
+ <td>The maximum total receiver queue size across partitions.<br
/>This setting reduces the receiver queue size for individual partitions if the
total receiver queue size exceeds this value.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.negativeAckRedeliveryDelayMicros</h5></td>
+ <td style="word-wrap: break-word;">60000000</td>
+ <td>Long</td>
+ <td>Delay (in μs) to wait before redelivering messages that failed
to be processed.<br />When an application uses <code
class="highlighter-rouge">Consumer.negativeAcknowledge(Message)</code>, failed
messages are redelivered after a fixed timeout.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.poolMessages</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Enable pooling of messages and the underlying data
buffers.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.priorityLevel</h5></td>
+ <td style="word-wrap: break-word;">0</td>
+ <td>Integer</td>
+ <td>Priority level for a consumer to which a broker gives more
priorities while dispatching messages in the shared subscription type.<br />The
broker follows descending priorities. For example, 0=max-priority, 1, 2,...<br
/>In shared subscription mode, the broker first dispatches messages to the
consumers on the highest priority level if they have permits. Otherwise, the
broker considers consumers on the next priority level.<br /><br />Example 1<br
/>If a subscription has con [...]
+C1, 0, 2
+C2, 0, 1
+C3, 0, 1
+C4, 1, 2
+C5, 1, 1
+<br />The order in which a broker dispatches messages to consumers is: C1, C2,
C3, C1, C4, C5, C4.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.properties</h5></td>
+ <td style="word-wrap: break-word;"></td>
+ <td>Map</td>
+ <td>A name or value property of this consumer. <code
class="highlighter-rouge">properties</code> is application defined metadata
attached to a consumer. When getting a topic stats, associate this metadata
with the consumer stats for easier identification.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.readCompacted</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If enabling <code
class="highlighter-rouge">readCompacted</code>, a consumer reads messages from
a compacted topic rather than reading a full message backlog of a topic.<br />A
consumer only sees the latest value for each key in the compacted topic, up
until reaching the point in the topic message when compacting backlog. Beyond
that point, send messages as normal.<br />Only enabling <code
class="highlighter-rouge">readCompacted</code> on subscriptions to persistent
topic [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.receiverQueueSize</h5></td>
+ <td style="word-wrap: break-word;">1000</td>
+ <td>Integer</td>
+ <td>Size of a consumer's receiver queue.<br />For example, the
number of messages accumulated by a consumer before an application calls <code
class="highlighter-rouge">Receive</code>.<br />A value higher than the default
value increases consumer throughput, though at the expense of more memory
utilization.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.replicateSubscriptionState</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If <code
class="highlighter-rouge">replicateSubscriptionState</code> is enabled, a
subscription state is replicated to geo-replicated clusters.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.retryEnable</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If enabled, the consumer will automatically retry
messages.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.subscriptionMode</h5></td>
+ <td style="word-wrap: break-word;">Durable</td>
+ <td><p>Enum</p></td>
+ <td>Select the subscription mode to be used when subscribing to
the topic.<ul><li><code class="highlighter-rouge">Durable</code>: Make the
subscription to be backed by a durable cursor that will retain messages and
persist the current position.</li><li><code
class="highlighter-rouge">NonDurable</code>: Lightweight subscription mode that
doesn't have a durable cursor associated</li></ul><br /><br />Possible
values:<ul><li>"Durable"</li><li>"NonDurable"</li></ul></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.subscriptionName</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>Specify the subscription name for this consumer. This argument
is required when constructing the consumer.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.subscriptionType</h5></td>
+ <td style="word-wrap: break-word;">Shared</td>
+ <td><p>Enum</p></td>
+ <td>Subscription type.<br /><br />Four subscription types are
available:<ul><li>Exclusive</li><li>Failover</li><li>Shared</li><li>Key_Shared</li></ul><br
/><br />Possible
values:<ul><li>"Exclusive"</li><li>"Shared"</li><li>"Failover"</li><li>"Key_Shared"</li></ul></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.consumer.tickDurationMillis</h5></td>
+ <td style="word-wrap: break-word;">1000</td>
+ <td>Long</td>
+ <td>Granularity (in ms) of the ack-timeout redelivery.<br />A
greater (for example, 1 hour) <code
class="highlighter-rouge">tickDurationMillis</code> reduces the memory overhead
to track messages.</td>
+ </tr>
+ </tbody>
+</table>
diff --git
a/docs/layouts/shortcodes/generated/pulsar_producer_configuration.html
b/docs/layouts/shortcodes/generated/pulsar_producer_configuration.html
new file mode 100644
index 0000000..7984903
--- /dev/null
+++ b/docs/layouts/shortcodes/generated/pulsar_producer_configuration.html
@@ -0,0 +1,78 @@
+<table class="configuration table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Key</th>
+ <th class="text-left" style="width: 15%">Default</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ <th class="text-left" style="width: 55%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><h5>pulsar.producer.batchingEnabled</h5></td>
+ <td style="word-wrap: break-word;">true</td>
+ <td>Boolean</td>
+ <td>Enable batch send ability, it was enabled by default.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.batchingMaxBytes</h5></td>
+ <td style="word-wrap: break-word;">131072</td>
+ <td>Integer</td>
+ <td>The maximum size of messages permitted in a batch. Keep the
maximum consistent as previous versions.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.batchingMaxMessages</h5></td>
+ <td style="word-wrap: break-word;">1000</td>
+ <td>Integer</td>
+ <td>The maximum number of messages permitted in a batch.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.batchingMaxPublishDelayMicros</h5></td>
+ <td style="word-wrap: break-word;">1000</td>
+ <td>Long</td>
+ <td>Batching time period of sending messages.</td>
+ </tr>
+ <tr>
+
<td><h5>pulsar.producer.batchingPartitionSwitchFrequencyByPublishDelay</h5></td>
+ <td style="word-wrap: break-word;">10</td>
+ <td>Integer</td>
+ <td>The maximum wait time for switching topic partitions.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.chunkingEnabled</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.compressionType</h5></td>
+ <td style="word-wrap: break-word;">NONE</td>
+ <td><p>Enum</p></td>
+ <td>Message data compression type used by a producer.Available
options:<ul><li><a href="LZ4">https://github.com/lz4/lz4</a></li><li><a
href="ZLIB">https://zlib.net/</a></li><li><a
href="ZSTD">https://facebook.github.io/zstd/</a></li><li><a
href="SNAPPY">https://google.github.io/snappy/</a></li></ul><br /><br
/>Possible
values:<ul><li>"NONE"</li><li>"LZ4"</li><li>"ZLIB"</li><li>"ZSTD"</li><li>"SNAPPY"</li></ul></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.initialSequenceId</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>Long</td>
+ <td>The sequence id for avoiding the duplication, it's used when
Pulsar doesn't have transaction.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.producerName</h5></td>
+ <td style="word-wrap: break-word;">(none)</td>
+ <td>String</td>
+ <td>A producer name which would be displayed in the Pulsar's
dashboard. If no producer name was provided, we would use a Pulsar generated
name instead.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.properties</h5></td>
+ <td style="word-wrap: break-word;"></td>
+ <td>Map</td>
+ <td>A name or value property of this consumer. <code
class="highlighter-rouge">properties</code> is application defined metadata
attached to a consumer. When getting a topic stats, associate this metadata
with the consumer stats for easier identification.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.producer.sendTimeoutMs</h5></td>
+ <td style="word-wrap: break-word;">30000</td>
+ <td>Long</td>
+ <td>Message send timeout in ms.If a message is not acknowledged by
a server before the <code class="highlighter-rouge">sendTimeout</code> expires,
an error occurs.</td>
+ </tr>
+ </tbody>
+</table>
diff --git a/docs/layouts/shortcodes/generated/pulsar_sink_configuration.html
b/docs/layouts/shortcodes/generated/pulsar_sink_configuration.html
new file mode 100644
index 0000000..f207e5f
--- /dev/null
+++ b/docs/layouts/shortcodes/generated/pulsar_sink_configuration.html
@@ -0,0 +1,54 @@
+<table class="configuration table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Key</th>
+ <th class="text-left" style="width: 15%">Default</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ <th class="text-left" style="width: 55%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><h5>pulsar.sink.deliveryGuarantee</h5></td>
+ <td style="word-wrap: break-word;">none</td>
+ <td><p>Enum</p></td>
+ <td>Optional delivery guarantee when committing.<br /><br
/>Possible values:<ul><li>"exactly-once": Records are only delivered
exactly-once also under failover scenarios. To build a complete exactly-once
pipeline is required that the source and sink support exactly-once and are
properly configured.</li><li>"at-least-once": Records are ensured to be
delivered but it may happen that the same record is delivered multiple times.
Usually, this guarantee is faster than the exactly- [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.sink.enableMetrics</h5></td>
+ <td style="word-wrap: break-word;">true</td>
+ <td>Boolean</td>
+ <td>The metrics from Pulsar Producer are only exposed if you
enable this option.You should set the <code
class="highlighter-rouge">pulsar.client.statsIntervalSeconds</code> to a
positive value if you enable this option.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.sink.enableSchemaEvolution</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If you enable this option and use
PulsarSerializationSchema.pulsarSchema(), we would consume and deserialize the
message by using Pulsar's <code class="highlighter-rouge">Schema</code>.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.sink.maxRecommitTimes</h5></td>
+ <td style="word-wrap: break-word;">5</td>
+ <td>Integer</td>
+ <td>The allowed transaction recommit times if we meet some
retryable exception. This is used in Pulsar Transaction.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.sink.messageKeyHash</h5></td>
+ <td style="word-wrap: break-word;">murmur-3-32-hash</td>
+ <td><p>Enum</p></td>
+ <td>The hash policy for routing message by calculating the hash
code of message key.<br /><br />Possible values:<ul><li>"java-hash": This hash
would use <code class="highlighter-rouge">String.hashCode()</code> to calculate
the message key string's hash code.</li><li>"murmur-3-32-hash": This hash would
calculate message key's hash code by using <a
href="https://en.wikipedia.org/wiki/MurmurHash";>Murmur3</a>
algorithm.</li></ul></td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.sink.topicMetadataRefreshInterval</h5></td>
+ <td style="word-wrap: break-word;">1800000</td>
+ <td>Long</td>
+ <td>Auto update the topic metadata in a fixed interval (in ms).
The default value is 30 minutes.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.sink.transactionTimeoutMillis</h5></td>
+ <td style="word-wrap: break-word;">10800000</td>
+ <td>Long</td>
+ <td>This option is used when the user require the <code
class="highlighter-rouge">DeliveryGuarantee.EXACTLY_ONCE</code> semantic.We
would use transaction for making sure the message could be write only once.</td>
+ </tr>
+ </tbody>
+</table>
diff --git a/docs/layouts/shortcodes/generated/pulsar_source_configuration.html
b/docs/layouts/shortcodes/generated/pulsar_source_configuration.html
new file mode 100644
index 0000000..e28ffd5
--- /dev/null
+++ b/docs/layouts/shortcodes/generated/pulsar_source_configuration.html
@@ -0,0 +1,66 @@
+<table class="configuration table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Key</th>
+ <th class="text-left" style="width: 15%">Default</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ <th class="text-left" style="width: 55%">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><h5>pulsar.source.allowKeySharedOutOfOrderDelivery</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>If enabled, it will relax the ordering requirement, allowing
the broker to send out-of-order messages in case of failures. This will make it
faster for new consumers to join without being stalled by an existing slow
consumer.<br />In this case, a single consumer will still receive all the keys,
but they may be coming in different orders.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.autoCommitCursorInterval</h5></td>
+ <td style="word-wrap: break-word;">5000</td>
+ <td>Long</td>
+ <td>This option is used only when the user disables the checkpoint
and uses Exclusive or Failover subscription. We would automatically commit the
cursor using the given period (in ms).</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.enableAutoAcknowledgeMessage</h5></td>
+ <td style="word-wrap: break-word;">false</td>
+ <td>Boolean</td>
+ <td>Flink commits the consuming position with pulsar transactions
on checkpoint. However, if you have disabled the Flink checkpoint or disabled
transaction for your Pulsar cluster, ensure that you have set this option to
<code class="highlighter-rouge">true</code>.<br />The source would use pulsar
client's internal mechanism and commit cursor in two ways.<ul><li>For <code
class="highlighter-rouge">Key_Shared</code> and <code
class="highlighter-rouge">Shared</code> subscriptio [...]
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.enableMetrics</h5></td>
+ <td style="word-wrap: break-word;">true</td>
+ <td>Boolean</td>
+ <td>The metrics from Pulsar Consumer are only exposed if you
enable this option.You should set the <code
class="highlighter-rouge">pulsar.client.statsIntervalSeconds</code> to a
positive value if you enable this option.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.maxFetchRecords</h5></td>
+ <td style="word-wrap: break-word;">100</td>
+ <td>Integer</td>
+ <td>The maximum number of records to fetch to wait when polling. A
longer time increases throughput but also latency. A fetch batch might be
finished earlier because of <code
class="highlighter-rouge">pulsar.source.maxFetchTime</code>.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.maxFetchTime</h5></td>
+ <td style="word-wrap: break-word;">10000</td>
+ <td>Long</td>
+ <td>The maximum time (in ms) to wait when fetching records. A
longer time increases throughput but also latency. A fetch batch might be
finished earlier because of <code
class="highlighter-rouge">pulsar.source.maxFetchRecords</code>.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.partitionDiscoveryIntervalMs</h5></td>
+ <td style="word-wrap: break-word;">30000</td>
+ <td>Long</td>
+ <td>The interval (in ms) for the Pulsar source to discover the new
partitions. A non-positive value disables the partition discovery.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.transactionTimeoutMillis</h5></td>
+ <td style="word-wrap: break-word;">10800000</td>
+ <td>Long</td>
+ <td>This option is used in <code
class="highlighter-rouge">Shared</code> or <code
class="highlighter-rouge">Key_Shared</code> subscription. You should configure
this option when you do not enable the <code
class="highlighter-rouge">pulsar.source.enableAutoAcknowledgeMessage</code>
option.<br />The value (in ms) should be greater than the checkpoint
interval.</td>
+ </tr>
+ <tr>
+ <td><h5>pulsar.source.verifyInitialOffsets</h5></td>
+ <td style="word-wrap: break-word;">WARN_ON_MISMATCH</td>
+ <td><p>Enum</p></td>
+ <td>Upon (re)starting the source, check whether the expected
message can be read. If failure is enabled, the application fails. Otherwise,
it logs a warning. A possible solution is to adjust the retention settings in
Pulsar or ignoring the check result.<br /><br />Possible
values:<ul><li>"FAIL_ON_MISMATCH": Fail the consuming from Pulsar when we don't
find the related cursor.</li><li>"WARN_ON_MISMATCH": Print a warn message and
start consuming from the valid offset.</li></ul></td>
+ </tr>
+ </tbody>
+</table>
