This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new c90dc2019ce Docs sync done from apache/pulsar(#423ab75)
c90dc2019ce is described below
commit c90dc2019ce9b771a733f717e7c98c4bed148d59
Author: Pulsar Site Updater <[email protected]>
AuthorDate: Tue Aug 23 06:09:14 2022 +0000
Docs sync done from apache/pulsar(#423ab75)
---
site2/website-next/docs/admin-api-topics.md | 16 +++++-----
site2/website-next/docs/functions-concepts.md | 3 +-
site2/website-next/docs/functions-develop-api.md | 30 +++++++++++++++++++
site2/website-next/docs/io-debug.md | 4 +--
site2/website-next/docs/io-develop.md | 20 ++++++-------
site2/website-next/docs/reference-metrics.md | 2 +-
.../static/swagger/restApiVersions.json | 34 +++++++++++-----------
7 files changed, 70 insertions(+), 39 deletions(-)
diff --git a/site2/website-next/docs/admin-api-topics.md
b/site2/website-next/docs/admin-api-topics.md
index 2bdab4a4e60..470f387906b 100644
--- a/site2/website-next/docs/admin-api-topics.md
+++ b/site2/website-next/docs/admin-api-topics.md
@@ -417,13 +417,13 @@ You can check the following statistics of a given
non-partitioned topic.
- **nonContiguousDeletedMessagesRangesSerializedSize**: The
serialized size of non-contiguous deleted messages ranges.
- - **throughEntryFilterMsgs**: The number of messages passes
through `EntryFilter`.
+ - **filterProcessedMsgCount**: The number of messages processed by
`EntryFilter`.
- - **entryFilterAccepted**: The number of messages accepted by
`EntryFilter`.
+ - **filterAcceptedMsgCount**: The number of messages accepted by
`EntryFilter`.
- - **entryFilterRejected**: The number of messages rejected by
`EntryFilter`.
+ - **filterRejectedMsgCount**: The number of messages rejected by
`EntryFilter`.
- - **entryFilterRescheduled**: The number of messages rescheduled
by `EntryFilter`.
+ - **filterRescheduledMsgCount**: The number of messages
rescheduled by `EntryFilter`.
- **consumers**: The list of connected consumers for this
subscription.
@@ -549,10 +549,10 @@ The following is an example of a topic status.
"lastConsumedTimestamp" : 1623230583946,
"lastAckedTimestamp" : 1623230584033,
"lastMarkDeleteAdvancedTimestamp" : 1623230584033,
- "throughEntryFilterMsgs": 100,
- "entryFilterAccepted": 100,
- "entryFilterRejected": 0,
- "entryFilterRescheduled": 0,
+ "filterProcessedMsgCount": 100,
+ "filterAcceptedMsgCount": 100,
+ "filterRejectedMsgCount": 0,
+ "filterRescheduledMsgCount": 0,
"consumers" : [ {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
diff --git a/site2/website-next/docs/functions-concepts.md
b/site2/website-next/docs/functions-concepts.md
index 4f40f8ec7be..8b49b62edc8 100644
--- a/site2/website-next/docs/functions-concepts.md
+++ b/site2/website-next/docs/functions-concepts.md
@@ -130,7 +130,8 @@ Java, Python, and Go SDKs provide access to a **context
object** that can be use
* An interface for storing and retrieving state in [state
storage](functions-develop-state.md).
* A function to publish new messages onto arbitrary topics.
* A function to acknowledge the message being processed (if auto-ack is
disabled).
-* (Java) get Pulsar admin client.
+* (Java) A function to get the Pulsar admin client.
+* (Java) A function to create a Record to return with default values taken
from the Context and the input Record.
:::tip
diff --git a/site2/website-next/docs/functions-develop-api.md
b/site2/website-next/docs/functions-develop-api.md
index baa21f5fbe6..0a1f4a408fb 100644
--- a/site2/website-next/docs/functions-develop-api.md
+++ b/site2/website-next/docs/functions-develop-api.md
@@ -102,6 +102,36 @@ public class ExclamationFunction implements
Function<String, String> {
For more details, see [code
example](https://github.com/apache/pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/ExclamationFunction.java).
+The return type of the function can be wrapped in a `Record` generic which
gives you more control over the output messages, such as topics, schemas,
properties, and so on.
+Use the `Context::newOutputRecordBuilder` method to build this `Record` output.
+
+```java
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.pulsar.functions.api.Context;
+import org.apache.pulsar.functions.api.Function;
+import org.apache.pulsar.functions.api.Record;
+
+public class RecordFunction implements Function<String, Record<String>> {
+
+ @Override
+ public Record<String> process(String input, Context context) throws
Exception {
+ String output = String.format("%s!", input);
+ Map<String, String> properties = new
HashMap<>(context.getCurrentRecord().getProperties());
+ context.getCurrentRecord().getTopicName().ifPresent(topic ->
properties.put("input_topic", topic));
+
+ return context.newOutputRecordBuilder(Schema.STRING)
+ .value(output)
+ .properties(properties)
+ .build();
+ }
+}
+
+```
+
+For more details, see [code
example](https://github.com/apache/pulsar/blob/master/pulsar-functions/java-examples/src/main/java/org/apache/pulsar/functions/api/examples/RecordFunction.java).
+
</TabItem>
<TabItem value="Python">
diff --git a/site2/website-next/docs/io-debug.md
b/site2/website-next/docs/io-debug.md
index dbde5ab82d4..7fa071bc486 100644
--- a/site2/website-next/docs/io-debug.md
+++ b/site2/website-next/docs/io-debug.md
@@ -4,7 +4,7 @@ title: How to debug Pulsar connectors
sidebar_label: "Debug"
---
This guide explains how to debug connectors in localrun or cluster mode and
gives a debugging checklist.
-To better demonstrate how to debug Pulsar connectors, here takes a Mongo sink
connector as an example.
+To better demonstrate how to debug Pulsar connectors, take the Mongo sink
connector as an example.
**Deploy a Mongo sink environment**
1. Start a Mongo service.
@@ -106,7 +106,7 @@ Use one of the following methods to get a connector log in
localrun mode:
```
-To clearly explain the log information, here breaks down the large block of
information into small blocks and add descriptions for each block.
+To clearly explain the log information, the following is a breakdown into
smaller blocks with added descriptions.
* This piece of log information shows the storage path of the nar package
after decompression.
```
diff --git a/site2/website-next/docs/io-develop.md
b/site2/website-next/docs/io-develop.md
index d361a302b25..1c814a85f6a 100644
--- a/site2/website-next/docs/io-develop.md
+++ b/site2/website-next/docs/io-develop.md
@@ -70,7 +70,7 @@ interface, which means you need to implement the {@inject:
github:open:/pulsar-i
```
- If nothing to return, the implementation should be blocking rather than
returning `null`.
+ If there is nothing to return, the implementation should be blocking rather
than returning `null`.
The returned {@inject:
github:Record:/pulsar-functions/api-java/src/main/java/org/apache/pulsar/functions/api/Record.java}
should encapsulate the following information, which is needed by Pulsar IO
runtime.
@@ -95,10 +95,10 @@ interface, which means you need to implement the {@inject:
github:open:/pulsar-i
`ack` |Acknowledge that the record is fully processed.
`fail`|Indicate that the record fails to be processed.
-## Handle schema information
+#### Handle schema information
Pulsar IO automatically handles the schema and provides a strongly typed API
based on Java generics.
-If you know the schema type that you are producing, you can declare the Java
class relative to that type in your sink declaration.
+If you know the schema type that you are producing, you can declare the Java
class relative to that type in your source declaration.
```
@@ -136,8 +136,8 @@ To handle the `KeyValue` type properly, follow the
guidelines for your record im
- It must return a `KeyValue` object as `Record.getValue()`
- It may return null in `Record.getSchema()`
-When Pulsar IO runtime encounters a `KVRecord`, it brings the following
changes automatically:
-- Set properly the `KeyValueSchema`
+When Pulsar IO runtime encounters a `KVRecord`, it does the following changes
automatically:
+- Set the proper `KeyValueSchema`
- Encode the Message Key and the Message Value according to the
`KeyValueEncoding` (SEPARATED or INLINE)
:::tip
@@ -179,14 +179,14 @@ Developing a sink connector **is similar to** developing
a source connector, tha
```
During the implementation, you can decide how to write the `Value` and
- the `Key` to the actual source, and leverage all the provided information
such as
+ the `Key` to the actual sink, and leverage all the provided information
such as
`PartitionId` and `RecordSequence` to achieve different processing
guarantees.
You also need to ack records (if messages are sent successfully) or fail
records (if messages fail to send).
-## Handling Schema information
+#### Handle schema information
-Pulsar IO handles automatically the Schema and provides a strongly typed API
based on Java generics.
+Pulsar IO automatically handles the Schema and provides a strongly typed API
based on Java generics.
If you know the Schema type that you are consuming from you can declare the
Java class relative to that type in your Sink declaration.
```
@@ -282,11 +282,11 @@ work with Pulsar Functions' runtime, that is, [NAR](#nar)
and [uber JAR](#uber-j
:::note
If you plan to package and distribute your connector for others to use, you
are obligated to
+license and copyright your own code properly. Remember to add the license and
copyright to
+all libraries your code uses and to your distribution.
:::
-license and copyright your own code properly. Remember to add the license and
copyright to
-all libraries your code uses and to your distribution.
>
> If you use the [NAR](#nar) method, the NAR plugin
automatically creates a `DEPENDENCIES` file in the generated NAR package,
including the proper
diff --git a/site2/website-next/docs/reference-metrics.md
b/site2/website-next/docs/reference-metrics.md
index 31ed3baf776..aa59c02fee8 100644
--- a/site2/website-next/docs/reference-metrics.md
+++ b/site2/website-next/docs/reference-metrics.md
@@ -444,7 +444,7 @@ All the subscription metrics are labelled with the
following labels:
| pulsar_subscription_total_msg_expired | Gauge | The total number of messages
expired on this subscription. |
| pulsar_subscription_msg_drop_rate | Gauge | The rate of messages dropped on
this subscription (message per second). |
| pulsar_subscription_consumers_count | Gauge | The number of connected
consumers on this subscription. |
-| pulsar_subscription_through_filter_msg_count | Counter | The number of
messages passes through `EntryFilter`. |
+| pulsar_subscription_filter_processed_msg_count | Counter | The number of
messages processed by `EntryFilter`. |
| pulsar_subscription_filter_accepted_msg_count | Counter | The number of
messages accepted by `EntryFilter`. |
| pulsar_subscription_filter_rejected_msg_count | Counter | The number of
messages rejected by `EntryFilter`. |
| pulsar_subscription_filter_rescheduled_msg_count | Counter | The number of
messages rescheduled by `EntryFilter`. |
diff --git a/site2/website-next/static/swagger/restApiVersions.json
b/site2/website-next/static/swagger/restApiVersions.json
index 34ad9254ef4..8d35bd4d0b0 100644
--- a/site2/website-next/static/swagger/restApiVersions.json
+++ b/site2/website-next/static/swagger/restApiVersions.json
@@ -1,4 +1,21 @@
{
+ "2.10.0": [
+ {
+ "version": "v2",
+ "fileName": [
+ "swagger"
+ ]
+ },
+ {
+ "version": "v3",
+ "fileName": [
+ "swaggerfunctions",
+ "swaggerpackages",
+ "swaggersink",
+ "swaggersource"
+ ]
+ }
+ ],
"2.10.1": [
{
"version": "v2",
@@ -339,23 +356,6 @@
]
}
],
- "2.10.0": [
- {
- "version": "v2",
- "fileName": [
- "swagger"
- ]
- },
- {
- "version": "v3",
- "fileName": [
- "swaggerfunctions",
- "swaggerpackages",
- "swaggersink",
- "swaggersource"
- ]
- }
- ],
"2.8.3": [
{
"version": "v2",
