sijie closed pull request #1760: Pulsar Functions resource docs
URL: https://github.com/apache/incubator-pulsar/pull/1760
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/site/_data/cli/pulsar-admin.yaml b/site/_data/cli/pulsar-admin.yaml
index 110be5c54d..9470d7c07c 100644
--- a/site/_data/cli/pulsar-admin.yaml
+++ b/site/_data/cli/pulsar-admin.yaml
@@ -120,6 +120,12 @@ commands:
- name: localrun
description: Run a Pulsar Function locally
options:
+ - flags: --cpu
+ description: The CPU to allocate to each function instance (in number of
cores)
+ - flags: --ram
+ description: The RAM to allocate to each function instance (in bytes)
+ - flags: --disk
+ description: The disk space to allocate to each function instance (in
bytes)
- flags: --brokerServiceUrl
description: The URL of the Pulsar broker
- flags: --className
@@ -163,6 +169,12 @@ commands:
- name: create
description: Creates a new Pulsar Function on the target infrastructure
options:
+ - flags: --cpu
+ description: The CPU to allocate to each function instance (in number of
cores)
+ - flags: --ram
+ description: The RAM to allocate to each function instance (in bytes)
+ - flags: --disk
+ description: The disk space to allocate to each function instance (in
bytes)
- flags: --className
description: The name of the function's class
- flags: --customSerdeInputs
diff --git a/site/docs/latest/admin-api/persistent-topics.md
b/site/docs/latest/admin-api/persistent-topics.md
index 452ece9773..4dbd73b664 100644
--- a/site/docs/latest/admin-api/persistent-topics.md
+++ b/site/docs/latest/admin-api/persistent-topics.md
@@ -93,8 +93,6 @@ Permission can be fetched using
[`permissions`](../../reference/CliTools#permiss
#### pulsar-admin
-TODO: admin
-
```shell
$ pulsar-admin persistent permissions \
persistent://test-tenant/ns1/tp1 \
diff --git a/site/docs/latest/functions/deployment.md
b/site/docs/latest/functions/deployment.md
index c0871bfa79..aceb7b0804 100644
--- a/site/docs/latest/functions/deployment.md
+++ b/site/docs/latest/functions/deployment.md
@@ -40,7 +40,7 @@ When managing Pulsar Functions, you'll need to specify a
variety of information
Parameter | Default
:---------|:-------
-Function name | Whichever value is specified for the class name. For example,
`--className org.example.MyFunction` would give the function a name of
`MyFunction`
+Function name | Whichever value is specified for the class name (minus org,
library, etc.). The flag `--className org.example.MyFunction`, for example,
would give the function a name of `MyFunction`.
Tenant | Derived from the input topics' names. If the input topics are under
the `marketing` tenant---i.e. the topic names have the form
`persistent://marketing/{namespace}/{topicName}`---then the tenant will be
`marketing`.
Namespace | Derived from the input topics' names. If the input topics are
under the `asia` namespace under the `marketing` tenant---i.e. the topic names
have the form `persistent://marketing/asia/{topicName}`, then the namespace
will be `asia`.
Output topic | `{input topic}-{function name}-output`. A function with an
input topic name of `incoming` and a function name of `exclamation`, for
example, would have an output topic of `incoming-exclamation-output`.
@@ -143,6 +143,30 @@ $ bin/pulsar-admin functions update \
--functionConfigFile function-config.yaml
```
+### Function instance resources {#resources}
+
+When you run Pulsar Functions in [cluster run](#cluster-run) mode, you can
specify the resources that are assigned to each function
[instance](#parallelism):
+
+Resource | Specified as... | Runtimes
+:--------|:----------------|:--------
+CPU | The number of cores | Docker (coming soon)
+RAM | The number of bytes | Process, Docker
+Disk space | The number of bytes | Docker
+
+Here's an example function creation command that allocates 8 cores, 8 GB of
RAM, and 10 GB of disk space to a function:
+
+```bash
+$ bin/pulsar-admin functions create \
+ --jar target/my-functions.jar \
+ --className org.example.functions.MyFunction \
+ --cpu 8 \
+ --ram 8589934592 \
+ --disk 10737418240
+```
+
+{% include admonition.html type="warning" title="Resources are *per instance*"
+ content="The resources that you apply to a given Pulsar Function are
applied to each [instance](#parallelism) of the function. If you apply 8 GB of
RAM to a function with a paralellism of 5, for example, then you are applying
40 GB of RAM total for the function. You should always make sure to factor
paralellism---i.e. the number of instances---into your resource calculations."
%}
+
## Triggering Pulsar Functions {#triggering}
If a Pulsar Function is running in [cluster mode](#cluster-mode), you can
**trigger** it at any time using the command line. Triggering a function means
that you send a message with a specific value to the function and get the
function's output (if any) via the command line.
diff --git a/site/docs/latest/functions/overview.md
b/site/docs/latest/functions/overview.md
index d9780ee8eb..8654b95319 100644
--- a/site/docs/latest/functions/overview.md
+++ b/site/docs/latest/functions/overview.md
@@ -194,9 +194,10 @@ Pulsar Functions can currently be written in
[Java](../../functions/api#java) an
## The Pulsar Functions API {#api}
-* Type safe (bytes versus specific types)
-* SerDe (built-in vs. custom)
-* Pulsar messages are always just bytes, but Pulsar Functions handles data
types for you *unless* you need custom types
+The Pulsar Functions API enables you to create processing logic that is:
+
+* Type safe. Pulsar Functions can process raw bytes or more complex,
application-specific types.
+* Based on SerDe (**Ser**ialization/**De**serialization). A variety of types
are supported "out of the box" but you can also create your own custom SerDe
logic.
### Function context {#context}
@@ -317,6 +318,29 @@ $ bin/pulsar-admin functions create \
--parallelism 5
```
+### Function instance resources {#resources}
+
+When you run Pulsar Functions in [cluster run](#cluster-run) mode, you can
specify the resources that are assigned to each function
[instance](#parallelism):
+
+Resource | Specified as... | Runtimes
+:--------|:----------------|:--------
+CPU | The number of cores | Docker (coming soon)
+RAM | The number of bytes | Process, Docker
+Disk space | The number of bytes | Docker
+
+Here's an example function creation command that allocates 8 cores, 8 GB of
RAM, and 10 GB of disk space to a function:
+
+```bash
+$ bin/pulsar-admin functions create \
+ --jar target/my-functions.jar \
+ --className org.example.functions.MyFunction \
+ --cpu 8 \
+ --ram 8589934592 \
+ --disk 10737418240
+```
+
+For more information on resources, see the [Deploying and Managing Pulsar
Functions](../deployment#resources) documentation.
+
### Logging
Pulsar Functions created using the [Pulsar Functions SDK(#sdk) can send logs
to a log topic that you specify as part of the function's configuration. The
function created using the command below, for example, would produce all logs
on the `persistent://public/default/my-func-1-log` topic:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
