This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new cb4f6ad Pulsar Functions resource docs (#1760)
cb4f6ad is described below
commit cb4f6ad71f6365b5369ca9705c3bf6985335995a
Author: Luc Perkins <[email protected]>
AuthorDate: Sat Jun 9 14:46:40 2018 -0500
Pulsar Functions resource docs (#1760)
* add section on function resources to overview
* add resource flags to YAML
* add section to deployment doc
* add runtime info
---
site/_data/cli/pulsar-admin.yaml | 12 ++++++++++
site/docs/latest/admin-api/persistent-topics.md | 2 --
site/docs/latest/functions/deployment.md | 26 ++++++++++++++++++++-
site/docs/latest/functions/overview.md | 30 ++++++++++++++++++++++---
4 files changed, 64 insertions(+), 6 deletions(-)
diff --git a/site/_data/cli/pulsar-admin.yaml b/site/_data/cli/pulsar-admin.yaml
index 110be5c..9470d7c 100644
--- a/site/_data/cli/pulsar-admin.yaml
+++ b/site/_data/cli/pulsar-admin.yaml
@@ -120,6 +120,12 @@ commands:
- name: localrun
description: Run a Pulsar Function locally
options:
+ - flags: --cpu
+ description: The CPU to allocate to each function instance (in number of
cores)
+ - flags: --ram
+ description: The RAM to allocate to each function instance (in bytes)
+ - flags: --disk
+ description: The disk space to allocate to each function instance (in
bytes)
- flags: --brokerServiceUrl
description: The URL of the Pulsar broker
- flags: --className
@@ -163,6 +169,12 @@ commands:
- name: create
description: Creates a new Pulsar Function on the target infrastructure
options:
+ - flags: --cpu
+ description: The CPU to allocate to each function instance (in number of
cores)
+ - flags: --ram
+ description: The RAM to allocate to each function instance (in bytes)
+ - flags: --disk
+ description: The disk space to allocate to each function instance (in
bytes)
- flags: --className
description: The name of the function's class
- flags: --customSerdeInputs
diff --git a/site/docs/latest/admin-api/persistent-topics.md
b/site/docs/latest/admin-api/persistent-topics.md
index 452ece9..4dbd73b 100644
--- a/site/docs/latest/admin-api/persistent-topics.md
+++ b/site/docs/latest/admin-api/persistent-topics.md
@@ -93,8 +93,6 @@ Permission can be fetched using
[`permissions`](../../reference/CliTools#permiss
#### pulsar-admin
-TODO: admin
-
```shell
$ pulsar-admin persistent permissions \
persistent://test-tenant/ns1/tp1 \
diff --git a/site/docs/latest/functions/deployment.md
b/site/docs/latest/functions/deployment.md
index c0871bf..aceb7b0 100644
--- a/site/docs/latest/functions/deployment.md
+++ b/site/docs/latest/functions/deployment.md
@@ -40,7 +40,7 @@ When managing Pulsar Functions, you'll need to specify a
variety of information
Parameter | Default
:---------|:-------
-Function name | Whichever value is specified for the class name. For example,
`--className org.example.MyFunction` would give the function a name of
`MyFunction`
+Function name | Whichever value is specified for the class name (minus org,
library, etc.). The flag `--className org.example.MyFunction`, for example,
would give the function a name of `MyFunction`.
Tenant | Derived from the input topics' names. If the input topics are under
the `marketing` tenant---i.e. the topic names have the form
`persistent://marketing/{namespace}/{topicName}`---then the tenant will be
`marketing`.
Namespace | Derived from the input topics' names. If the input topics are
under the `asia` namespace under the `marketing` tenant---i.e. the topic names
have the form `persistent://marketing/asia/{topicName}`, then the namespace
will be `asia`.
Output topic | `{input topic}-{function name}-output`. A function with an
input topic name of `incoming` and a function name of `exclamation`, for
example, would have an output topic of `incoming-exclamation-output`.
@@ -143,6 +143,30 @@ $ bin/pulsar-admin functions update \
--functionConfigFile function-config.yaml
```
+### Function instance resources {#resources}
+
+When you run Pulsar Functions in [cluster run](#cluster-run) mode, you can
specify the resources that are assigned to each function
[instance](#parallelism):
+
+Resource | Specified as... | Runtimes
+:--------|:----------------|:--------
+CPU | The number of cores | Docker (coming soon)
+RAM | The number of bytes | Process, Docker
+Disk space | The number of bytes | Docker
+
+Here's an example function creation command that allocates 8 cores, 8 GB of
RAM, and 10 GB of disk space to a function:
+
+```bash
+$ bin/pulsar-admin functions create \
+ --jar target/my-functions.jar \
+ --className org.example.functions.MyFunction \
+ --cpu 8 \
+ --ram 8589934592 \
+ --disk 10737418240
+```
+
+{% include admonition.html type="warning" title="Resources are *per instance*"
+ content="The resources that you apply to a given Pulsar Function are
applied to each [instance](#parallelism) of the function. If you apply 8 GB of
RAM to a function with a paralellism of 5, for example, then you are applying
40 GB of RAM total for the function. You should always make sure to factor
paralellism---i.e. the number of instances---into your resource calculations."
%}
+
## Triggering Pulsar Functions {#triggering}
If a Pulsar Function is running in [cluster mode](#cluster-mode), you can
**trigger** it at any time using the command line. Triggering a function means
that you send a message with a specific value to the function and get the
function's output (if any) via the command line.
diff --git a/site/docs/latest/functions/overview.md
b/site/docs/latest/functions/overview.md
index d9780ee..8654b95 100644
--- a/site/docs/latest/functions/overview.md
+++ b/site/docs/latest/functions/overview.md
@@ -194,9 +194,10 @@ Pulsar Functions can currently be written in
[Java](../../functions/api#java) an
## The Pulsar Functions API {#api}
-* Type safe (bytes versus specific types)
-* SerDe (built-in vs. custom)
-* Pulsar messages are always just bytes, but Pulsar Functions handles data
types for you *unless* you need custom types
+The Pulsar Functions API enables you to create processing logic that is:
+
+* Type safe. Pulsar Functions can process raw bytes or more complex,
application-specific types.
+* Based on SerDe (**Ser**ialization/**De**serialization). A variety of types
are supported "out of the box" but you can also create your own custom SerDe
logic.
### Function context {#context}
@@ -317,6 +318,29 @@ $ bin/pulsar-admin functions create \
--parallelism 5
```
+### Function instance resources {#resources}
+
+When you run Pulsar Functions in [cluster run](#cluster-run) mode, you can
specify the resources that are assigned to each function
[instance](#parallelism):
+
+Resource | Specified as... | Runtimes
+:--------|:----------------|:--------
+CPU | The number of cores | Docker (coming soon)
+RAM | The number of bytes | Process, Docker
+Disk space | The number of bytes | Docker
+
+Here's an example function creation command that allocates 8 cores, 8 GB of
RAM, and 10 GB of disk space to a function:
+
+```bash
+$ bin/pulsar-admin functions create \
+ --jar target/my-functions.jar \
+ --className org.example.functions.MyFunction \
+ --cpu 8 \
+ --ram 8589934592 \
+ --disk 10737418240
+```
+
+For more information on resources, see the [Deploying and Managing Pulsar
Functions](../deployment#resources) documentation.
+
### Logging
Pulsar Functions created using the [Pulsar Functions SDK(#sdk) can send logs
to a log topic that you specify as part of the function's configuration. The
function created using the command below, for example, would produce all logs
on the `persistent://public/default/my-func-1-log` topic:
--
To stop receiving notification emails like this one, please contact
[email protected].
