This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
The following commit(s) were added to refs/heads/main by this push:
new 5f7b09315 chore: the Google Workspace Kamelets can log in with a
service account, and Vertex AI and IBM COS take more options (#3170) (#3229)
5f7b09315 is described below
commit 5f7b093155950c9dc1c6b15a244d138b6c09a516
Author: Claus Ibsen <[email protected]>
AuthorDate: Wed Nov 18 22:36:16 2026 +0100
chore: the Google Workspace Kamelets can log in with a service account, and
Vertex AI and IBM COS take more options (#3170) (#3229)
google-calendar-source, google-mail-source, google-sheets-source and
google-sheets-sink could only log in with OAuth, and its four options
were required. They now also take, all optional and without a default:
- serviceAccountKey, the JSON key of a service account as a Camel
resource (file:, classpath: or base64:), a secret mapped in RAW()
- delegate, the Workspace user to act for (domain-wide delegation)
- scopes, which Gmail and Sheets need with a service account, as their
components have no default scopes
clientId, clientSecret, accessToken and refreshToken are no longer
required, and are now optional placeholders. Sheets falls back to the
Application Default Credentials when no login option is set.
The sources take the paging and rendering options of their APIs:
- google-calendar-source: maxResults, query (in RAW()) and
considerLastUpdate
- google-mail-source: maxResults, and raw, for the whole email with its
attachments
- google-sheets-source: valueRenderOption and maxResults
google-vertexai-sink takes publisher and anthropicVersion, so rawPredict
can call partner models such as Claude, and jsonMode. candidateCount and
the streaming operations are left out: in a sink the body gets only the
first candidate, and streaming gives the same result as the operations
the Kamelet already has.
ibm-cos-source takes fileName, to read one object, and ibm-cos-sink
takes deleteAfterWrite, which deletes the local file of the body once
it has been uploaded.
When none of the new options is set, the endpoint URIs are the same as
before, checked with camel get endpoint for all seven Kamelets. The
descriptions and the partial pages say how the logins combine, and have
new examples.
Verified with Camel JBang against local stand-ins, not against the real
services. For Calendar, Gmail and Sheets, the clients came from the
components' own factories, pointed at a recording HTTP server, with a
generated service account key whose token endpoint is local. Vertex AI
used GOOGLE_VERTEX_BASE_URL and a fake Prediction Service client. For
IBM COS, the Kamelet logged in with its API key through a local IAM
endpoint, against floci as the S3 stand-in.
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
.../google-calendar-source-description.adoc | 55 ++++++++++++++++++----
.../partials/google-mail-source-description.adoc | 54 +++++++++++++++++----
.../partials/google-sheets-sink-description.adoc | 33 +++++++++++--
.../partials/google-sheets-source-description.adoc | 47 +++++++++++++-----
.../partials/google-vertexai-sink-description.adoc | 32 ++++++++++++-
.../ROOT/partials/ibm-cos-sink-description.adoc | 28 +++++++++++
.../ROOT/partials/ibm-cos-source-description.adoc | 22 +++++++++
kamelets/google-calendar-source.kamelet.yaml | 52 ++++++++++++++++----
kamelets/google-mail-source.kamelet.yaml | 48 ++++++++++++++-----
kamelets/google-sheets-sink.kamelet.yaml | 46 +++++++++++++-----
kamelets/google-sheets-source.kamelet.yaml | 48 +++++++++++++++----
kamelets/google-vertexai-sink.kamelet.yaml | 33 +++++++++++--
kamelets/ibm-cos-sink.kamelet.yaml | 7 ++-
kamelets/ibm-cos-source.kamelet.yaml | 9 +++-
.../main/resources/META-INF/kamelets-index.json | 14 +++---
.../kamelets/google-calendar-source.kamelet.yaml | 52 ++++++++++++++++----
.../kamelets/google-mail-source.kamelet.yaml | 48 ++++++++++++++-----
.../kamelets/google-sheets-sink.kamelet.yaml | 46 +++++++++++++-----
.../kamelets/google-sheets-source.kamelet.yaml | 48 +++++++++++++++----
.../kamelets/google-vertexai-sink.kamelet.yaml | 33 +++++++++++--
.../resources/kamelets/ibm-cos-sink.kamelet.yaml | 7 ++-
.../resources/kamelets/ibm-cos-source.kamelet.yaml | 9 +++-
22 files changed, 630 insertions(+), 141 deletions(-)
diff --git a/docs/modules/ROOT/partials/google-calendar-source-description.adoc
b/docs/modules/ROOT/partials/google-calendar-source-description.adoc
index 6a312f247..1069d369a 100644
--- a/docs/modules/ROOT/partials/google-calendar-source-description.adoc
+++ b/docs/modules/ROOT/partials/google-calendar-source-description.adoc
@@ -2,12 +2,12 @@
=== Authentication methods
-This Kamelet uses OAuth 2.0 authentication to connect to Google Calendar. You
need to provide:
+The Kamelet logs in to Google in one of two ways:
-- Client ID and Client Secret from your Google Cloud Console application
-- Access Token for immediate access (typically expires after an hour)
-- Refresh Token for long-term usage (allows automatic token renewal)
-- An application name of your choice, sent to Google with each call
+- With OAuth 2.0: `clientId` and `clientSecret` of an OAuth client from your
Google Cloud Console application, an `accessToken` for immediate access (it
expires after about an hour) and a `refreshToken` to get new ones.
+- With a service account: its JSON key in `serviceAccountKey`, as
`file:<path>`, `classpath:<resource>` or `base64:<Base64 of the key file>`. The
service account reads the calendars shared with its email address. To read the
calendar of a user of your Google Workspace, set `delegate` to the email
address of the user; a Workspace administrator must first allow the client ID
of the service account under Security > Access and data control > API controls
> Domain-wide delegation, with the sc [...]
+
+An application name of your choice is sent to Google with each call.
=== Output format
@@ -35,16 +35,30 @@ The Kamelet requires the following parameters:
- `index`: An index for the Google Calendar endpoint
- `calendarId`: The calendar ID to use as the source of event data
+- `applicationName`: The Google Calendar application name
+
+To log in with OAuth:
+
- `clientId`: The Client ID of the Google Calendar application
- `clientSecret`: The Client secret of the Google Calendar application
- `accessToken`: The OAuth 2 access token
- `refreshToken`: The OAuth 2 refresh token for long term usage
-- `applicationName`: The Google Calendar application name
+
+To log in with a service account:
+
+- `serviceAccountKey`: The JSON key of the service account, such as
`file:/etc/google/service-account.json`
+- `delegate`: The email address of the user whose calendar to read, with
domain-wide delegation
+- `scopes`: The permissions to ask for, such as
`https://www.googleapis.com/auth/calendar.readonly` (default:
`https://www.googleapis.com/auth/calendar`)
Optional parameters:
- `delay`: The number of milliseconds before the next poll (default: 500)
-- `syncFlow`: Specifies to sync events for incremental synchronization
(default: false)
+- `syncFlow`: Specifies to sync events for incremental synchronization
(default: false); do not combine it with `query` or `considerLastUpdate`
- `consumeFromNow`: Specifies to consume events in the calendar from now on
(default: true)
+- `maxResults`: The most events in one page; each poll reads one page, or all
the pages with `considerLastUpdate` (default: 10)
+- `query`: Only read the events that contain this text, such as `standup`
+- `considerLastUpdate`: Each poll reads all the pages, and after the first
only the events changed since the last one received (default: false)
+
+Each poll asks for the events that changed least recently first. Without
`syncFlow` or `considerLastUpdate`, it reads the first page again on every
poll, so the same events can come again.
=== Usage example
@@ -75,4 +89,29 @@ spec:
kind: Service
apiVersion: v1
name: my-service
-```
\ No newline at end of file
+```
+
+=== Example with a Service Account
+
+Reads the events of the calendar of `[email protected]` that mention
`standup`, with the key of a service account that has domain-wide delegation
for the read-only scope. With `considerLastUpdate`, an event comes again only
when it changes.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:google-calendar-source"
+ parameters:
+ index: "standups"
+ calendarId: "primary"
+ applicationName: "my-company-calendar/1.0"
+ serviceAccountKey: "file:/etc/google/service-account.json"
+ delegate: "[email protected]"
+ scopes: "https://www.googleapis.com/auth/calendar.readonly"
+ query: "standup"
+ maxResults: 50
+ considerLastUpdate: true
+ delay: 60000
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
+----
diff --git a/docs/modules/ROOT/partials/google-mail-source-description.adoc
b/docs/modules/ROOT/partials/google-mail-source-description.adoc
index a6d9e3ee9..68feef6ae 100644
--- a/docs/modules/ROOT/partials/google-mail-source-description.adoc
+++ b/docs/modules/ROOT/partials/google-mail-source-description.adoc
@@ -2,16 +2,18 @@
=== Authentication methods
-This Kamelet uses OAuth 2.0 authentication to connect to Google Mail. You need
to provide:
+The Kamelet logs in to Google in one of two ways:
-- Client ID and Client Secret from your Google Cloud Console application
-- Access Token for immediate access (typically expires after an hour)
-- Refresh Token for long-term usage (allows automatic token renewal)
-- An application name of your choice, sent to Google with each call
+- With OAuth 2.0: `clientId` and `clientSecret` of an OAuth client from your
Google Cloud Console application, an `accessToken` for immediate access (it
expires after about an hour) and a `refreshToken` to get new ones.
+- With a service account: its JSON key in `serviceAccountKey`, as
`file:<path>`, `classpath:<resource>` or `base64:<Base64 of the key file>`. It
reads the mailbox of the user of your Google Workspace in `delegate`, and needs
`scopes`, such as `https://www.googleapis.com/auth/gmail.modify`, which marking
the emails as read needs. A Workspace administrator must first allow the client
ID of the service account, with these scopes, under Security > Access and data
control > API controls > Dom [...]
+
+An application name of your choice is sent to Google with each call.
=== Output format
-The body is the text of the mail (its first part with content, such as the
plain text part) as JSON text, that is the text in double quotes with JSON
escapes. It supports two output data types:
+The body is the text of the mail (its first part with content, such as the
plain text part) as JSON text, that is the text in double quotes with JSON
escapes. With `raw` set to true, the body is instead the whole email, with its
attachments, in the format of an `.eml` file (RFC 2822), encoded in base64url,
as JSON text; the subject, sender and recipient headers are then not set.
+
+It supports two output data types:
- **json**: Json representation of a Google Mail event object
- **cloudevents**: Output data as CloudEvent V1 format with specific headers
@@ -39,17 +41,28 @@ For CloudEvents output format, additional headers are set:
The Kamelet requires the following parameters:
- `index`: An index for the google mail endpoint
+- `applicationName`: Google Mail application name
+
+To log in with OAuth:
+
- `clientId`: Client ID of the gmail application
- `clientSecret`: Client Secret of the gmail application
- `accessToken`: OAuth 2 access token
- `refreshToken`: OAuth 2 refresh token for long term usage
-- `applicationName`: Google Mail application name
-- `labels`: Comma separated list of labels to take into account (example:
"INBOX"); when it is not set, the emails are not filtered by label
+
+To log in with a service account:
+
+- `serviceAccountKey`: The JSON key of the service account, such as
`file:/etc/google/service-account.json`
+- `delegate`: The email address of the user whose mailbox to read
+- `scopes`: The permissions to ask for, such as
`https://www.googleapis.com/auth/gmail.modify`
Optional parameters:
- `delay`: The number of milliseconds before the next poll (default: 500)
- `markAsRead`: Mark the message as read once it has been consumed (default:
true)
- `query`: The query to execute on gmail box (default: "is:unread", example:
"is:unread -category:(promotions OR social)")
+- `labels`: Comma separated list of labels to take into account (example:
"INBOX"); when it is not set, the emails are not filtered by label
+- `maxResults`: The most emails to read at each poll (default: 10)
+- `raw`: The body is the whole email with its attachments, encoded in
base64url (default: false)
=== Usage example
@@ -80,4 +93,27 @@ spec:
kind: Service
apiVersion: v1
name: my-service
-```
\ No newline at end of file
+```
+
+=== Example with a Service Account and Attachments
+
+Reads the unread emails of `[email protected]`, at most 20 at each poll, as
whole emails with their attachments. Each body is a JSON string; remove the
quotes and decode it as base64url to get the `.eml` content.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:google-mail-source"
+ parameters:
+ index: "support"
+ applicationName: "my-company-mail/1.0"
+ serviceAccountKey: "file:/etc/google/service-account.json"
+ delegate: "[email protected]"
+ scopes: "https://www.googleapis.com/auth/gmail.modify"
+ maxResults: 20
+ raw: true
+ delay: 60000
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
+----
diff --git a/docs/modules/ROOT/partials/google-sheets-sink-description.adoc
b/docs/modules/ROOT/partials/google-sheets-sink-description.adoc
index 8e3734704..ffbc820d8 100644
--- a/docs/modules/ROOT/partials/google-sheets-sink-description.adoc
+++ b/docs/modules/ROOT/partials/google-sheets-sink-description.adoc
@@ -2,10 +2,11 @@
=== Authentication
-This Kamelet uses OAuth 2.0 authentication with Google Sheets API. You need to
provide:
-- Client ID and Client Secret from your Google Cloud Console application
-- Access Token (typically expires after an hour)
-- Refresh Token for long-term usage (recommended)
+The Kamelet logs in to Google in one of three ways:
+
+- With OAuth 2.0: `clientId` and `clientSecret` of an OAuth client from your
Google Cloud Console application, an `accessToken` (it expires after about an
hour) and a `refreshToken` to get new ones. Do not set `scopes` with OAuth.
+- With a service account: its JSON key in `serviceAccountKey`, as
`file:<path>`, `classpath:<resource>` or `base64:<Base64 of the key file>`, and
`scopes`, such as `https://www.googleapis.com/auth/spreadsheets`. Share the
spreadsheet with the email address of the service account as an editor, or set
`delegate` to the email address of a user of your Google Workspace; a Workspace
administrator must first allow the client ID of the service account, with these
scopes, under Security > Access [...]
+- With none of them set, the Application Default Credentials and `scopes`: the
key file named in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable, or
the service account of the Google Cloud machine it runs on.
=== Input Format
@@ -32,4 +33,26 @@ The Kamelet sets these headers from its options before it
writes, so headers tha
- `CamelGoogleSheets.spreadsheetId`: the `spreadsheetId` option
- `CamelGoogleSheets.majorDimension`: the `majorDimension` option
- `CamelGoogleSheets.columnNames`: the `columnNames` option
-- `CamelGoogleSheets.valueInputOption`: the `valueInputOption` option
\ No newline at end of file
+- `CamelGoogleSheets.valueInputOption`: the `valueInputOption` option
+
+=== Example with a Service Account
+
+Appends each message as a row to columns A and B of the spreadsheet, with the
key of a service account the spreadsheet is shared with as an editor.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: '{"A": "Alice", "B": 3}'
+ contentType: "application/json"
+ steps:
+ - to:
+ uri: "kamelet:google-sheets-sink"
+ parameters:
+ spreadsheetId: "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms"
+ serviceAccountKey: "file:/etc/google/service-account.json"
+ scopes: "https://www.googleapis.com/auth/spreadsheets"
+ range: "A:B"
+----
diff --git a/docs/modules/ROOT/partials/google-sheets-source-description.adoc
b/docs/modules/ROOT/partials/google-sheets-source-description.adoc
index 1f7b03e42..c19b5bd1c 100644
--- a/docs/modules/ROOT/partials/google-sheets-source-description.adoc
+++ b/docs/modules/ROOT/partials/google-sheets-source-description.adoc
@@ -2,12 +2,11 @@
=== Authentication methods
-This Kamelet uses OAuth 2.0 authentication to connect to Google Sheets. You
need to provide:
+The Kamelet logs in to Google in one of three ways:
-- Client ID and Client Secret from your Google Cloud Console application
-- Access Token for immediate access (typically expires after an hour)
-- Refresh Token for long-term usage (allows automatic token renewal)
-- The Spreadsheet ID from which to read data
+- With OAuth 2.0: `clientId` and `clientSecret` of an OAuth client from your
Google Cloud Console application, an `accessToken` (it expires after about an
hour) and a `refreshToken` to get new ones. Do not set `scopes` with OAuth.
+- With a service account: its JSON key in `serviceAccountKey`, as
`file:<path>`, `classpath:<resource>` or `base64:<Base64 of the key file>`, and
`scopes`, such as `https://www.googleapis.com/auth/spreadsheets.readonly`.
Share the spreadsheet with the email address of the service account, or set
`delegate` to the email address of a user of your Google Workspace; a Workspace
administrator must first allow the client ID of the service account, with these
scopes, under Security > Access and [...]
+- With none of them set, the Application Default Credentials and `scopes`: the
key file named in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable, or
the service account of the Google Cloud machine it runs on.
=== Output format
@@ -15,13 +14,17 @@ The Kamelet produces spreadsheet data in JSON format from
the specified Google S
=== Configuration
-The Kamelet requires the following parameters:
+The Kamelet requires the following parameter:
- `spreadsheetId`: The Spreadsheet ID to be used as events source
-- `clientId`: Client ID of the sheets application
-- `clientSecret`: Client Secret of the sheets application
-- `accessToken`: OAuth 2 access token
-- `refreshToken`: OAuth 2 refresh token for long term usage
+
+To log in with OAuth: `clientId`, `clientSecret`, `accessToken` and
`refreshToken`. To log in with a service account: `serviceAccountKey`, `scopes`
and optionally `delegate`.
+
+Optional parameters about the cells:
+
+- `range`: The cells to read, such as `Sheet1!A2:C100`
+- `valueRenderOption`: `FORMATTED_VALUE` (as shown, such as `$1.50`, the
default), `UNFORMATTED_VALUE` (numbers as numbers, such as `1.5`) or `FORMULA`
(the formula, such as `=B2*2`)
+- `maxResults`: The most rows of each range to send, counted from the first;
Google still returns the whole range
=== Usage example
@@ -47,4 +50,26 @@ spec:
kind: Service
apiVersion: v1
name: my-service
-```
\ No newline at end of file
+```
+
+=== Example with a Service Account
+
+Reads the first 100 rows of `Prices!A2:C` once, with numbers as numbers, with
the key of a service account the spreadsheet is shared with.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:google-sheets-source"
+ parameters:
+ spreadsheetId: "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms"
+ serviceAccountKey: "file:/etc/google/service-account.json"
+ scopes: "https://www.googleapis.com/auth/spreadsheets.readonly"
+ range: "Prices!A2:C"
+ valueRenderOption: "UNFORMATTED_VALUE"
+ maxResults: 100
+ repeatCount: 1
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
+----
diff --git a/docs/modules/ROOT/partials/google-vertexai-sink-description.adoc
b/docs/modules/ROOT/partials/google-vertexai-sink-description.adoc
index 628954fdf..5b67f3db2 100644
--- a/docs/modules/ROOT/partials/google-vertexai-sink-description.adoc
+++ b/docs/modules/ROOT/partials/google-vertexai-sink-description.adoc
@@ -16,6 +16,9 @@ If you provide a service account key, give its location:
`file:` followed by the
- **Service Account Key**: The location of the service account key, as
`file:<path>` or `base64:<Base64 of the key file>`
- **Operation**: The operation to perform (default: `generateText`). Supported
operations are `generateText`, `generateChat`, `generateImage`,
`generateEmbeddings`, `generateCode`, `generateMultimodal`, and `rawPredict`
+- **JSON Mode** (`jsonMode`): if true, the model answers with JSON text; for
`generateText`, `generateChat`, `generateCode` and `generateMultimodal`
+- **Model Publisher** (`publisher`): the publisher of a partner model for
`rawPredict`, such as `anthropic`, `meta` or `mistralai`
+- **Anthropic Version** (`anthropicVersion`): the `anthropic_version` sent to
Claude with `rawPredict` (default: `vertex-2023-10-16`)
- **Temperature**: Controls randomness in generation (0.0 to 1.0)
- **Max Output Tokens**: Maximum number of tokens to generate in the response
- **Top P**: Nucleus sampling parameter (0.0 to 1.0)
@@ -23,4 +26,31 @@ If you provide a service account key, give its location:
`file:` followed by the
=== Content Generation
-The Kamelet sends the message body as a prompt to the specified Google Vertex
AI model and returns the generated content. It supports Google native models,
such as Gemini and Imagen. The `rawPredict` operation for partner models needs
a publisher, which this Kamelet cannot set, so it fails.
+The Kamelet sends the message body as a prompt to the specified Google Vertex
AI model and returns the generated content. It supports Google native models,
such as Gemini and Imagen, and with `rawPredict` and `publisher`, partner
models of the Model Garden, such as Claude.
+
+=== Example with Claude
+
+Sends each message to Claude Sonnet 4.5 on Vertex AI. The body can be the
prompt as text, or a JSON request of the Anthropic Messages API, which is sent
as it is. With the publisher `anthropic`, the body afterwards is the text of
the answer, and the `CamelGoogleVertexAIRawResponse` header has the whole
answer.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: "Summarize the plot of Hamlet in one sentence."
+ steps:
+ - to:
+ uri: "kamelet:google-vertexai-sink"
+ parameters:
+ projectId: "my-project-123"
+ region: "us-east5"
+ modelId: "claude-sonnet-4-5"
+ operation: "rawPredict"
+ publisher: "anthropic"
+ serviceAccountKey: "file:/etc/google/service-account.json"
+ - to:
+ uri: "kamelet:log-sink"
+----
+
+The model ID and the regions where it is available are listed on the page of
the model in the Model Garden.
diff --git a/docs/modules/ROOT/partials/ibm-cos-sink-description.adoc
b/docs/modules/ROOT/partials/ibm-cos-sink-description.adoc
index 41f74e2d8..1df47dc7f 100644
--- a/docs/modules/ROOT/partials/ibm-cos-sink-description.adoc
+++ b/docs/modules/ROOT/partials/ibm-cos-sink-description.adoc
@@ -80,6 +80,34 @@ You can specify a fixed key name for all uploaded objects:
keyName: "data/myfile.txt"
----
+==== Deleting the local file after the upload
+
+When the body is a local file, such as the file of `file-watch-source`,
`deleteAfterWrite` deletes it from the local disk once it has been uploaded. A
body that is not a file is uploaded, and nothing is deleted. This example
uploads each new file of `/var/outbox` under its own name, and then deletes it.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:file-watch-source"
+ parameters:
+ filePath: "/var/outbox"
+ events: "CREATE"
+ steps:
+ - to:
+ uri: "kamelet:insert-header-action"
+ parameters:
+ name: "file"
+ value: "$\{header.CamelFileName}"
+ - to:
+ uri: "kamelet:ibm-cos-sink"
+ parameters:
+ bucketName: "my-bucket"
+ apiKey: "{{ibm.api.key}}"
+ serviceInstanceId: "{{ibm.service.instance.id}}"
+ endpointUrl:
"https://s3.us-south.cloud-object-storage.appdomain.cloud"
+ deleteAfterWrite: true
+----
+
==== Using multi-part upload for large files
For large files, you can enable multi-part upload:
diff --git a/docs/modules/ROOT/partials/ibm-cos-source-description.adoc
b/docs/modules/ROOT/partials/ibm-cos-source-description.adoc
index 5325719b8..4d6236fd4 100644
--- a/docs/modules/ROOT/partials/ibm-cos-source-description.adoc
+++ b/docs/modules/ROOT/partials/ibm-cos-source-description.adoc
@@ -102,6 +102,28 @@ You can filter objects using a prefix:
This consumes only objects with keys starting with `inbox/`.
+==== Reading one file
+
+To read one file of the bucket instead of all of them, set `fileName` to its
key. With `deleteAfterRead` false, the same file is read again on every poll;
with the default `deleteAfterRead`, it is read once and deleted, and each later
poll logs a warning that the key does not exist, until a file with that key is
uploaded again.
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:ibm-cos-source"
+ parameters:
+ bucketName: "my-bucket"
+ apiKey: "{{ibm.api.key}}"
+ serviceInstanceId: "{{ibm.service.instance.id}}"
+ endpointUrl: "https://s3.us-south.cloud-object-storage.appdomain.cloud"
+ fileName: "config/prices.csv"
+ deleteAfterRead: false
+ delay: 60000
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
+----
+
==== Consuming only metadata
To consume only object metadata without the body, set `includeBody` to false:
diff --git a/kamelets/google-calendar-source.kamelet.yaml
b/kamelets/google-calendar-source.kamelet.yaml
index a04b7d472..0673a7533 100644
--- a/kamelets/google-calendar-source.kamelet.yaml
+++ b/kamelets/google-calendar-source.kamelet.yaml
@@ -37,14 +37,10 @@ spec:
Output: the body is the event as JSON text, with the fields of the
Google Calendar API such as `id`, `summary`, `start` and `attendees`. The
`CamelGoogleCalendarEventId` header gives the ID of the event.
- Good to know: by default it only receives the events that have not ended
yet at the time of the poll (`consumeFromNow`). With `syncFlow` set to true,
each poll after the first asks only for the events that are new or changed
since the previous one. It logs in to Google with OAuth: an OAuth client of a
Google Cloud project, and an access token and a refresh token of a Google
account that may read the calendar.
+ Good to know: by default it only receives the events that have not ended
yet at the time of the poll (`consumeFromNow`). Each poll asks for one page of
up to `maxResults` events (10 by default), the events that changed least
recently first, so the same events can come again on every poll. With
`considerLastUpdate` set to true, each poll reads all the pages, and after the
first poll only the events that changed since the last one received. With
`syncFlow` set to true, each poll afte [...]
required:
- index
- calendarId
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
- applicationName
type: object
properties:
@@ -60,32 +56,53 @@ spec:
example: primary
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Calendar API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Calendar API is enabled. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Google account, with
access to Google Calendar. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Google account, with
access to Google Calendar. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. The service account reads the calendars that
are shared with its email address, or the calendar of the user in `delegate`.
Only used when `clientId` [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of a user of your Google Workspace,
whose calendar the service account of `serviceAccountKey` reads on their
behalf. A Workspace administrator must first allow the client ID of the service
account, with the scope of `scopes`, under Security > Access and data control >
API controls > Domain-wide delegation. Only used with `serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: The permissions to ask Google for when logging in with
`serviceAccountKey`, as URLs separated by commas, such as
`https://www.googleapis.com/auth/calendar.readonly` to only read. When it is
not set, `https://www.googleapis.com/auth/calendar` is used. Only used with
`serviceAccountKey`.
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/calendar.readonly
delay:
title: Delay
description: How long to wait between two polls of the calendar, in
milliseconds. For example, 60000 is one minute.
@@ -107,6 +124,21 @@ spec:
description: If true, only the events that have not ended yet at the
time of each poll are read. If false, past events are read too.
type: boolean
default: true
+ maxResults:
+ title: Max Results
+ description: The most events to ask Google for at once, in one page.
Each poll reads one page, or all the pages when `considerLastUpdate` is true.
When it is not set, 10 is used.
+ type: integer
+ example: 50
+ query:
+ title: Search Text
+ description: Only read the events that contain this text, as the
search box of Google Calendar finds them, for example in their title,
description, location or attendees. Do not combine it with `syncFlow`.
+ type: string
+ pattern: "^[^\\x00-\\x1f]+$"
+ example: standup
+ considerLastUpdate:
+ title: Only Changed Events
+ description: If true, each poll reads all the pages, and each poll
after the first asks only for the events that changed since the last one
received, so an event comes again only when it changes. When it is not set, it
is false. Do not combine it with `syncFlow`.
+ type: boolean
dataTypes:
out:
default: json
@@ -151,7 +183,7 @@ spec:
- "camel:kamelet"
template:
from:
- uri:
"google-calendar-stream://{{index}}?accessToken=RAW({{accessToken}})&applicationName={{applicationName}}&calendarId={{calendarId}}&clientId={{clientId}}&clientSecret=RAW({{clientSecret}})&consumeFromNow={{consumeFromNow}}&delay={{delay}}&refreshToken=RAW({{refreshToken}})&syncFlow={{syncFlow}}"
+ uri:
"google-calendar-stream://{{index}}?accessToken=RAW({{?accessToken}})&applicationName={{applicationName}}&calendarId={{calendarId}}&clientId={{?clientId}}&clientSecret=RAW({{?clientSecret}})&considerLastUpdate={{?considerLastUpdate}}&consumeFromNow={{consumeFromNow}}&delay={{delay}}&delegate=RAW({{?delegate}})&maxResults={{?maxResults}}&query=RAW({{?query}})&refreshToken=RAW({{?refreshToken}})&scopes={{?scopes}}&serviceAccountKey=RAW({{?serviceAccountKey}})&syncFlow={{syncFlow}}"
steps:
- marshal:
json: {}
diff --git a/kamelets/google-mail-source.kamelet.yaml
b/kamelets/google-mail-source.kamelet.yaml
index 6c98d60e6..633714da2 100644
--- a/kamelets/google-mail-source.kamelet.yaml
+++ b/kamelets/google-mail-source.kamelet.yaml
@@ -35,15 +35,11 @@ spec:
Use it to act on incoming email, for example to open a ticket for each
email sent to a support address, or to collect orders or invoices that arrive
by mail.
- Output: the body is the text of the email (its first part with content,
such as the plain text part) as JSON text, that is the text in double quotes
with JSON escapes. The headers give the subject
(`CamelGoogleMailStreamSubject`), the sender (`CamelGoogleMailStreamFrom`), the
recipients (`CamelGoogleMailStreamTo`, `CamelGoogleMailStreamCc`,
`CamelGoogleMailStreamBcc`) and the Gmail ID of the email (`CamelGoogleMailId`).
+ Output: the body is the text of the email (its first part with content,
such as the plain text part) as JSON text, that is the text in double quotes
with JSON escapes. The headers give the subject
(`CamelGoogleMailStreamSubject`), the sender (`CamelGoogleMailStreamFrom`), the
recipients (`CamelGoogleMailStreamTo`, `CamelGoogleMailStreamCc`,
`CamelGoogleMailStreamBcc`) and the Gmail ID of the email
(`CamelGoogleMailId`). With `raw` set to true, the body is the whole email
instead, w [...]
- Good to know: it reads the emails that match `query`, by default the
unread ones. With `markAsRead` set to false and the default query, the same
emails are read again on every poll. An email whose processing fails is marked
as unread again. It logs in to Google with OAuth: an OAuth client of a Google
Cloud project, and an access token and a refresh token of the Gmail account.
+ Good to know: it reads the emails that match `query`, by default the
unread ones. With `markAsRead` set to false and the default query, the same
emails are read again on every poll. An email whose processing fails is marked
as unread again. Each poll reads up to `maxResults` emails, 10 by default. It
logs in to Google either with OAuth (an OAuth client of a Google Cloud project,
and an access token and a refresh token of the Gmail account), or with the key
of a service account in ` [...]
required:
- index
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
- applicationName
type: object
properties:
@@ -54,32 +50,53 @@ spec:
example: support
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the Gmail
API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the Gmail
API is enabled. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Gmail account, with access
to read and change its emails. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Gmail account, with access
to read and change its emails. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Gmail account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Gmail account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. It reads the mailbox of the user in
`delegate`, and needs `scopes`. Only used when `clientSecret` is not set."
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of the user of your Google Workspace
whose mailbox the service account of `serviceAccountKey` reads on their behalf.
A Workspace administrator must first allow the client ID of the service
account, with the scopes of `scopes`, under Security > Access and data control
> API controls > Domain-wide delegation. Only used with `serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: "The permissions to ask Google for, as URLs separated by
commas. Needed with `serviceAccountKey`: without them, the login asks Google
for no permission at all. Use `https://www.googleapis.com/auth/gmail.modify`,
which marking the emails as read needs, or
`https://www.googleapis.com/auth/gmail.readonly` with `markAsRead` set to
false. Leave it unset with `clientId` and `clientSecret`; with them, setting it
makes the Kamelet fail."
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/gmail.modify
delay:
title: Delay
description: How long to wait between two polls of the mailbox, in
milliseconds. For example, 60000 is one minute.
@@ -107,6 +124,15 @@ spec:
type: string
example: "is:unread -category:(promotions OR social)"
default: "is:unread"
+ maxResults:
+ title: Max Results
+ description: The most emails to read at each poll. When it is not set,
10 is used.
+ type: integer
+ example: 50
+ raw:
+ title: Raw Email
+ description: If true, the body is the whole email, with its headers
and all its parts, such as attachments, in the format of an `.eml` file (RFC
2822), encoded in base64url, as JSON text, that is in double quotes. Remove the
quotes and decode it to get the email. The subject, sender and recipients are
then not set as headers. When it is not set, it is false, and the body is the
text of the email, without its attachments.
+ type: boolean
dataTypes:
out:
default: json
@@ -171,7 +197,7 @@ spec:
- "camel:google-mail"
template:
from:
- uri:
"google-mail-stream://{{index}}?accessToken=RAW({{accessToken}})&applicationName={{applicationName}}&clientId={{clientId}}&clientSecret=RAW({{clientSecret}})&delay={{delay}}&labels={{?labels}}&markAsRead={{markAsRead}}&query={{query}}&refreshToken=RAW({{refreshToken}})"
+ uri:
"google-mail-stream://{{index}}?accessToken=RAW({{?accessToken}})&applicationName={{applicationName}}&clientId={{?clientId}}&clientSecret=RAW({{?clientSecret}})&delay={{delay}}&delegate=RAW({{?delegate}})&labels={{?labels}}&markAsRead={{markAsRead}}&maxResults={{?maxResults}}&query={{query}}&raw={{?raw}}&refreshToken=RAW({{?refreshToken}})&scopes={{?scopes}}&serviceAccountKey=RAW({{?serviceAccountKey}})"
steps:
- marshal:
json: {}
diff --git a/kamelets/google-sheets-sink.kamelet.yaml
b/kamelets/google-sheets-sink.kamelet.yaml
index 926222458..acd60e298 100644
--- a/kamelets/google-sheets-sink.kamelet.yaml
+++ b/kamelets/google-sheets-sink.kamelet.yaml
@@ -39,13 +39,9 @@ spec:
Output: when it is used inside a route, the body afterwards is the
response of Google Sheets, which says which cells were written.
- Good to know: without `range`, only column `A` is written. With the
default `operation` of `append`, the rows are added after the last row of the
table found in `range`; with `update`, they overwrite the cells of `range`. It
logs in to Google with OAuth: an OAuth client of a Google Cloud project, and an
access token and a refresh token of the Google account that owns or may edit
the spreadsheet.
+ Good to know: without `range`, only column `A` is written. With the
default `operation` of `append`, the rows are added after the last row of the
table found in `range`; with `update`, they overwrite the cells of `range`. It
logs in to Google either with OAuth (an OAuth client of a Google Cloud project,
and an access token and a refresh token of the Google account that owns or may
edit the spreadsheet), or with the key of a service account in
`serviceAccountKey` together with `scop [...]
required:
- spreadsheetId
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
type: object
properties:
spreadsheetId:
@@ -55,32 +51,53 @@ spec:
example: 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. The service account writes to the spreadsheets
that are shared with its email address as an editor, or those of the user in
`delegate`. It needs `sc [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of a user of your Google Workspace,
whose spreadsheets the service account of `serviceAccountKey` writes to on
their behalf. A Workspace administrator must first allow the client ID of the
service account, with the scopes of `scopes`, under Security > Access and data
control > API controls > Domain-wide delegation. Only used with
`serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: "The permissions to ask Google for, as URLs separated by
commas, such as `https://www.googleapis.com/auth/spreadsheets`. Needed with
`serviceAccountKey` and with the Application Default Credentials: without them,
the login asks Google for no permission at all. Leave it unset with `clientId`
and `clientSecret`; with them, setting it makes the Kamelet fail."
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/spreadsheets
applicationName:
title: Application Name
description: A name for your application, sent to Google with each
call, such as `my-company-sheets/1.0`.
@@ -191,10 +208,13 @@ spec:
parameters:
apiName: data
methodName: "{{operation}}"
- accessToken: "RAW({{accessToken}})"
+ accessToken: "RAW({{?accessToken}})"
applicationName: "{{?applicationName}}"
- clientId: "{{clientId}}"
- clientSecret: "RAW({{clientSecret}})"
+ clientId: "{{?clientId}}"
+ clientSecret: "RAW({{?clientSecret}})"
+ delegate: "RAW({{?delegate}})"
range: "{{?range}}"
- refreshToken: "RAW({{refreshToken}})"
+ refreshToken: "RAW({{?refreshToken}})"
+ scopes: "{{?scopes}}"
+ serviceAccountKey: "RAW({{?serviceAccountKey}})"
spreadsheetId: "{{spreadsheetId}}"
diff --git a/kamelets/google-sheets-source.kamelet.yaml
b/kamelets/google-sheets-source.kamelet.yaml
index 789a167a8..648fbbbda 100644
--- a/kamelets/google-sheets-source.kamelet.yaml
+++ b/kamelets/google-sheets-source.kamelet.yaml
@@ -37,13 +37,9 @@ spec:
Output: with `range` set, the body is one row as JSON text, a JSON array
of the values of the row (or of one column, with `majorDimension` set to
`COLUMNS`), such as `["Alice","3"]`; with `splitResults` set to false, it is
the whole range as JSON text, with the rows in its `values` field. Without
`range`, the body is the description of the spreadsheet (its sheets and
properties) as JSON text, not its cells. The `CamelGoogleSheets.spreadsheetId`
and `CamelGoogleSheets.range` headers [...]
- Good to know: every poll reads the whole range again, by default every
500 milliseconds, so the same rows come again and again until `repeatCount` is
reached; it does not track which rows are new. It logs in to Google with OAuth:
an OAuth client of a Google Cloud project, and an access token and a refresh
token of a Google account that may read the spreadsheet.
+ Good to know: every poll reads the whole range again, by default every
500 milliseconds, so the same rows come again and again until `repeatCount` is
reached; it does not track which rows are new. It logs in to Google either with
OAuth (an OAuth client of a Google Cloud project, and an access token and a
refresh token of a Google account that may read the spreadsheet), or with the
key of a service account in `serviceAccountKey` together with `scopes`; share
the spreadsheet with the [...]
required:
- spreadsheetId
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
type: object
properties:
spreadsheetId:
@@ -53,32 +49,53 @@ spec:
example: 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. The service account reads the spreadsheets
that are shared with its email address, or those of the user in `delegate`. It
needs `scopes`. Only used [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of a user of your Google Workspace,
whose spreadsheets the service account of `serviceAccountKey` reads on their
behalf. A Workspace administrator must first allow the client ID of the service
account, with the scopes of `scopes`, under Security > Access and data control
> API controls > Domain-wide delegation. Only used with `serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: "The permissions to ask Google for, as URLs separated by
commas, such as `https://www.googleapis.com/auth/spreadsheets.readonly`. Needed
with `serviceAccountKey` and with the Application Default Credentials: without
them, the login asks Google for no permission at all. Leave it unset with
`clientId` and `clientSecret`; with them, setting it makes the Kamelet fail."
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/spreadsheets.readonly
delay:
title: Delay
description: How long to wait between two reads of the spreadsheet, in
milliseconds. For example, 60000 is one minute.
@@ -118,6 +135,17 @@ spec:
type: string
default: "A"
example: name,qty
+ valueRenderOption:
+ title: Value Render Option
+ description: "How the cells are read: `FORMATTED_VALUE` as the
spreadsheet shows them, as text such as `$1.50`; `UNFORMATTED_VALUE` as their
values, so a number is a number such as `1.5`; `FORMULA` gives the formula of a
cell, such as `=B2*2`, instead of its result. When it is not set,
`FORMATTED_VALUE` is used."
+ type: string
+ enum: [ "FORMATTED_VALUE", "UNFORMATTED_VALUE", "FORMULA" ]
+ example: "UNFORMATTED_VALUE"
+ maxResults:
+ title: Max Results
+ description: The most rows of each range to send, counted from the
first, or the most columns with `majorDimension` set to `COLUMNS`. For example,
1 sends only the first row. Google still returns the whole range, and the other
rows are left out. When it is not set, all the rows are sent.
+ type: integer
+ example: 100
dataTypes:
out:
default: json
@@ -186,7 +214,7 @@ spec:
- "camel:google-sheets"
template:
from:
- uri:
"google-sheets-stream://{{spreadsheetId}}?accessToken=RAW({{accessToken}})&applicationName={{?applicationName}}&clientId={{clientId}}&clientSecret=RAW({{clientSecret}})&delay={{delay}}&majorDimension={{majorDimension}}&range={{?range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}"
+ uri:
"google-sheets-stream://{{spreadsheetId}}?accessToken=RAW({{?accessToken}})&applicationName={{?applicationName}}&clientId={{?clientId}}&clientSecret=RAW({{?clientSecret}})&delay={{delay}}&delegate=RAW({{?delegate}})&majorDimension={{majorDimension}}&maxResults={{?maxResults}}&range={{?range}}&refreshToken=RAW({{?refreshToken}})&repeatCount={{?repeatCount}}&scopes={{?scopes}}&serviceAccountKey=RAW({{?serviceAccountKey}})&splitResults={{splitResults}}&valueRenderOption={{?valueR
[...]
steps:
- marshal:
json: {}
diff --git a/kamelets/google-vertexai-sink.kamelet.yaml
b/kamelets/google-vertexai-sink.kamelet.yaml
index a14706e8c..468ba1252 100644
--- a/kamelets/google-vertexai-sink.kamelet.yaml
+++ b/kamelets/google-vertexai-sink.kamelet.yaml
@@ -33,11 +33,13 @@ spec:
description: |-
Sends each message as a prompt to a generative AI model in Google Vertex
AI, such as Gemini, and replaces the body with the answer of the model.
- Use it to summarize, classify, translate or answer questions about text
as it flows, for example to summarize support tickets or extract the topic of
an email.
+ Use it to summarize, classify, translate or answer questions about text
as it flows, for example to summarize support tickets or extract the topic of
an email. To call a partner model of the Model Garden instead, such as Claude
of Anthropic, set `operation` to `rawPredict` and `publisher`.
- Input: the body is the prompt, as text; bytes and streams are read as
text. The optional `CamelGoogleVertexAIPrompt` header gives the prompt instead
of the body.
+ Input: the body is the prompt, as text; bytes and streams are read as
text. The optional `CamelGoogleVertexAIPrompt` header gives the prompt instead
of the body. With `rawPredict`, the header is not used: the body is the prompt
as text, or a JSON request in the format of the publisher, such as the Messages
API of Anthropic, which is sent as it is, with `anthropic_version` added for
Anthropic.
- Output: when it is used inside a route, the body afterwards is the
answer of the model, as text, for the default `generateText` operation. The
`CamelGoogleVertexAIFinishReason` header says why the model stopped, and the
token counts are in the `CamelGoogleVertexAIPromptTokenCount`,
`CamelGoogleVertexAICandidatesTokenCount` and
`CamelGoogleVertexAITotalTokenCount` headers.
+ Output: when it is used inside a route, the body afterwards is the
answer of the model, as text, for the default `generateText` operation; with
`jsonMode` set to true, it is JSON text. The `CamelGoogleVertexAIFinishReason`
header says why the model stopped, and the token counts are in the
`CamelGoogleVertexAIPromptTokenCount`,
`CamelGoogleVertexAICandidatesTokenCount` and
`CamelGoogleVertexAITotalTokenCount` headers. With `rawPredict` and the
publisher `anthropic`, the body afterwa [...]
+
+ Good to know: with a prompt as text and the publisher `anthropic`,
`rawPredict` also sends `temperature`, `topP` and `topK`, as 0.7, 0.95 and 40
when they are not set, and `maxOutputTokens` as `max_tokens`, 1024 when it is
not set. Send a JSON request as the body to choose the fields of the request
yourself.
required:
- projectId
- region
@@ -68,7 +70,7 @@ spec:
- urn:camel:group:credentials
operation:
title: Operation
- description: What to ask the model for. `generateText` (the default),
`generateChat` and `generateCode` answer the prompt with text, `generateImage`
gives a List of images, `generateEmbeddings` gives one vector of numbers per
text, and `generateMultimodal` sends media together with the prompt.
`rawPredict` fails, as it needs a publisher that this Kamelet cannot set.
+ description: What to ask the model for. `generateText` (the default),
`generateChat` and `generateCode` answer the prompt with text, `generateImage`
gives a List of images, `generateEmbeddings` gives one vector of numbers per
text, and `generateMultimodal` sends media together with the prompt.
`rawPredict` sends the body to a partner model of `publisher`, such as Claude.
type: string
default: generateText
enum: ["generateText", "generateChat", "generateImage",
"generateEmbeddings", "generateCode", "generateMultimodal", "rawPredict"]
@@ -92,6 +94,22 @@ spec:
description: Only choose each next word among this many most likely
ones, a whole number. When it is not set, 40 is used.
type: string
example: "40"
+ publisher:
+ title: Model Publisher
+ description: The publisher of a partner model of the Model Garden, for
the `rawPredict` operation, such as `anthropic` for Claude, `meta` for Llama or
`mistralai` for Mistral. The model of `modelId` is then called as
`projects/<projectId>/locations/<region>/publishers/<publisher>/models/<modelId>`.
Only used with `rawPredict`, which fails without it.
+ type: string
+ pattern: "^[a-z0-9-]+$"
+ example: anthropic
+ anthropicVersion:
+ title: Anthropic Version
+ description: The version of the Anthropic API to ask Claude for, sent
as `anthropic_version` in each request, unless the JSON request of the body has
one. When it is not set, `vertex-2023-10-16` is used. Only used with
`rawPredict` and the publisher `anthropic`.
+ type: string
+ pattern: "^[A-Za-z0-9._-]+$"
+ example: vertex-2023-10-16
+ jsonMode:
+ title: JSON Mode
+ description: If true, the model is asked to answer with JSON (the
response type `application/json`), so the body afterwards is JSON text. Say in
the prompt which fields you want. Only used with `generateText`,
`generateChat`, `generateCode` and `generateMultimodal`. When it is not set, it
is false.
+ type: boolean
dataTypes:
in:
headers:
@@ -117,6 +135,10 @@ spec:
title: Total Token Count
description: The number of tokens of the prompt and the answer
together.
type: integer
+ CamelGoogleVertexAIRawResponse:
+ title: Raw Response
+ description: With `rawPredict` and the publisher `anthropic`, the
whole answer of the model, as JSON text.
+ type: string
dependencies:
- "camel:kamelet"
- "camel:google-vertexai"
@@ -130,8 +152,11 @@ spec:
projectId: "{{projectId}}"
location: "{{region}}"
modelId: "{{modelId}}"
+ anthropicVersion: "{{?anthropicVersion}}"
+ jsonMode: "{{?jsonMode}}"
maxOutputTokens: "{{?maxOutputTokens}}"
operation: "{{operation}}"
+ publisher: "{{?publisher}}"
serviceAccountKey: "{{?serviceAccountKey}}"
temperature: "{{?temperature}}"
topK: "{{?topK}}"
diff --git a/kamelets/ibm-cos-sink.kamelet.yaml
b/kamelets/ibm-cos-sink.kamelet.yaml
index 9dd3b3c2f..bd344b50b 100644
--- a/kamelets/ibm-cos-sink.kamelet.yaml
+++ b/kamelets/ibm-cos-sink.kamelet.yaml
@@ -40,7 +40,7 @@ spec:
Output: the message is unchanged, with the ETag of the uploaded file in
the `CamelIBMCOSETag` header.
- Good to know: a file with the same name is overwritten. The bucket must
exist, unless `autoCreateBucket` is true. It logs in with an IBM Cloud API key.
+ Good to know: a file with the same name is overwritten. With
`deleteAfterWrite` set to true, a body that is a local file, such as the file
of `file-watch-source`, is deleted from the local disk once it has been
uploaded. The bucket must exist, unless `autoCreateBucket` is true. It logs in
with an IBM Cloud API key.
required:
- bucketName
- apiKey
@@ -87,6 +87,10 @@ spec:
description: The name of the file in the bucket, the same for every
message, so each message overwrites the previous file. When it is not set, the
name comes from the `file` or `ce-file` header, or is generated. A
`CamelIBMCOSKey` header on the message, as `ibm-cos-source` sets, is used
instead of it.
type: string
example: reports/latest.csv
+ deleteAfterWrite:
+ title: Delete Local File
+ description: If true and the body is a local file, such as the file of
`file-watch-source`, the file is deleted from the local disk once it has been
uploaded. A body that is not a file, such as text or bytes, is uploaded and
nothing is deleted. A reader that moves the file itself after the upload, such
as the Camel file component by default, then logs a warning that it cannot move
the file. When it is not set, it is false.
+ type: boolean
multiPartUpload:
title: Multi-Part Upload
description: If true, a file larger than `partSize` is uploaded in
several parts, which suits large files.
@@ -166,6 +170,7 @@ spec:
bucketName: "{{bucketName}}"
apiKey: "RAW({{apiKey}})"
autoCreateBucket: "{{autoCreateBucket}}"
+ deleteAfterWrite: "{{?deleteAfterWrite}}"
endpointUrl: "{{endpointUrl}}"
keyName: "{{?keyName}}"
location: "{{?region}}"
diff --git a/kamelets/ibm-cos-source.kamelet.yaml
b/kamelets/ibm-cos-source.kamelet.yaml
index d525a5fd1..7a47bca30 100644
--- a/kamelets/ibm-cos-source.kamelet.yaml
+++ b/kamelets/ibm-cos-source.kamelet.yaml
@@ -38,7 +38,7 @@ spec:
Output: the body is the content of the file, as a stream. The headers
give the bucket name (`CamelIBMCOSBucketName`), the key, that is the path of
the file in the bucket (`CamelIBMCOSKey`), the content type, the size and the
ETag of the file.
- Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. `moveAfterRead` moves them to another bucket instead. A file whose
processing fails is kept, and read again on the next poll. It logs in with an
IBM Cloud API key.
+ Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. `moveAfterRead` moves them to another bucket instead. To read one
file only, set `fileName`. A file whose processing fails is kept, and read
again on the next poll. It logs in with an IBM Cloud API key.
required:
- bucketName
- apiKey
@@ -105,6 +105,12 @@ spec:
description: If true, the bucket is created when it does not exist yet.
type: boolean
default: false
+ fileName:
+ title: File Name
+ description: The key of one file to read, such as `inbox/orders.csv`,
instead of all the files of the bucket; `prefix` and `delimiter` are then not
used. With `deleteAfterRead` true, the default, the file is read once and
deleted, and each later poll logs a warning that the key does not exist, until
a file with that key is uploaded again. With `deleteAfterRead` false, the same
file is read again on every poll.
+ type: string
+ pattern: "^[^\\x00-\\x1f]+$"
+ example: inbox/orders.csv
prefix:
title: Prefix
description: Only read the files whose key starts with this text, such
as a folder followed by `/`.
@@ -183,6 +189,7 @@ spec:
destinationBucketPrefix: "{{?destinationBucketPrefix}}"
destinationBucketSuffix: "{{?destinationBucketSuffix}}"
endpointUrl: "{{endpointUrl}}"
+ fileName: "{{?fileName}}"
includeBody: "{{includeBody}}"
includeFolders: "{{includeFolders}}"
location: "{{?region}}"
diff --git
a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
index 64ec3ca0a..46ee4bd73 100644
--- a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
+++ b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
@@ -885,7 +885,7 @@
"supportLevel" : "Stable",
"title" : "Google Calendar Source",
"summary" : "Receives the events of a Google Calendar, such as meetings
and appointments, one message per event, by asking the calendar for events at a
regular interval.",
- "description" : "Receives the events of a Google Calendar, such as
meetings and appointments, one message per event, by asking the calendar for
events at a regular interval.\n\nUse it to act on calendar entries, for example
to send a reminder before a meeting, copy new appointments into another system,
or notify a team when a booking is made.\n\nOutput: the body is the event as
JSON text, with the fields of the Google Calendar API such as `id`, `summary`,
`start` and `attendees`. The [...]
+ "description" : "Receives the events of a Google Calendar, such as
meetings and appointments, one message per event, by asking the calendar for
events at a regular interval.\n\nUse it to act on calendar entries, for example
to send a reminder before a meeting, copy new appointments into another system,
or notify a team when a booking is made.\n\nOutput: the body is the event as
JSON text, with the fields of the Google Calendar API such as `id`, `summary`,
`start` and `attendees`. The [...]
"group" : "Google Calendar",
"namespace" : "GCP",
"provider" : "Apache Software Foundation"
@@ -895,7 +895,7 @@
"supportLevel" : "Stable",
"title" : "Google Mail Source",
"summary" : "Receives the emails of a Gmail mailbox that match a search,
by default the unread ones, one message per email, and marks each email as read
once it has been processed.",
- "description" : "Receives the emails of a Gmail mailbox that match a
search, by default the unread ones, one message per email, and marks each email
as read once it has been processed.\n\nUse it to act on incoming email, for
example to open a ticket for each email sent to a support address, or to
collect orders or invoices that arrive by mail.\n\nOutput: the body is the text
of the email (its first part with content, such as the plain text part) as JSON
text, that is the text in doub [...]
+ "description" : "Receives the emails of a Gmail mailbox that match a
search, by default the unread ones, one message per email, and marks each email
as read once it has been processed.\n\nUse it to act on incoming email, for
example to open a ticket for each email sent to a support address, or to
collect orders or invoices that arrive by mail.\n\nOutput: the body is the text
of the email (its first part with content, such as the plain text part) as JSON
text, that is the text in doub [...]
"group" : "Google Mail",
"namespace" : "GCP",
"provider" : "Apache Software Foundation"
@@ -925,7 +925,7 @@
"supportLevel" : "Stable",
"title" : "Google Sheets Sink",
"summary" : "Writes each message as one or more rows to a Google Sheets
spreadsheet, either appended after the last row or written over a given range
of cells.",
- "description" : "Writes each message as one or more rows to a Google
Sheets spreadsheet, either appended after the last row or written over a given
range of cells.\n\nUse it to collect data in a spreadsheet that people can read
and share, such as form submissions, orders or daily figures. To read the rows
of a spreadsheet, use `google-sheets-source`.\n\nInput: the body is JSON text:
a JSON object for one row, or a JSON array of objects for several rows. The
keys are the column letter [...]
+ "description" : "Writes each message as one or more rows to a Google
Sheets spreadsheet, either appended after the last row or written over a given
range of cells.\n\nUse it to collect data in a spreadsheet that people can read
and share, such as form submissions, orders or daily figures. To read the rows
of a spreadsheet, use `google-sheets-source`.\n\nInput: the body is JSON text:
a JSON object for one row, or a JSON array of objects for several rows. The
keys are the column letter [...]
"group" : "Google Sheets",
"namespace" : "GCP",
"provider" : "Apache Software Foundation"
@@ -935,7 +935,7 @@
"supportLevel" : "Stable",
"title" : "Google Sheets Source",
"summary" : "Reads the cells of a Google Sheets spreadsheet again and
again, at a regular interval, and sends each row of the range as a message.",
- "description" : "Reads the cells of a Google Sheets spreadsheet again and
again, at a regular interval, and sends each row of the range as a
message.\n\nUse it to feed the content of a spreadsheet that people maintain,
such as a price list, a contact list or a list of tasks, to another system on a
schedule. To write rows to a spreadsheet, use `google-sheets-sink`.\n\nOutput:
with `range` set, the body is one row as JSON text, a JSON array of the values
of the row (or of one column, w [...]
+ "description" : "Reads the cells of a Google Sheets spreadsheet again and
again, at a regular interval, and sends each row of the range as a
message.\n\nUse it to feed the content of a spreadsheet that people maintain,
such as a price list, a contact list or a list of tasks, to another system on a
schedule. To write rows to a spreadsheet, use `google-sheets-sink`.\n\nOutput:
with `range` set, the body is one row as JSON text, a JSON array of the values
of the row (or of one column, w [...]
"group" : "Google Sheets",
"namespace" : "GCP",
"provider" : "Apache Software Foundation"
@@ -975,7 +975,7 @@
"supportLevel" : "Preview",
"title" : "Google Vertex AI Sink",
"summary" : "Sends each message as a prompt to a generative AI model in
Google Vertex AI, such as Gemini, and replaces the body with the answer of the
model.",
- "description" : "Sends each message as a prompt to a generative AI model
in Google Vertex AI, such as Gemini, and replaces the body with the answer of
the model.\n\nUse it to summarize, classify, translate or answer questions
about text as it flows, for example to summarize support tickets or extract the
topic of an email.\n\nInput: the body is the prompt, as text; bytes and streams
are read as text. The optional `CamelGoogleVertexAIPrompt` header gives the
prompt instead of the body [...]
+ "description" : "Sends each message as a prompt to a generative AI model
in Google Vertex AI, such as Gemini, and replaces the body with the answer of
the model.\n\nUse it to summarize, classify, translate or answer questions
about text as it flows, for example to summarize support tickets or extract the
topic of an email. To call a partner model of the Model Garden instead, such as
Claude of Anthropic, set `operation` to `rawPredict` and `publisher`.\n\nInput:
the body is the prompt [...]
"group" : "Google Vertex AI",
"namespace" : "GCP",
"provider" : "Apache Software Foundation"
@@ -1045,7 +1045,7 @@
"supportLevel" : "Preview",
"title" : "IBM Cloud Object Storage Sink",
"summary" : "Uploads each message as a file (object) to a bucket in IBM
Cloud Object Storage.",
- "description" : "Uploads each message as a file (object) to a bucket in
IBM Cloud Object Storage.\n\nUse it to store data for later use or for other
systems, such as exported reports, archived events or files received from
another system. To read the files of a bucket, use `ibm-cos-source`.\n\nInput:
the body is the content of the file, as text, bytes, a stream or a file. Turn
structured data such as a Map into JSON text first, for example with
`json-serialize-action`. The name of th [...]
+ "description" : "Uploads each message as a file (object) to a bucket in
IBM Cloud Object Storage.\n\nUse it to store data for later use or for other
systems, such as exported reports, archived events or files received from
another system. To read the files of a bucket, use `ibm-cos-source`.\n\nInput:
the body is the content of the file, as text, bytes, a stream or a file. Turn
structured data such as a Map into JSON text first, for example with
`json-serialize-action`. The name of th [...]
"group" : "IBM Cloud Object Storage",
"namespace" : "IBM",
"provider" : "Apache Software Foundation"
@@ -1055,7 +1055,7 @@
"supportLevel" : "Preview",
"title" : "IBM Cloud Object Storage Source",
"summary" : "Reads the files (objects) in a bucket of IBM Cloud Object
Storage, one message per file, and by default deletes each file once it has
been read.",
- "description" : "Reads the files (objects) in a bucket of IBM Cloud Object
Storage, one message per file, and by default deletes each file once it has
been read.\n\nUse it to process files that another system drops into a bucket,
such as uploaded documents, exported reports or log files. To upload files to a
bucket, use `ibm-cos-sink`.\n\nOutput: the body is the content of the file, as
a stream. The headers give the bucket name (`CamelIBMCOSBucketName`), the key,
that is the path of [...]
+ "description" : "Reads the files (objects) in a bucket of IBM Cloud Object
Storage, one message per file, and by default deletes each file once it has
been read.\n\nUse it to process files that another system drops into a bucket,
such as uploaded documents, exported reports or log files. To upload files to a
bucket, use `ibm-cos-sink`.\n\nOutput: the body is the content of the file, as
a stream. The headers give the bucket name (`CamelIBMCOSBucketName`), the key,
that is the path of [...]
"group" : "IBM Cloud Object Storage",
"namespace" : "IBM",
"provider" : "Apache Software Foundation"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/google-calendar-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/google-calendar-source.kamelet.yaml
index a04b7d472..0673a7533 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/google-calendar-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/google-calendar-source.kamelet.yaml
@@ -37,14 +37,10 @@ spec:
Output: the body is the event as JSON text, with the fields of the
Google Calendar API such as `id`, `summary`, `start` and `attendees`. The
`CamelGoogleCalendarEventId` header gives the ID of the event.
- Good to know: by default it only receives the events that have not ended
yet at the time of the poll (`consumeFromNow`). With `syncFlow` set to true,
each poll after the first asks only for the events that are new or changed
since the previous one. It logs in to Google with OAuth: an OAuth client of a
Google Cloud project, and an access token and a refresh token of a Google
account that may read the calendar.
+ Good to know: by default it only receives the events that have not ended
yet at the time of the poll (`consumeFromNow`). Each poll asks for one page of
up to `maxResults` events (10 by default), the events that changed least
recently first, so the same events can come again on every poll. With
`considerLastUpdate` set to true, each poll reads all the pages, and after the
first poll only the events that changed since the last one received. With
`syncFlow` set to true, each poll afte [...]
required:
- index
- calendarId
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
- applicationName
type: object
properties:
@@ -60,32 +56,53 @@ spec:
example: primary
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Calendar API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Calendar API is enabled. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Google account, with
access to Google Calendar. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Google account, with
access to Google Calendar. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. The service account reads the calendars that
are shared with its email address, or the calendar of the user in `delegate`.
Only used when `clientId` [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of a user of your Google Workspace,
whose calendar the service account of `serviceAccountKey` reads on their
behalf. A Workspace administrator must first allow the client ID of the service
account, with the scope of `scopes`, under Security > Access and data control >
API controls > Domain-wide delegation. Only used with `serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: The permissions to ask Google for when logging in with
`serviceAccountKey`, as URLs separated by commas, such as
`https://www.googleapis.com/auth/calendar.readonly` to only read. When it is
not set, `https://www.googleapis.com/auth/calendar` is used. Only used with
`serviceAccountKey`.
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/calendar.readonly
delay:
title: Delay
description: How long to wait between two polls of the calendar, in
milliseconds. For example, 60000 is one minute.
@@ -107,6 +124,21 @@ spec:
description: If true, only the events that have not ended yet at the
time of each poll are read. If false, past events are read too.
type: boolean
default: true
+ maxResults:
+ title: Max Results
+ description: The most events to ask Google for at once, in one page.
Each poll reads one page, or all the pages when `considerLastUpdate` is true.
When it is not set, 10 is used.
+ type: integer
+ example: 50
+ query:
+ title: Search Text
+ description: Only read the events that contain this text, as the
search box of Google Calendar finds them, for example in their title,
description, location or attendees. Do not combine it with `syncFlow`.
+ type: string
+ pattern: "^[^\\x00-\\x1f]+$"
+ example: standup
+ considerLastUpdate:
+ title: Only Changed Events
+ description: If true, each poll reads all the pages, and each poll
after the first asks only for the events that changed since the last one
received, so an event comes again only when it changes. When it is not set, it
is false. Do not combine it with `syncFlow`.
+ type: boolean
dataTypes:
out:
default: json
@@ -151,7 +183,7 @@ spec:
- "camel:kamelet"
template:
from:
- uri:
"google-calendar-stream://{{index}}?accessToken=RAW({{accessToken}})&applicationName={{applicationName}}&calendarId={{calendarId}}&clientId={{clientId}}&clientSecret=RAW({{clientSecret}})&consumeFromNow={{consumeFromNow}}&delay={{delay}}&refreshToken=RAW({{refreshToken}})&syncFlow={{syncFlow}}"
+ uri:
"google-calendar-stream://{{index}}?accessToken=RAW({{?accessToken}})&applicationName={{applicationName}}&calendarId={{calendarId}}&clientId={{?clientId}}&clientSecret=RAW({{?clientSecret}})&considerLastUpdate={{?considerLastUpdate}}&consumeFromNow={{consumeFromNow}}&delay={{delay}}&delegate=RAW({{?delegate}})&maxResults={{?maxResults}}&query=RAW({{?query}})&refreshToken=RAW({{?refreshToken}})&scopes={{?scopes}}&serviceAccountKey=RAW({{?serviceAccountKey}})&syncFlow={{syncFlow}}"
steps:
- marshal:
json: {}
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/google-mail-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/google-mail-source.kamelet.yaml
index 6c98d60e6..633714da2 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/google-mail-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/google-mail-source.kamelet.yaml
@@ -35,15 +35,11 @@ spec:
Use it to act on incoming email, for example to open a ticket for each
email sent to a support address, or to collect orders or invoices that arrive
by mail.
- Output: the body is the text of the email (its first part with content,
such as the plain text part) as JSON text, that is the text in double quotes
with JSON escapes. The headers give the subject
(`CamelGoogleMailStreamSubject`), the sender (`CamelGoogleMailStreamFrom`), the
recipients (`CamelGoogleMailStreamTo`, `CamelGoogleMailStreamCc`,
`CamelGoogleMailStreamBcc`) and the Gmail ID of the email (`CamelGoogleMailId`).
+ Output: the body is the text of the email (its first part with content,
such as the plain text part) as JSON text, that is the text in double quotes
with JSON escapes. The headers give the subject
(`CamelGoogleMailStreamSubject`), the sender (`CamelGoogleMailStreamFrom`), the
recipients (`CamelGoogleMailStreamTo`, `CamelGoogleMailStreamCc`,
`CamelGoogleMailStreamBcc`) and the Gmail ID of the email
(`CamelGoogleMailId`). With `raw` set to true, the body is the whole email
instead, w [...]
- Good to know: it reads the emails that match `query`, by default the
unread ones. With `markAsRead` set to false and the default query, the same
emails are read again on every poll. An email whose processing fails is marked
as unread again. It logs in to Google with OAuth: an OAuth client of a Google
Cloud project, and an access token and a refresh token of the Gmail account.
+ Good to know: it reads the emails that match `query`, by default the
unread ones. With `markAsRead` set to false and the default query, the same
emails are read again on every poll. An email whose processing fails is marked
as unread again. Each poll reads up to `maxResults` emails, 10 by default. It
logs in to Google either with OAuth (an OAuth client of a Google Cloud project,
and an access token and a refresh token of the Gmail account), or with the key
of a service account in ` [...]
required:
- index
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
- applicationName
type: object
properties:
@@ -54,32 +50,53 @@ spec:
example: support
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the Gmail
API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the Gmail
API is enabled. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Gmail account, with access
to read and change its emails. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Gmail account, with access
to read and change its emails. It expires after about an hour; the Kamelet then
gets a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Gmail account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Gmail account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. It reads the mailbox of the user in
`delegate`, and needs `scopes`. Only used when `clientSecret` is not set."
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of the user of your Google Workspace
whose mailbox the service account of `serviceAccountKey` reads on their behalf.
A Workspace administrator must first allow the client ID of the service
account, with the scopes of `scopes`, under Security > Access and data control
> API controls > Domain-wide delegation. Only used with `serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: "The permissions to ask Google for, as URLs separated by
commas. Needed with `serviceAccountKey`: without them, the login asks Google
for no permission at all. Use `https://www.googleapis.com/auth/gmail.modify`,
which marking the emails as read needs, or
`https://www.googleapis.com/auth/gmail.readonly` with `markAsRead` set to
false. Leave it unset with `clientId` and `clientSecret`; with them, setting it
makes the Kamelet fail."
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/gmail.modify
delay:
title: Delay
description: How long to wait between two polls of the mailbox, in
milliseconds. For example, 60000 is one minute.
@@ -107,6 +124,15 @@ spec:
type: string
example: "is:unread -category:(promotions OR social)"
default: "is:unread"
+ maxResults:
+ title: Max Results
+ description: The most emails to read at each poll. When it is not set,
10 is used.
+ type: integer
+ example: 50
+ raw:
+ title: Raw Email
+ description: If true, the body is the whole email, with its headers
and all its parts, such as attachments, in the format of an `.eml` file (RFC
2822), encoded in base64url, as JSON text, that is in double quotes. Remove the
quotes and decode it to get the email. The subject, sender and recipients are
then not set as headers. When it is not set, it is false, and the body is the
text of the email, without its attachments.
+ type: boolean
dataTypes:
out:
default: json
@@ -171,7 +197,7 @@ spec:
- "camel:google-mail"
template:
from:
- uri:
"google-mail-stream://{{index}}?accessToken=RAW({{accessToken}})&applicationName={{applicationName}}&clientId={{clientId}}&clientSecret=RAW({{clientSecret}})&delay={{delay}}&labels={{?labels}}&markAsRead={{markAsRead}}&query={{query}}&refreshToken=RAW({{refreshToken}})"
+ uri:
"google-mail-stream://{{index}}?accessToken=RAW({{?accessToken}})&applicationName={{applicationName}}&clientId={{?clientId}}&clientSecret=RAW({{?clientSecret}})&delay={{delay}}&delegate=RAW({{?delegate}})&labels={{?labels}}&markAsRead={{markAsRead}}&maxResults={{?maxResults}}&query={{query}}&raw={{?raw}}&refreshToken=RAW({{?refreshToken}})&scopes={{?scopes}}&serviceAccountKey=RAW({{?serviceAccountKey}})"
steps:
- marshal:
json: {}
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/google-sheets-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/google-sheets-sink.kamelet.yaml
index 926222458..acd60e298 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/google-sheets-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/google-sheets-sink.kamelet.yaml
@@ -39,13 +39,9 @@ spec:
Output: when it is used inside a route, the body afterwards is the
response of Google Sheets, which says which cells were written.
- Good to know: without `range`, only column `A` is written. With the
default `operation` of `append`, the rows are added after the last row of the
table found in `range`; with `update`, they overwrite the cells of `range`. It
logs in to Google with OAuth: an OAuth client of a Google Cloud project, and an
access token and a refresh token of the Google account that owns or may edit
the spreadsheet.
+ Good to know: without `range`, only column `A` is written. With the
default `operation` of `append`, the rows are added after the last row of the
table found in `range`; with `update`, they overwrite the cells of `range`. It
logs in to Google either with OAuth (an OAuth client of a Google Cloud project,
and an access token and a refresh token of the Google account that owns or may
edit the spreadsheet), or with the key of a service account in
`serviceAccountKey` together with `scop [...]
required:
- spreadsheetId
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
type: object
properties:
spreadsheetId:
@@ -55,32 +51,53 @@ spec:
example: 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. The service account writes to the spreadsheets
that are shared with its email address as an editor, or those of the user in
`delegate`. It needs `sc [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of a user of your Google Workspace,
whose spreadsheets the service account of `serviceAccountKey` writes to on
their behalf. A Workspace administrator must first allow the client ID of the
service account, with the scopes of `scopes`, under Security > Access and data
control > API controls > Domain-wide delegation. Only used with
`serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: "The permissions to ask Google for, as URLs separated by
commas, such as `https://www.googleapis.com/auth/spreadsheets`. Needed with
`serviceAccountKey` and with the Application Default Credentials: without them,
the login asks Google for no permission at all. Leave it unset with `clientId`
and `clientSecret`; with them, setting it makes the Kamelet fail."
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/spreadsheets
applicationName:
title: Application Name
description: A name for your application, sent to Google with each
call, such as `my-company-sheets/1.0`.
@@ -191,10 +208,13 @@ spec:
parameters:
apiName: data
methodName: "{{operation}}"
- accessToken: "RAW({{accessToken}})"
+ accessToken: "RAW({{?accessToken}})"
applicationName: "{{?applicationName}}"
- clientId: "{{clientId}}"
- clientSecret: "RAW({{clientSecret}})"
+ clientId: "{{?clientId}}"
+ clientSecret: "RAW({{?clientSecret}})"
+ delegate: "RAW({{?delegate}})"
range: "{{?range}}"
- refreshToken: "RAW({{refreshToken}})"
+ refreshToken: "RAW({{?refreshToken}})"
+ scopes: "{{?scopes}}"
+ serviceAccountKey: "RAW({{?serviceAccountKey}})"
spreadsheetId: "{{spreadsheetId}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/google-sheets-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/google-sheets-source.kamelet.yaml
index 789a167a8..648fbbbda 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/google-sheets-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/google-sheets-source.kamelet.yaml
@@ -37,13 +37,9 @@ spec:
Output: with `range` set, the body is one row as JSON text, a JSON array
of the values of the row (or of one column, with `majorDimension` set to
`COLUMNS`), such as `["Alice","3"]`; with `splitResults` set to false, it is
the whole range as JSON text, with the rows in its `values` field. Without
`range`, the body is the description of the spreadsheet (its sheets and
properties) as JSON text, not its cells. The `CamelGoogleSheets.spreadsheetId`
and `CamelGoogleSheets.range` headers [...]
- Good to know: every poll reads the whole range again, by default every
500 milliseconds, so the same rows come again and again until `repeatCount` is
reached; it does not track which rows are new. It logs in to Google with OAuth:
an OAuth client of a Google Cloud project, and an access token and a refresh
token of a Google account that may read the spreadsheet.
+ Good to know: every poll reads the whole range again, by default every
500 milliseconds, so the same rows come again and again until `repeatCount` is
reached; it does not track which rows are new. It logs in to Google either with
OAuth (an OAuth client of a Google Cloud project, and an access token and a
refresh token of a Google account that may read the spreadsheet), or with the
key of a service account in `serviceAccountKey` together with `scopes`; share
the spreadsheet with the [...]
required:
- spreadsheetId
- - clientId
- - accessToken
- - refreshToken
- - clientSecret
type: object
properties:
spreadsheetId:
@@ -53,32 +49,53 @@ spec:
example: 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms
clientId:
title: Client ID
- description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled.
+ description: The client ID of an OAuth client, created in the Google
Cloud console under APIs & Services > Credentials, in a project where the
Google Sheets API is enabled. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
clientSecret:
title: Client Secret
- description: The client secret of the OAuth client of `clientId`.
+ description: The client secret of the OAuth client of `clientId`. Not
needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
accessToken:
title: Access Token
- description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`.
+ description: An OAuth 2 access token of the Google account, with
access to Google Sheets. It expires after about an hour; the Kamelet then gets
a new one with `refreshToken`. Not needed when logging in with
`serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
refreshToken:
title: Refresh Token
- description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground.
+ description: An OAuth 2 refresh token of the Google account, which the
Kamelet uses to get a new access token when the current one expires. You get it
together with the access token, for example from the Google OAuth 2.0
Playground. Not needed when logging in with `serviceAccountKey`.
type: string
format: password
x-descriptors:
- urn:camel:group:credentials
+ serviceAccountKey:
+ title: Service Account Key
+ description: "The JSON key of a Google service account, to log in with
instead of OAuth: `file:` followed by the path of the key file, `classpath:`
followed by a resource of the application, or `base64:` followed by the Base64
encoding of the key. Create the key in the Google Cloud console under IAM &
Admin > Service Accounts > Keys. The service account reads the spreadsheets
that are shared with its email address, or those of the user in `delegate`. It
needs `scopes`. Only used [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^(file|classpath|base64):.+"
+ example: file:/etc/google/service-account.json
+ delegate:
+ title: Delegate User
+ description: The email address of a user of your Google Workspace,
whose spreadsheets the service account of `serviceAccountKey` reads on their
behalf. A Workspace administrator must first allow the client ID of the service
account, with the scopes of `scopes`, under Security > Access and data control
> API controls > Domain-wide delegation. Only used with `serviceAccountKey`.
+ type: string
+ pattern: "^[^@\\s]+@[^@\\s]+$"
+ example: [email protected]
+ scopes:
+ title: Scopes
+ description: "The permissions to ask Google for, as URLs separated by
commas, such as `https://www.googleapis.com/auth/spreadsheets.readonly`. Needed
with `serviceAccountKey` and with the Application Default Credentials: without
them, the login asks Google for no permission at all. Leave it unset with
`clientId` and `clientSecret`; with them, setting it makes the Kamelet fail."
+ type: string
+ pattern: "^https://[^,\\s]+(,https://[^,\\s]+)*$"
+ example: https://www.googleapis.com/auth/spreadsheets.readonly
delay:
title: Delay
description: How long to wait between two reads of the spreadsheet, in
milliseconds. For example, 60000 is one minute.
@@ -118,6 +135,17 @@ spec:
type: string
default: "A"
example: name,qty
+ valueRenderOption:
+ title: Value Render Option
+ description: "How the cells are read: `FORMATTED_VALUE` as the
spreadsheet shows them, as text such as `$1.50`; `UNFORMATTED_VALUE` as their
values, so a number is a number such as `1.5`; `FORMULA` gives the formula of a
cell, such as `=B2*2`, instead of its result. When it is not set,
`FORMATTED_VALUE` is used."
+ type: string
+ enum: [ "FORMATTED_VALUE", "UNFORMATTED_VALUE", "FORMULA" ]
+ example: "UNFORMATTED_VALUE"
+ maxResults:
+ title: Max Results
+ description: The most rows of each range to send, counted from the
first, or the most columns with `majorDimension` set to `COLUMNS`. For example,
1 sends only the first row. Google still returns the whole range, and the other
rows are left out. When it is not set, all the rows are sent.
+ type: integer
+ example: 100
dataTypes:
out:
default: json
@@ -186,7 +214,7 @@ spec:
- "camel:google-sheets"
template:
from:
- uri:
"google-sheets-stream://{{spreadsheetId}}?accessToken=RAW({{accessToken}})&applicationName={{?applicationName}}&clientId={{clientId}}&clientSecret=RAW({{clientSecret}})&delay={{delay}}&majorDimension={{majorDimension}}&range={{?range}}&refreshToken=RAW({{refreshToken}})&repeatCount={{?repeatCount}}&splitResults={{splitResults}}"
+ uri:
"google-sheets-stream://{{spreadsheetId}}?accessToken=RAW({{?accessToken}})&applicationName={{?applicationName}}&clientId={{?clientId}}&clientSecret=RAW({{?clientSecret}})&delay={{delay}}&delegate=RAW({{?delegate}})&majorDimension={{majorDimension}}&maxResults={{?maxResults}}&range={{?range}}&refreshToken=RAW({{?refreshToken}})&repeatCount={{?repeatCount}}&scopes={{?scopes}}&serviceAccountKey=RAW({{?serviceAccountKey}})&splitResults={{splitResults}}&valueRenderOption={{?valueR
[...]
steps:
- marshal:
json: {}
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/google-vertexai-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/google-vertexai-sink.kamelet.yaml
index a14706e8c..468ba1252 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/google-vertexai-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/google-vertexai-sink.kamelet.yaml
@@ -33,11 +33,13 @@ spec:
description: |-
Sends each message as a prompt to a generative AI model in Google Vertex
AI, such as Gemini, and replaces the body with the answer of the model.
- Use it to summarize, classify, translate or answer questions about text
as it flows, for example to summarize support tickets or extract the topic of
an email.
+ Use it to summarize, classify, translate or answer questions about text
as it flows, for example to summarize support tickets or extract the topic of
an email. To call a partner model of the Model Garden instead, such as Claude
of Anthropic, set `operation` to `rawPredict` and `publisher`.
- Input: the body is the prompt, as text; bytes and streams are read as
text. The optional `CamelGoogleVertexAIPrompt` header gives the prompt instead
of the body.
+ Input: the body is the prompt, as text; bytes and streams are read as
text. The optional `CamelGoogleVertexAIPrompt` header gives the prompt instead
of the body. With `rawPredict`, the header is not used: the body is the prompt
as text, or a JSON request in the format of the publisher, such as the Messages
API of Anthropic, which is sent as it is, with `anthropic_version` added for
Anthropic.
- Output: when it is used inside a route, the body afterwards is the
answer of the model, as text, for the default `generateText` operation. The
`CamelGoogleVertexAIFinishReason` header says why the model stopped, and the
token counts are in the `CamelGoogleVertexAIPromptTokenCount`,
`CamelGoogleVertexAICandidatesTokenCount` and
`CamelGoogleVertexAITotalTokenCount` headers.
+ Output: when it is used inside a route, the body afterwards is the
answer of the model, as text, for the default `generateText` operation; with
`jsonMode` set to true, it is JSON text. The `CamelGoogleVertexAIFinishReason`
header says why the model stopped, and the token counts are in the
`CamelGoogleVertexAIPromptTokenCount`,
`CamelGoogleVertexAICandidatesTokenCount` and
`CamelGoogleVertexAITotalTokenCount` headers. With `rawPredict` and the
publisher `anthropic`, the body afterwa [...]
+
+ Good to know: with a prompt as text and the publisher `anthropic`,
`rawPredict` also sends `temperature`, `topP` and `topK`, as 0.7, 0.95 and 40
when they are not set, and `maxOutputTokens` as `max_tokens`, 1024 when it is
not set. Send a JSON request as the body to choose the fields of the request
yourself.
required:
- projectId
- region
@@ -68,7 +70,7 @@ spec:
- urn:camel:group:credentials
operation:
title: Operation
- description: What to ask the model for. `generateText` (the default),
`generateChat` and `generateCode` answer the prompt with text, `generateImage`
gives a List of images, `generateEmbeddings` gives one vector of numbers per
text, and `generateMultimodal` sends media together with the prompt.
`rawPredict` fails, as it needs a publisher that this Kamelet cannot set.
+ description: What to ask the model for. `generateText` (the default),
`generateChat` and `generateCode` answer the prompt with text, `generateImage`
gives a List of images, `generateEmbeddings` gives one vector of numbers per
text, and `generateMultimodal` sends media together with the prompt.
`rawPredict` sends the body to a partner model of `publisher`, such as Claude.
type: string
default: generateText
enum: ["generateText", "generateChat", "generateImage",
"generateEmbeddings", "generateCode", "generateMultimodal", "rawPredict"]
@@ -92,6 +94,22 @@ spec:
description: Only choose each next word among this many most likely
ones, a whole number. When it is not set, 40 is used.
type: string
example: "40"
+ publisher:
+ title: Model Publisher
+ description: The publisher of a partner model of the Model Garden, for
the `rawPredict` operation, such as `anthropic` for Claude, `meta` for Llama or
`mistralai` for Mistral. The model of `modelId` is then called as
`projects/<projectId>/locations/<region>/publishers/<publisher>/models/<modelId>`.
Only used with `rawPredict`, which fails without it.
+ type: string
+ pattern: "^[a-z0-9-]+$"
+ example: anthropic
+ anthropicVersion:
+ title: Anthropic Version
+ description: The version of the Anthropic API to ask Claude for, sent
as `anthropic_version` in each request, unless the JSON request of the body has
one. When it is not set, `vertex-2023-10-16` is used. Only used with
`rawPredict` and the publisher `anthropic`.
+ type: string
+ pattern: "^[A-Za-z0-9._-]+$"
+ example: vertex-2023-10-16
+ jsonMode:
+ title: JSON Mode
+ description: If true, the model is asked to answer with JSON (the
response type `application/json`), so the body afterwards is JSON text. Say in
the prompt which fields you want. Only used with `generateText`,
`generateChat`, `generateCode` and `generateMultimodal`. When it is not set, it
is false.
+ type: boolean
dataTypes:
in:
headers:
@@ -117,6 +135,10 @@ spec:
title: Total Token Count
description: The number of tokens of the prompt and the answer
together.
type: integer
+ CamelGoogleVertexAIRawResponse:
+ title: Raw Response
+ description: With `rawPredict` and the publisher `anthropic`, the
whole answer of the model, as JSON text.
+ type: string
dependencies:
- "camel:kamelet"
- "camel:google-vertexai"
@@ -130,8 +152,11 @@ spec:
projectId: "{{projectId}}"
location: "{{region}}"
modelId: "{{modelId}}"
+ anthropicVersion: "{{?anthropicVersion}}"
+ jsonMode: "{{?jsonMode}}"
maxOutputTokens: "{{?maxOutputTokens}}"
operation: "{{operation}}"
+ publisher: "{{?publisher}}"
serviceAccountKey: "{{?serviceAccountKey}}"
temperature: "{{?temperature}}"
topK: "{{?topK}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-sink.kamelet.yaml
index 9dd3b3c2f..bd344b50b 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-sink.kamelet.yaml
@@ -40,7 +40,7 @@ spec:
Output: the message is unchanged, with the ETag of the uploaded file in
the `CamelIBMCOSETag` header.
- Good to know: a file with the same name is overwritten. The bucket must
exist, unless `autoCreateBucket` is true. It logs in with an IBM Cloud API key.
+ Good to know: a file with the same name is overwritten. With
`deleteAfterWrite` set to true, a body that is a local file, such as the file
of `file-watch-source`, is deleted from the local disk once it has been
uploaded. The bucket must exist, unless `autoCreateBucket` is true. It logs in
with an IBM Cloud API key.
required:
- bucketName
- apiKey
@@ -87,6 +87,10 @@ spec:
description: The name of the file in the bucket, the same for every
message, so each message overwrites the previous file. When it is not set, the
name comes from the `file` or `ce-file` header, or is generated. A
`CamelIBMCOSKey` header on the message, as `ibm-cos-source` sets, is used
instead of it.
type: string
example: reports/latest.csv
+ deleteAfterWrite:
+ title: Delete Local File
+ description: If true and the body is a local file, such as the file of
`file-watch-source`, the file is deleted from the local disk once it has been
uploaded. A body that is not a file, such as text or bytes, is uploaded and
nothing is deleted. A reader that moves the file itself after the upload, such
as the Camel file component by default, then logs a warning that it cannot move
the file. When it is not set, it is false.
+ type: boolean
multiPartUpload:
title: Multi-Part Upload
description: If true, a file larger than `partSize` is uploaded in
several parts, which suits large files.
@@ -166,6 +170,7 @@ spec:
bucketName: "{{bucketName}}"
apiKey: "RAW({{apiKey}})"
autoCreateBucket: "{{autoCreateBucket}}"
+ deleteAfterWrite: "{{?deleteAfterWrite}}"
endpointUrl: "{{endpointUrl}}"
keyName: "{{?keyName}}"
location: "{{?region}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-source.kamelet.yaml
index d525a5fd1..7a47bca30 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/ibm-cos-source.kamelet.yaml
@@ -38,7 +38,7 @@ spec:
Output: the body is the content of the file, as a stream. The headers
give the bucket name (`CamelIBMCOSBucketName`), the key, that is the path of
the file in the bucket (`CamelIBMCOSKey`), the content type, the size and the
ETag of the file.
- Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. `moveAfterRead` moves them to another bucket instead. A file whose
processing fails is kept, and read again on the next poll. It logs in with an
IBM Cloud API key.
+ Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. `moveAfterRead` moves them to another bucket instead. To read one
file only, set `fileName`. A file whose processing fails is kept, and read
again on the next poll. It logs in with an IBM Cloud API key.
required:
- bucketName
- apiKey
@@ -105,6 +105,12 @@ spec:
description: If true, the bucket is created when it does not exist yet.
type: boolean
default: false
+ fileName:
+ title: File Name
+ description: The key of one file to read, such as `inbox/orders.csv`,
instead of all the files of the bucket; `prefix` and `delimiter` are then not
used. With `deleteAfterRead` true, the default, the file is read once and
deleted, and each later poll logs a warning that the key does not exist, until
a file with that key is uploaded again. With `deleteAfterRead` false, the same
file is read again on every poll.
+ type: string
+ pattern: "^[^\\x00-\\x1f]+$"
+ example: inbox/orders.csv
prefix:
title: Prefix
description: Only read the files whose key starts with this text, such
as a folder followed by `/`.
@@ -183,6 +189,7 @@ spec:
destinationBucketPrefix: "{{?destinationBucketPrefix}}"
destinationBucketSuffix: "{{?destinationBucketSuffix}}"
endpointUrl: "{{endpointUrl}}"
+ fileName: "{{?fileName}}"
includeBody: "{{includeBody}}"
includeFolders: "{{includeFolders}}"
location: "{{?region}}"