davsclaus opened a new issue, #3170: URL: https://github.com/apache/camel-kamelets/issues/3170
While documenting every Kamelet (#3140, #3141–#3155), I listed the options of the Camel components that people commonly need but that the Kamelets do not expose. Some gaps make a Kamelet unusable in a common setup; the rest are features the component has and the Kamelet hides. The Kamelets removed or moved by #3165, #3166 and #3167 are left out. ## Unusable in a common setup (to fix first) | Kamelet | Gap | |---|---| | `redis-sink`, `redis-source` | No password, TLS or database number: the only way to set them is a `connectionFactory` or `redisTemplate` bean, which a Kamelet cannot take. A Redis server with a password cannot be used. | | `mail-sink` | No `additionalJavaMailProperties`, so STARTTLS on port 587, which most providers use for SMTP, is not possible. | | `postgresql-*`, `mysql-*`, `mariadb-*` | No extra JDBC URL parameters, so `sslmode` or `useSSL` cannot be set, and a database that requires TLS cannot be used. | | `solr-*` | No HTTPS (`enableSSL`, `certificatePath`). | | `ceph-source`, `ceph-sink` | No `forcePathStyle`, so a gateway on `localhost` or on an IP address does not work. | | `jms-*-source` | No durable subscription (`subscriptionDurable`, `durableSubscriptionName`, `clientId`) and no `selector`: a topic subscriber loses the messages sent while it is down. | | `ftp-source`, `ftps-source`, `sftp-source` | No `idempotentRepository`: with `noop` or without `delete`, every file is read again after a restart. | | `aws-kinesis-source` | No `iteratorType`: it reads the whole stream again on every start. | ## AWS - All AWS Kamelets: `proxyHost`, `proxyPort`, `proxyProtocol`. - `aws-s3-sink`: `storageClass`; `useAwsKMS` and `awsKMSKeyId`; `multiPartUpload` and `partSize` (for objects over 5 GB); `conditionalWritesEnabled`. `aws-s3-streaming-upload-sink`: `storageClass`, `useAwsKMS`, `awsKMSKeyId`. - `aws-s3-event-based-source`: default, profile and session credentials (IAM roles on EKS and ECS); `maxMessagesPerPoll`, `waitTimeSeconds`, `visibilityTimeout`. - `aws-sqs-source`: `messageAttributeNames` and `attributeNames` (the attributes are always empty today); `concurrentConsumers`; `extendMessageVisibility`; `queueOwnerAWSAccountId`. - `aws-sqs-*-sink`: `queueUrl` (to avoid ListQueues); `delaySeconds`; `queueOwnerAWSAccountId`; `messageHeaderExceededLimit`; `serverSideEncryption` and `kmsMasterKeyId`. - `aws-sns-*-sink`: the subject as an option; `messageStructure`; `serverSideEncryption` and `kmsMasterKeyId`. - `aws-ses-sink`: to, subject and reply-to addresses as options; cc and bcc; `configurationSet`; `returnPath`; endpoint override. - `aws-kinesis-source`: `sequenceNumber`, `messageTimestamp`; `maxResultsPerRequest`; `shardId`; `shardClosed`; `applicationName`. - `aws-cloudwatch-sink`: the metric name, value and unit as options. - `aws-cloudtrail-source`: `lookBackWindow`, `lookBackInterval` (Camel 4.23 and later). - `aws-ddb-sink`: `consistentRead`; no read operations (GetItem, Query, Scan). `aws-ddb-streams-source`: `maxResultsPerRequest`. - `aws-lambda-sink`: `operation`; asynchronous invocation. `aws-ec2-sink`: `operation` (status only today). - `aws-secrets-manager-sink`: `operation` (update, get); `binaryPayload`. - `aws-timestream-query-sink`: paging. `aws-translate-action`: `autodetectSourceLanguage`, terminology. `aws-polly-action`: `lexiconNames`. - `aws-bedrock-text-sink`: `operation=converse`; `streamOutputMode`; `guardrailIdentifier` and `guardrailVersion`. `aws-bedrock-agent-runtime-sink`: `operation` (retrieve, invokeAgent); the number of results and the search type. - `aws-redshift-*`: extra JDBC parameters (`sslmode`, IAM); pool settings. ## Files, mail and HTTP - `file-watch-source`: `antInclude`; `recursive` (always on today). - `ftp-source`, `ftps-source`, `sftp-source`: `include`, `antInclude`, `exclude`; `move`, `moveFailed`, `preMove`; `readLock`, `readLockMinAge`; `delay`, `maxMessagesPerPoll`; `sortBy`; `streamDownload`; `connectTimeout`, `timeout`. `ftps-*`: `implicit` (port 990); a trust store. `sftp-*`: the private key as bytes; proxy; `serverAliveInterval`. - `ftp-sink`, `ftps-sink`, `sftp-sink`: `tempFileName` or `tempPrefix` (no half-written files); `doneFileName`; a file name expression; `chmod` (sftp). - `scp-sink`: `directoryName`; `knownHostsFile`; `chmod`; the private key as bytes. - `ssh-*`: key login (`certResource`, `keyPairProvider`; only a password today); `failOnUnknownHost`; `timeout`. - `mail-imap-source`, `ms-exchange-*`: `folderName`; `delete`, `moveTo`, `copyTo`; `unseen`; `searchTerm`; `maxMessagesPerPoll`; a trust store; `skipFailedMessage`; `peek`. - `mail-sink`: cc, bcc and reply-to as options; `contentType`; a trust store. - `http-source`, `http-sink`: proxy; a trust store; `throwExceptionOnFailure` or `okStatusCodeRange`; `followRedirects`; `authBearerToken` (a static API token); the method of the source; NTLM domain and host. - `rest-openapi-sink`: `consumes`, `produces`; authentication. ## Azure - `azure-storage-blob-source`: `prefix`, `regex`; `moveAfterRead` with `destinationContainer`; `maxResultsPerPage`; `sasToken`. `azure-storage-blob-sink`, `azure-storage-blob-append-sink`: `sasToken`; `blobType`, `createPageBlob`. - `azure-storage-blob-event-based-source`: `serviceBusType` (topics); `maxConcurrentCalls`. - `azure-storage-queue-sink`: `createQueue`; `visibilityTimeout`. `azure-storage-queue-source`: `visibilityTimeout`; `delay`. - `azure-storage-datalake-source`: path or directory name, `recursive`; `regex` or file name; no delete. `azure-storage-datalake-sink`: `directoryName`; account key or SAS signature. - `azure-storage-files-source`: `maxMessagesPerPoll`; `include`, `antInclude`; `move`, `noop`, idempotent. `azure-storage-files-sink`: `fileExist`. - `azure-eventhubs-source`: `consumerGroupName` (only `$Default` today); `eventPosition`; connection string; `checkpointBatchSize`, `checkpointBatchTimeout`. `azure-eventhubs-sink`: `partitionKey`; connection string. - `azure-servicebus-source`: `sessionEnabled`; `subQueue` (dead letter); `maxConcurrentCalls`; `enableDeadLettering`; `fullyQualifiedNamespace`. `azure-servicebus-sink`: `sessionId`; `binary`; `scheduledEnqueueTime`; `fullyQualifiedNamespace`. - `azure-cosmosdb-sink`: `operation=upsertItem`; `createContainerIfNotExists`, `createDatabaseIfNotExists`; `containerPartitionKeyPath`. `azure-cosmosdb-source`: change feed processor options; `hostName`. - `azure-functions-sink`: the key goes only in `?code=`, not in the `x-functions-key` header. ## Databases - SQL sources: `maxMessagesPerPoll`; `onConsumeFailed`; `onConsumeBatchComplete`; `transacted`; `initialDelay`; `routeEmptyResultSet`; `useIterator=false` or `outputType`. `oracle-database-source` has no `delay` (fixed at 500 ms). - SQL sinks: `batch` (a JSON array of rows). - All SQL Kamelets: pool settings (`maxTotal`, `maxIdle`, `maxWaitMillis`). Snowflake: warehouse, schema, role, key-pair login. - `cassandra-sink`: datacenter (fixed to `datacenter1`); TLS; `loadBalancingPolicyClass`; `clusterName`. - `couchbase-sink`: scope and collection; key; `operation` (delete); `persistTo`, `replicateTo`; timeouts. - `mongodb-*`: `operation` (save, update, upsert); `fullDocument=updateLookup` (updates give `null` today); `authSource`; a connection string (`mongodb+srv` and Atlas); `readPreference`; `persistentId`; `tailTrackDb`, `tailTrackCollection`; `changeStreamToken`. ## Messaging - `jms-*-source`: `concurrentConsumers`; `transacted`, `acknowledgementModeName`; `subscriptionShared`. `jms-*-sink`: `deliveryPersistent`, `timeToLive`, `priority`; `jmsMessageType`; `transacted`. `jms-amqp-10-*`: `username` and `password`. - `jms-ibm-mq-*`: TLS key and trust stores; a list of connections for multi-instance queue managers. - `mqtt-*`, `mqtt5-*`: `qos`; `retained`; `cleanSession`, `cleanStart`, `sessionExpiryInterval`; `clientId` on the sinks; TLS properties; `automaticReconnect`. - `nats-*`: TLS; `credentialsFilePath`; `queueName`; `durableName`; `ackPolicy`, `maxDeliver`; `requestTimeout`. - `pulsar-*`: `ackTimeoutMillis`; the key from a plain header; a trust store. - `smpp-*`: `registeredDelivery`; source and destination TON and NPI; `dataCoding`, `alphabet`; `lazySessionCreation`; `enquireLinkTimer`. - `rabbitmq-source`: `exchangeType`; `concurrentConsumers`; `prefetchCount`; `acknowledgeMode`; retry; dead letter. `rabbitmq-sink`: `allowNullBody`; content type. Both: a trust store. ## Other - `exec-sink`: arguments (none can be passed today); `timeout`; `workingDir`; `exitValues`. - `pqc-*`: `keyPair`, `keyStore`, `keyPairAlias`; `symmetricKeyLength`. - `tika-*`: `tikaParseOutputFormat`. `docling-*`: `outputFormat`; OCR options; timeouts; Docling Serve login. - `langchain4j-ingest-file-source`: `maxMessagesPerPoll`; `initialDelay`; `readLockMinAge`. - `infinispan-*`: configuration properties (TLS); the operation of the sink (only PUT today); `lifespan`, `maxIdle`. - `elasticsearch-index-sink`, `opensearch-index-sink`: timeouts; a trust store. - `solr-*`: the number of rows (more than 10); timeouts; `requestHandler`. - `fhir-sink`: timeouts; `compress`; `summary`; `validationMode`. `fhir-source`: `delay` (0.5 s); an OAuth access token; proxy. - `nominatim-*`: language. `ogcapi-*`: timeouts; login. `graphql-sink`: `queryFile`, `variables`, `operationName`; proxy. - `splunk-hec-sink`: `splunkEndpoint` (raw). `snmp-source`: `timeout`, `retries`; `snmpContextName`; `initialDelay`. ## Google Cloud and IBM Cloud - `google-bigquery-sink`: `useAsInsertId` (no duplicate rows when a write is retried). - `google-calendar-source`: `maxResults` (10 today); `query`; `considerLastUpdate`; service account key and delegate. - `google-mail-source`: `maxResults`; `raw` (attachments); service account key and delegate. - `google-sheets-*`: service account key and delegate. `google-sheets-source`: `valueRenderOption`; `maxResults`. - `google-pubsub-sink`: `messageOrderingEnabled` with `pubsubEndpoint`. `google-pubsub-source`: `ackMode`; `maxDeliveryAttempts`; `maxAckExtensionPeriod`. - `google-storage-event-based-source`: `ackMode`; `maxDeliveryAttempts`. - `google-storage-source`: `moveAfterRead` with `destinationBucket`; `includeBody`; `includeFolders`; `delay`; workload identity or impersonation. `google-storage-sink`: `objectName`; `storageClass`; `storageLocation`; workload identity. - `google-vertexai-sink`: `publisher` and `anthropicVersion` (rawPredict); `jsonMode`; `candidateCount`; streaming. - `ibm-cos-source`: `fileName`. `ibm-cos-sink`: `deleteAfterWrite`. The plan is to fix the first table in a few PRs, one per family, each tested against a local stand-in of the service, and to pick up the rest when someone needs them. _Claude Code on behalf of Claus Ibsen_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
