Rishavbhattarai opened a new pull request, #12831: URL: https://github.com/apache/seatunnel/pull/12831
Yes, I'm contributing to Apache SeaTunnel. Closes #12823 ### Purpose of this pull request This adds an Intercom source connector, `connector-http-intercom`, built on `connector-http-base` the same way the Asana connector is. It reads contacts from Intercom's `GET /contacts` endpoint and loads them in batch mode. The scope is the one agreed in #12823: contacts only, access token auth only. ### What changed - New module `connector-http-intercom` with `IntercomSource`, `IntercomSourceFactory`, `IntercomSourceReader` and the option and parameter classes. Plugin name and factory identifier are both `Intercom`. - Auth is a Bearer token from the required `api_key` option. Every request also sends an `Intercom-Version` header, set by `api_version` (default `2.17`), so the response shape does not shift when a workspace's default version changes. - `base_url` defaults to `https://api.intercom.io`. EU and AU workspaces can point it at `https://api.eu.intercom.io` or `https://api.au.intercom.io`. - Pagination uses the existing `CURSOR` type. The cursor goes back as `starting_after` and is read from `$.pages.next.starting_after`. The read ends when `pages.next` is missing. `per_page` defaults to 150, which is Intercom's maximum, and is validated to 1..150. - The reader retries HTTP 429 and 5xx with bounded exponential backoff, reusing the existing `retry`, `retry_backoff_multiplier_ms` and `retry_backoff_max_ms` options. Other 4xx responses are not retried. `Retry-After` is not supported because `HttpResponse` does not expose headers. - Output columns: `id`, `role`, `email`, `name`, `phone`, `external_id` as strings, and `created_at`, `updated_at` as BIGINT (Intercom returns Unix seconds). I checked these field names and types against Intercom's 2.17 OpenAPI spec. The issue left that open. - Streaming mode is rejected. - Registered in `plugin-mapping.properties`, `config/plugin_config`, `connector-http/pom.xml`, `seatunnel-dist/pom.xml` and the e2e `pom.xml`. - English and Chinese docs and changelogs. ### Behavior to know about - Each run is a bounded snapshot of all contacts. The list endpoint has no "updated since" filter, so there is no incremental mode. - Nothing is persisted. After a failure the job restarts from the first page and can emit rows it already sent. Contacts created or merged during a run can also appear or disappear between pages. Delivery is at-least-once, and every row carries the contact `id` so a sink can deduplicate or upsert. - The token goes only into the `Authorization` header. It is not put in query params or the body, and the connector logs status codes and attempt counts only. I did not check whether the engine masks `api_key` when it logs the job config. ### Not in this PR OAuth, conversations, tickets, companies, the search endpoints, nested and custom attributes, and `Retry-After`. ### Dependencies None added. The connector uses `connector-http-base`. `okhttp3:mockwebserver` is test scope only, as in the Asana and Splunk modules. ### How it was tested Ran locally with JDK 11: - 13 unit tests in `connector-http-intercom` pass (`./mvnw -pl seatunnel-connectors-v2/connector-http/connector-http-intercom verify`). They use `MockWebServer` and cover multi-page cursor reading including a last page with no `next`, null `phone`, `name` and `external_id`, the auth and version headers, `per_page` and blank `api_key` validation, retry on 429 and 5xx, no retry on 401, giving up after the retry cap, and that error messages do not contain the token. - `spotless:check` passes. Not run locally: - The E2E test `connector-intercom-e2e` is written (a mock server that serves a 429, then two pages of contacts) but I have not run it. Docker was not available on my machine, and the Zeta starter modules it depends on were not built locally. I'm relying on CI for it. - I have not run the full repo test suite. I only touched the new modules, the registration files and docs. - I have not tested against a live Intercom workspace. All requests in the tests go to a mock server. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
