goutamadwant opened a new pull request, #13167:
URL: https://github.com/apache/seatunnel/pull/13167
### Purpose of this pull request
This contribution to Apache SeaTunnel adds Amplitude Source, requested in
#10753. It implements this source only and does not close the connector
umbrella.
- Bounded raw-event exports using inclusive UTC `server_upload_time` hours,
US/EU endpoints and project Basic authentication.
- Complete ZIP/gzip integrity checks before emission, with bounded archive,
decoded-byte, record and metadata processing.
- Single-split source and existing HTTP/JSON dependencies, plus
repository-managed Commons Compress; no Amplitude SDK or dependency-version
change.
- Owned background download/validation/parsing, bounded polling-thread
emission and cancellation without holding checkpoint locks across I/O waits.
- Factory/plugin discovery, distribution/CI/E2E wiring, examples and
English/Chinese documentation.
### Does this PR introduce _any_ user-facing change?
Yes. Users can configure `Amplitude` as a batch source with credentials, an
explicit schema, upload-time hours and optional JSON Pointer mappings.
This slice requires `parallelism = 1`. The inclusive end hour must have
fully elapsed at least two hours ago. An eligible-window 404 means no files are
currently available, not proof against late data.
Recovery replays the whole window, including earlier emitted rows:
at-least-once replay, not exactly-once delivery or an immutable snapshot.
Downstream deduplication/upsert keys are required where duplicates are
unacceptable. Local disk is bounded by `max_archive_bytes`; integrity
validation adds a full decompression pass. Archive limits and rejected variants
are documented.
No sink or continuous CDC is added. Existing connector options, shared HTTP
behavior and public APIs are unchanged.
### How was this patch tested?
- Missing factory discovery reproduced before implementation; failing
regressions added before correcting review-found resource and
checkpoint-boundary defects.
- JDK 11 and 17: all 42 Amplitude tests passed on each, covering
configuration/SPI, mapping and decimal precision, ZIP64/CRC/truncation,
resource limits, retries, cancellation, cleanup and partial-emission replay.
- HTTP base, PostHog, Asana and Mailchimp: 172 adjacent tests passed on each
JDK.
- The 68-module E2E prerequisite reactor compiled and installed with test
execution skipped for that prerequisite build.
- Docker Zeta/JDK 11: all three cases passed: retry/multi-entry export;
archive-specific job failure; checkpoint advancement during blocked download,
clean savepoint stop and successful full-window restore.
- Scoped package verification and Spotless passed.
Reproduction commands; repeat unit/adjacent checks with JAVA_HOME set to JDK
11 and then 17:
```sh
./mvnw -Pci -pl :connector-amplitude-e2e -am -DskipTests -DskipIT=true
-Dskip.ui=true install
./mvnw -f
seatunnel-connectors-v2/connector-http/connector-http-amplitude/pom.xml
spotless:check verify
./mvnw -f seatunnel-connectors-v2/connector-http/pom.xml -pl
connector-http-base,connector-http-posthog,connector-http-asana,connector-http-mailchimp
test
RUN_ENGINES=zeta ./mvnw -f
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amplitude-e2e/pom.xml
-DskipUT=true -DskipIT=false -Dit.test=AmplitudeIT verify
```
The fixture uses dummy credentials, not a live Amplitude project.
Flink/Spark E2E, native JDK 8, Windows, abrupt worker-crash recovery,
production throughput and full-repository tests remain unverified. Local
results do not imply hosted CI passed.
### Check list
* [x] License/NOTICE reviewed: Commons Compress is already covered; no new
third-party version or committed binary.
* [x] English/Chinese documentation and changelogs added.
* [ ] `incompatible-changes.md`: not applicable; existing behavior is
unchanged.
* [x] Connector integration updated:
1. `plugin-mapping.properties`
2. `seatunnel-dist/pom.xml`
3. `.github/workflows/labeler/label-scope-conf.yml`
4. `seatunnel-e2e` module and cases
5. `config/plugin_config`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]