Rishavbhattarai commented on issue #12822: URL: https://github.com/apache/seatunnel/issues/12822#issuecomment-6224768218
Thanks @DanielLeens. I checked the open points against Typeform's docs and ran a probe through the existing HTTP base reader. Findings below. **Auth:** `Authorization: Bearer <token>` (Typeform walkthrough, curl example). **Cursor:** `before` is an exclusive query parameter. Pass the `token` of the last item on the page. The walkthrough is explicit about `token`, so I'll use it. The mock tests will give `token` and `response_id` different values, so a wrong field fails the test. **Page size:** the walkthrough gives a max of 1000. Plan: default 100, max 1000. **Sort:** I won't expose `sort` in v1. `before` paging relies on the default newest-first order. **Freshness:** the reference page says responses from roughly the last 30 minutes may not be returned yet. I'll note that in the docs. **JSON strings:** I ran a probe through the real `HttpSourceReader` and `JsonDeserializationSchema`, with a MockWebServer serving three pages and paging by `before`. The `answers` field, with 12 answer shapes, came out byte-identical to the source JSON. That covers key order, escaped quotes, unicode, decimals, and integers beyond double precision. Unknown answer types pass through unchanged, and a string that looks like JSON stays a string. **Edge case:** `"answers": []` and a missing `answers` come out as NULL, not `"[]"`. The cause is the shared `JsonPathProcessorImpl`. This PR shouldn't touch shared code, so I'll document the NULL behavior and pin it in a unit test. **Not re-verified:** the 2 requests/second rate limit and the default page_size of 25. I'll check both before relying on them. **Next:** once the design is agreed, I'll start `connector-http-typeform` from upstream/dev. Could you assign this issue to me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
