corgy-w opened a new pull request, #13218:
URL: https://github.com/apache/seatunnel/pull/13218
### What
The Qdrant sink now writes each numeric payload with the value type its
SeaTunnel type maps to: `BIGINT` as a Qdrant integer, `FLOAT`/`DOUBLE` as a
Qdrant double.
### Why
`QdrantBatchWriter.buildPayload` grouped the numeric types under the wrong
primitive parsers:
```java
case SMALLINT:
case INT:
case BIGINT:
return ValueFactory.value(Integer.parseInt(value.toString())); //
BIGINT beyond 2^31-1 overflows
case FLOAT:
case DOUBLE:
return ValueFactory.value(Long.parseLong(value.toString())); //
"1.5" is not a long
```
Consequences, both reproduced locally by calling the method directly:
| input | result before the fix |
| --- | --- |
| `DOUBLE 1.5` | `java.lang.NumberFormatException: For input string: "1.5"` |
| `FLOAT 2.5` | `java.lang.NumberFormatException: For input string: "2.5"` |
| `BIGINT 5_000_000_000` | `java.lang.NumberFormatException: For input
string: "5000000000"` |
| `INT 42` | works |
The exception is thrown from `addToBatch`/`buildPoint`, so the sink task
dies on the first row that carries a fractional double or a big integer — the
job never completes. Even when a value does parse (for example `2.0`), it is
stored as a Qdrant integer instead of a double, which contradicts the
documented mapping in `docs/en/connectors/sink/Qdrant.md:77-79` (`BIGINT ->
payload integer`, `FLOAT/DOUBLE -> payload double`) and is read back by the
Qdrant source as `0.0`.
### Fix
Split the cases so each type uses the matching parser:
```java
case SMALLINT:
case INT:
return ValueFactory.value(Integer.parseInt(value.toString()));
case BIGINT:
return ValueFactory.value(Long.parseLong(value.toString()));
case FLOAT:
case DOUBLE:
return ValueFactory.value(Double.parseDouble(value.toString()));
```
`ValueFactory` has `value(long)` and `value(double)` overloads (io.qdrant
client 1.11.0), so the payload keeps the declared Qdrant type.
### How verified
**Reproduced first, then fixed.** `connector-qdrant` had no test sources at
all, so a unit test class is added (`QdrantBatchWriterTest`) covering the four
numeric cases:
- before the fix: `Tests run: 4, Failures: 0, Errors: 3` — the three
`NumberFormatException`s above;
- after the fix: `Tests run: 4, Failures: 0, Errors: 0, Skipped: 0` (`mvn -o
-pl seatunnel-connectors-v2/connector-qdrant test
-Dtest=QdrantBatchWriterTest`).
**E2E coverage** — `connector-qdrant-e2e` gains `testQdrantNumericPayload`
plus `fake_to_qdrant_numeric_payload.conf` (FakeSource with `id = bigint`,
`score = double`, `ratio = float`, `my_vector = float_vector` → Qdrant). It
asserts the job exits 0, that five points are written, and that each payload
keeps its kind (`INTEGER_VALUE` for the bigint, `DOUBLE_VALUE` for the
float/double). Before the fix that job fails with the `NumberFormatException`
above, so the IT is red today and green with the change.
**Note on the e2e run:** I could not execute this IT in my environment — it
needs `org.testcontainers:qdrant:1.21.4` (only 1.20.1 is cached), the SeaTunnel
starters, and the `seatunnel:<version>` image built by `seatunnel-dist`, and
the configured Maven mirror returns 502. The reproduction above is the exact
code path the IT exercises (`buildPayload`), executed as a unit test, so the
failure is demonstrated deterministically; CI will run the IT itself.
### Compatibility
No option name, default value, or public API change.
`INT`/`SMALLINT`/`STRING`/`DATE`/`BOOLEAN` payloads are unchanged; only
`BIGINT`, `FLOAT` and `DOUBLE` payloads change, from "throws" (or a truncated
integer) to the documented Qdrant type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]