SEPURI-SAI-KRISHNA opened a new issue, #13124: URL: https://github.com/apache/seatunnel/issues/13124
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Bug%22) and found no similar issues. ### What happened The Cassandra source reads a NULL number or boolean as a real `0` or `false`, and the job succeeds. Downstream gets values that were never in Cassandra, and a NULL `balance` cannot be told apart from a balance of 0. `TypeConvertUtil.buildSeaTunnelRow` reads these cells with the driver's primitive getters (`row.getInt(i)` and so on) and never calls `row.isNull(i)`. The driver documents that these getters return 0 or false for a CQL NULL. Other types go the opposite way and fail the job. `TypeConvertUtil.reconvertAndInject`, used by the sink, has no NULL check either. Measured on `dev` against a real Cassandra 4.1.1 with driver 4.14.0, through a real prepared statement: | CQL type | Source reads NULL as | Sink given NULL | |---|---|---| | `tinyint`, `smallint`, `int`, `bigint` | `0` | NullPointerException | | `float`, `double` | `0.0` | NullPointerException | | `boolean` | `false` | NullPointerException | | `decimal`, `varint`, `uuid`, `timeuuid`, `blob` | NullPointerException | NullPointerException | | `inet` | NullPointerException | writes `127.0.0.1` | | `timestamp` | NULL | NullPointerException | | `list`, `set` | empty | NullPointerException | | `map` | empty | NULL | | `ascii`, `text`, `time`, `date` | NULL | NULL | A real Cassandra to Cassandra job on Zeta, where the source row `id = 1` has every other column NULL, finishes successfully and writes this row: ``` status=FINISHED id=1: c_bigint=0 c_boolean=false c_double=0.0 c_float=0.0 c_int=0 c_smallint=0 c_tinyint=0 ``` With the other column types in the table, the same job fails with a NullPointerException. The source and sink have to be fixed together. The sink only gets by on numbers today because the source never hands it a NULL, so fixing the source alone would make this successful job fail. A NULL `timestamp` shows this already: #9849 / #11068 fixed it in the source, and a NULL `timestamp` now fails in the sink. A NULL primary key needs care in that fix. A NULL `text` key from another source already reaches the server today, which rejects it, and the job still finishes: with `async_write = true` the row is lost and one error is logged, and with `async_write = false` the good rows in the same batch are lost too. A NULL `bigint` key only fails loudly because of the NullPointerException, so writing NULL values naively would make it lose rows the same way. The sink should fail on a NULL key with the table and column name. `CassandraIT` does not catch any of this because its fixture has no NULL cells. I have a fix ready and will open a PR for this issue. ### SeaTunnel Version dev (941276923f). 2.3.12, 2.3.13 and 3.0.0 have the same code. ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "BATCH" } source { Cassandra { host = "127.0.0.1:9042" username = "" password = "" datacenter = "datacenter1" keyspace = "test" cql = "select * from src" plugin_output = "src" } } sink { Cassandra { plugin_input = "src" host = "127.0.0.1:9042" username = "" password = "" datacenter = "datacenter1" keyspace = "test" table = "dst" } } ``` ### Running Command I ran this on an embedded Zeta server from a test harness, with my own host, keyspace and table names. With a binary distribution it is: ```shell bin/seatunnel.sh --config job.conf -m local ``` ### Error Exception ```log No error for the numeric and boolean columns: the job finishes with 0 and false in the sink. java.lang.NullPointerException for a NULL decimal, varint, uuid, timeuuid, inet or blob in the source. ``` ### Zeta or Flink or Spark Version Zeta (dev) ### Java or Scala Version JDK 17 ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
