SEPURI-SAI-KRISHNA opened a new issue, #13220:
URL: https://github.com/apache/seatunnel/issues/13220

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Bug%22)
 and found no similar issues. No open PR changes `connector-graphql`; #11772, 
#12409 and #13216 change only the sink docs.
   
   ### What happened
   
   GraphQL servers answer a rejected query or mutation with HTTP 200 and an 
`errors` array. The GraphQL connector does not look at `errors`, so a rejected 
request is treated as a success:
   
   1. **Sink.** `GraphQLSinkWriter.write` returns when the status is 200. For 
any other status it only logs the response and returns, so that row is dropped 
as well. The job finishes as if every row was written.
   2. **Source (HTTP query mode).** `GraphQLSourceHttpReader` passes the body 
to `content_field`. With only `errors` there is nothing to read, so the job 
finishes with 0 rows. With partial `data` plus `errors`, the partial rows are 
read and the errors are ignored.
   
   Two more problems show up once errors are no longer hidden:
   
   3. **`variables` not set.** `GraphQLSourceParameter` and 
`GraphQLSinkParameter` then put the string `"{}"` into the body, so the request 
is `{"variables":"{}", ...}`. Hasura rejects it with `parsing HashMap failed, 
expected Object, but encountered String`. The sink casts that value to a `Map`, 
so a sink without `variables` fails at the first row with `ClassCastException`.
   4. **Date and time fields in the sink.** The body is built with Gson 2.10.1, 
which has no adapter for `java.time`. On Java 11 a `LocalDateTime` is sent as 
`{"date":{"year":2020,...},"time":{...}}`; on Java 17 Gson throws 
`JsonIOException`.
   
   This is why `GraphQLIT` passes even though its tables were never tracked in 
Hasura. The log of the `updated-modules-integration-test-part-1 (11)` job for 
#12273 
(https://github.com/KaustAbhinand/seatunnel/actions/runs/34638120927/job/103393313711)
 has 21 `/v1/graphql` responses, all HTTP 200. 7 say `field 'source' not found 
in type: 'query_root'` and 14 say `no mutations exist`. `GraphQLIT` reports 7 
tests, 0 failures. No row reached Hasura or came back from it: the jobs that 
write count their rows as written although every mutation was rejected, and the 
jobs that read get 0 rows.
   
   I also ran each case with the connector built from dev (`bin/seatunnel.sh -m 
local`, Zeta), against a local HTTP server that answers like Hasura, with HTTP 
200 and an `errors` array:
   
   - The config below: the job ends `FINISHED` with `Total Write Count 1` and 
`Total Failed Count 0`.
   - The source with only `errors`: `FINISHED`, 0 rows. With 2 rows of partial 
`data` plus `errors`: 2 rows read.
   - No `variables`: the source sends `{"variables":"{}",...}`, and the sink 
fails with `ClassCastException: class java.lang.String cannot be cast to class 
java.util.Map`.
   - A `timestamp` field: on Java 11 the server receives 
`"ts":{"date":{"year":2020,"month":2,"day":2},"time":{"hour":2,...}}`. On Java 
17 the sink fails with `JsonIOException: Failed making field 
'java.time.LocalDateTime#date' accessible`.
   
   I would like to submit a PR that:
   
   - fails the sink on a non-200 status or a non-empty `errors` array, and 
fails the HTTP query source on a non-empty `errors` array, with the server's 
errors in the message;
   - sends an empty object when `variables` is not set;
   - sends date and time fields as ISO 8601 strings;
   - tracks the tables in `GraphQLIT`, checks the rows the sink writes, and 
adds a rejected mutation and a rejected query that must fail the job.
   
   WebSocket subscription mode is not part of this.
   
   ### SeaTunnel Version
   
   dev (535d4237dd). The GraphQL code is the same on the latest dev.
   
   ### SeaTunnel Config
   
   ```conf
   env {
     parallelism = 1
     job.mode = "BATCH"
   }
   
   source {
     FakeSource {
       schema = { fields { id = int } }
       rows = [ { kind = INSERT, fields = [1] } ]
     }
   }
   
   sink {
     GraphQL {
       url = "http://localhost:8080/v1/graphql";
       query = """
         mutation Insert($id: Int!) {
           insert_missing_table(objects: {id: $id}) { affected_rows }
         }
       """
       variables = {}
     }
   }
   ```
   
   ### Running Command
   
   ```shell
   ./bin/seatunnel.sh --config fake_to_graphql.conf -m local
   ```
   
   ### Error Exception
   
   ```log
   No error. The server answers HTTP 200 with
   {"errors":[{"message":"field 'insert_missing_table' not found in type: 
'mutation_root'", ...}]}
   and the job ends with:
   Job (...) end with state FINISHED
   Total Read Count          :                   1
   Total Write Count         :                   1
   Total Failed Count        :                   0
   ```
   
   ### Zeta or Flink or Spark Version
   
   Zeta for the runs above. The CI log above covers Zeta, Flink and Spark. The 
connector code is the same for all engines.
   
   ### Java or Scala Version
   
   Java 11 and 17 (the date and time behaviour differs, as above).
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [I] [Bug][Connector-V2][GraphQ... SEPURI-SAI-KRISHNA (via GitHub) via commits

Reply via email to