bitflicker64 opened a new pull request, #805:
URL: https://github.com/apache/hugegraph-toolchain/pull/805
## Purpose of the PR
More test coverage across the toolchain modules before the 1.8.0 release, as
requested for the Java 17 migration. This PR only adds or registers tests. It
changes no production code, dependencies, CI workflows or documentation.
Each module is a separate commit, so the PR can be reviewed one commit at a
time.
## Main Changes
**hugegraph-client**
- Register tests that compiled but never ran: `HugeClientBuilderTest` and
`GremlinManagerTest` (unit), `CypherApiTest` and `AdamicAdarAPITest` (API) and
`GraphModeTest` (functional). All pass against the locally built Server 1.8;
the Server 1.7 baseline is exercised by CI only. `SameNeighborsBatchApiTest` is
left to #673.
- `ServerExceptionTest`: error bodies with the standard keys, the
alternative keys, non-JSON bodies, empty bodies and empty causes.
- `HugeClientFakeServerTest`: the client against a local `HttpServer` that
returns 5xx HTML, non-JSON bodies or an unsupported API version, answers slowly
or drops the connection. It also checks that requests to a legacy server leave
out the `graphspaces/DEFAULT` prefix.
- `GraphElementTest` and `JsonUtilTest`: property value validation and
serialization error paths.
**hugegraph-tools** (all registered in `FuncTestSuite`)
- `BackupFailureTest`: backup against a fake server. A 5xx on the shard
query is retried and then fails with the original `ServerException`; non-JSON
shards and schema errors fail the backup; an invalid split size or label/type
combination is rejected.
- `LocalDirectoryTest`: plain and zip streams, append and override, corrupt,
empty and missing archives, directory checks.
- `RetryManagerTest`: retry success, exhaustion and the default of no
retries.
- `JsonGraphTest`: the dump graph, `JsonFormatter` and formatter loading.
**hugegraph-loader** (all registered in the `unit` profile)
- `DataTypeUtilTest`: conversion of numbers, booleans, UUIDs, text, dates
(extra formats, time zones, timestamps, JDBC temporal types) and list/set
values, each with invalid inputs.
- `LineParserTest`: text, CSV (quotes, escaped quotes) and JSON line
parsing, missing and extra columns.
- `LoadMappingTest`: valid and invalid struct files (version, empty or
duplicate structs, a missing input type, an unknown source type).
- `BatchInsertRetryTest`: `BatchInsertTask` against a fake server. A 5xx is
retried until it succeeds or the retry limit is reached.
`IllegalArgumentException` and the unacceptable messages fail at once.
**hugegraph-spark-connector**
- `DateUtilsTest` and more `DataTypeUtilsTest` cases: invalid numbers, text
conversion, string dates, null values, unsupported list/set cardinality.
**hugegraph-hubble**
- hubble-be: `EntityUtilTest` covers `@MergeProperty` update semantics.
`HubbleUtilTest` covers collection equality, date helpers, digests, the host
pattern and the size and duration serializers.
- hubble-fe: Jest tests for storage and duration formatting, `removeNilKeys`
(algorithm request payloads) and `convertStringToJSON`.
**hugegraph-client-go** (`httptest` only, no live server)
- `hgtransport`: URL prefix, basic auth from config or URL, explicit
`Authorization`, user agent, content type, 5xx pass-through, dropped
connections, timeouts, logger body handling.
- `api.Response` helpers, `NewCommonClient` config validation, `/versions`
with non-JSON, JSON error and dropped-connection responses.
- Vertex create, batch create and property update, with and without a
graphspace. The package's only test, `TestVertex`, has its whole body commented
out, so the vertex package showed 0.0% coverage with the CI test flags; it now
shows 85.5%.
## Verifying these changes
- [ ] Trivial rework / code cleanup without any test coverage. (No Need)
- [ ] Already covered by existing tests, such as *(please modify tests
here)*.
- [x] Need tests and can be verified as follows:
- Environment: Toolchain at 039153cd, Java 17.0.20, Maven 3.9.14, Node
18.20.8, Go 1.22.2. The Server 1.8 SDK and server were built from the CI-pinned
commit 68855199 with `install-candidate-sdk.sh` into a separate Maven
repository. Authentication was on, with the batch limits that CI sets.
- Each new class was also run on its own twice, and a timeout check was
confirmed to fail when its condition is removed.
| Command | Before | After |
| --- | --- | --- |
| client `mvn test -Dtest=UnitTestSuite` | 77 passed | 118 passed |
| client `mvn test -Dtest=ApiTestSuite` | 376 run, 3 skipped | 382 run, 3
skipped, 0 failed |
| client `mvn test -Dtest=FuncTestSuite` | 94 run, 1 error
(`HugeClientHttpsTest`) | 104 run, 6 errors, all in `HugeClientHttpsTest` |
| tools `mvn test -Dtest=FuncTestSuite -Dtools.test.disposable=true` | 30
passed | 55 passed |
| loader `mvn test -P unit` | 100 passed | 134 passed |
| loader `mvn test -P file` (unchanged) | | 99 run, 2 errors, both HTTPS
tests |
| spark-connector `mvn test` | 59 passed | 67 passed |
| hubble-be `mvn test -P unit-test` | 614 passed | 623 passed |
| hubble-fe `yarn lint`, `yarn i18n:check`, `yarn test` | 162 suites, 1032
tests | 165 suites, 1079 tests, lint clean |
| client-go `go test -race -count=1 ./...` | all packages pass | all
packages pass |
Notes:
1. Port 8443 on the test host was held by an unrelated process, so the HTTPS
server could not start. `HugeClientHttpsTest` and the loader HTTPS tests fail
for that reason in both runs. CI starts its own HTTPS server.
2. Not run: the loader `hdfs`, `jdbc` and `kafka` profiles (no HDFS, MySQL
or Kafka started), the Hubble packaged API/browser test, and the opt-in
`HugeGraphIdentityIntegrationTest`, which needs a server without authentication.
### Production issues found (tests not included)
These tests fail on the current code, so they are not part of this PR:
1. **A malformed JSON line aborts the whole load.** `JsonLineParser.parse`
catches `SerializeException`, but `JsonUtil.convertMap` throws the loader's
`DeserializeException`, so no `ReadException` is raised and `--max-read-errors`
has no effect. Reproduced with `FileLoadTest` resources
(`reload_json_failure_files`): one valid edge line plus `{"source_name": `,
with `--max-read-errors 10`. The load threw `DeserializeException`, and the
valid edge was not loaded either.
2. **Batch inserts never retry network failures.** `BatchInsertTask` retries
on `ClientException`, but the client now throws a raw `IOException` for
timeouts and dropped connections. With `retryTimes=3`, a fake server that
answers after 2.5 s (client timeout 1 s) got one request before
`SocketTimeoutException` reached the caller. A dropped connection got two
requests: the first try and one OkHttp retry, none from the loader. 5xx
responses are retried correctly; `BatchInsertRetryTest` covers that case.
3. **LONG values above `Long.MAX_VALUE` wrap around.**
`DataTypeUtil.parseLong` uses `Long.parseUnsignedLong` for non-negative input,
so `"9223372036854775808"` becomes `-9223372036854775808` and
`"18446744073709551615"` becomes `-1` instead of being rejected. The same
helper parses numeric vertex IDs. The Spark connector's
`DataTypeUtils.parseLong` has the same code (checked by reading, not run).
4. **BLOB values become null.** `DataTypeUtil.parseBlob` has its body
commented out and returns `null` for any input, for example `"0x01"` or
`"AQI="`.
5. **Unimplemented Go vertex methods panic.** `client.Vertex.GetByID()` and
the other unassigned function fields cause a nil pointer panic instead of
returning an error.
6. **A latent race in the Spark connector `DateUtils`.** It shares one
`SafeDateFormat` per pattern and calls `setTimeZone` before `parse`, the code
the loader fixed in #734. A port of the loader's concurrent test returned 1
wrong instant out of 1600. The connector currently always passes GMT+8, so the
sink path does not hit it.
## Does this PR potentially affect the following parts?
- [x] Nope
- [ ] Dependencies (add/update license info)
- [ ] Modify configurations
- [ ] The public API
- [ ] Other affects (typed here)
## Documentation Status
- [ ] `Doc - TODO`
- [ ] `Doc - Done`
- [x] `Doc - No Need`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
