SebastianGruza opened a new pull request, #3209:
URL: https://github.com/apache/hugegraph/pull/3209
## Purpose of the PR
- close #3206
The numeric property types today are `BYTE/INT/LONG/FLOAT/DOUBLE`. Values
that do not fit a `long` and must not be rounded (token balances in wei, up to
2^256 - 1; money amounts in general) can only be stored as `TEXT`, which loses
the one place where the server itself does arithmetic: `update_strategies` in
`PUT /graph/{vertices,edges}/batch` (`SUM` / `BIGGER` / `SMALLER`).
`UpdateStrategy` already computes in `BigDecimal`, but the result goes back to
the property's type: with `DOUBLE` a `SUM` of `10^18 + 1` is `10^18`, and
`TEXT` fails the strategy's `Number` type check. This PR adds an exact decimal
type so that accumulating balances during an import works. The design points
were posted in #3206 on 2026-09-14; no objections so far.
## Main Changes
- **`DataType.DECIMAL(12, "decimal", BigDecimal.class)`** in the server enum
and in the `hugegraph-struct` copy, with `isDecimal()` and `valueToDecimal()`
(exact for `BigDecimal`, `BigInteger` and integral Java numbers;
`Float`/`Double` through their shortest decimal representation; decimal
strings). `PropertyKey.Builder.asDecimal()`, REST `data_type: DECIMAL`.
- **No sort key, no index, no OLAP range**: `isNumber()` stays `false` on
purpose; `PropertyKeyBuilder`, `IndexLabelBuilder` and `EdgeLabelBuilder`
reject these with an explicit message. There is no fixed-width
byte-order-preserving encoding for a decimal, and faking one through
`LongEncoding` would be lossy. `SUM/MAX/MIN` aggregate types on the property
key are allowed, as for numbers.
- **Encoding** in `BytesBuffer` (server core and struct): `vint(len)` +
unscaled two's-complement bytes + `vint(scale)`. Exact for any precision, scale
preserved, 33 bytes for a uint256. Existing encodings untouched; `OffheapCache`
gets the new value type appended at the end of its enum.
- **JSON**: always a plain string on output (`toPlainString()`,
`HugeGraphSONModule`); a string or a number literal accepted on input. A JSON
number is a `double` to most clients, so a string is the only lossless
representation.
- **`ConditionQuery`** compares exactly when one side is a `BigDecimal`
(instead of through `doubleValue()`); the store-side row decoder
(`GraphStoreIterator`) maps a decimal to a string variant.
- **`BatchAPI.updateExistElement`**: the JSON value is normalised through
the property key *before* the `update_strategies` strategy runs, on both paths
(two entries of one id within a request; request vs stored element). Found by
the new API test: the strategy used to receive the raw JSON value, which only
worked for the types Jackson happens to produce, so a decimal (or a date) sent
as a string failed the type check.
Known limit, documented in the issue: in the batch update a **fraction has
to be sent as a string**, because the request's `properties` map is parsed by
Jackson before any schema is known (`0.000000000000000001` becomes a `double`
literal); integral literals are exact. `hugegraph-client` / loader / Hubble
will get the type in a separate toolchain change.
## Verifying these changes
- [ ] Trivial rebase, no need to test
- [x] Unit tests / core tests / API tests added and passing locally (JDK 11,
through the CI scripts):
- `unit/core/DataTypeTest`: predicates, `valueToDecimal` for uint256 max,
wei scale, integral and binary numbers, invalid strings
- `unit/serializer/BytesBufferTest`: exact byte layout for `-1.5`, `0`,
uint256 max; scale round trip; decimal lists
- `unit/util/JsonUtilTest`: string on output, string or number on input
- `core/PropertyKeyCoreTest`: create, value normalisation, `calcSum()`,
list cardinality
- `core/IndexLabelCoreTest`: secondary / range / shard / unique on a
decimal all rejected
- `core/EdgeLabelCoreTest`: decimal sort key rejected, decimal edge
property fine
- `core/VertexCoreTest`: uint256 and 18-fraction-digit values through
commit and reload, exact `has()` vs the neighbouring value, `gt/lt/gte`,
update, invalid values
- `api/VertexApiTest`: `PUT /graph/vertices/batch` with `SUM`: `2^256-2` +
`1` (number literal), then two entries of one vertex in one request
(`"0.000000000000000000"`, `"0.000000000000000001"`), then `BIGGER`; response
and `GET` carry the exact string
- struct `PropertyKeyTest`: groovy schema string, struct `BytesBuffer`
round trip
- Results: struct 4/4; `unit-test` 687/688 (`SecurityManagerTest.testFile`
fails identically on plain master on a non-English locale, unrelated);
`core-test` on rocksdb and memory for the four touched classes green (383 tests
on rocksdb); `api-test,rocksdb` 162 tests, 0 failures.
- [x] Cluster check on HStore (PD + 3 stores) and RocksDB: 10 000 vertices
with `balance/hi/lo DECIMAL`, 5 rounds of `PUT /graph/vertices/batch` with
`update_strategies: {balance: SUM, hi: BIGGER, lo: SMALLER}`, random increments
up to 2^255 with 18 fraction digits, 30 % negative, batches of 500, 4 writer
threads, 50 accounts per round appearing twice in one request; 2 000 edges with
a decimal `amount` behind an INT sort key. Every value read back and compared
exactly with a Python `Decimal` oracle: 50 250 upserts, 0 errors, **0 / 10 000
mismatches** on both backends; decimal sort key / range index rejected with the
intended message. Script and logs: `cluster/decimal_sum_bench.py` and
`results/decimal/` in https://github.com/SebastianGruza/hugegraph-validation.
- [ ] Docs: the property-key data type list in the `hugegraph-doc`
repository needs a `DECIMAL` row; I will open that PR once the type is in.
## Note on public API
New enum constant `DataType.DECIMAL` (code 12), new builder method
`PropertyKey.Builder.asDecimal()`, new REST value `data_type: DECIMAL`. No
change to existing types, encodings or endpoints; graphs without decimal
properties are unaffected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
