lplalbert commented on code in PR #422:
URL: https://github.com/apache/hugegraph-ai/pull/422#discussion_r4579428483
##########
hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py:
##########
@@ -179,10 +235,41 @@ def load_into_graph(self, vertices, edges, schema): #
pylint: disable=too-many-
"(Input) EdgeLabel %s not found in schema, skip & need
check it!",
label,
)
+ skipped_edges += 1
continue
+ out_v, in_v = edge.get("outV"), edge.get("inV")
+ failed_endpoints = [endpoint for endpoint in (out_v, in_v) if
endpoint in skipped_ids]
Review Comment:
Good catch — pushed (ca011d2). Root cause: the endpoint check keyed only on
`skipped_ids`, but extraction without cross-chunk dedupe can leave a duplicated
input id in both `skipped_ids` (one copy skipped) and `vid_mapping` (another
copy created), so every edge to that id was dropped although the vertex exists.
Structural fix: an endpoint is skipped only when it is in `skipped_ids` and
no created vertex claimed it (`endpoint not in vid_mapping`). A duplicated id
whose copy was created keeps resolving through `vid_mapping` to the created
server id.
Test:
`test_load_into_graph_keeps_edges_when_a_duplicated_vertex_was_created` — one
copy of `1:Sarah` skipped on a non-nullable coercion failure, the other created
under the same id; the edge must run against the created copy's server id
(asserted on the `add_edge` call). Red without the fix (the edge was dropped
and `add_edge` never called).
##########
hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py:
##########
@@ -307,14 +394,80 @@ def _check_single_data_type(self, data_type: str, value)
-> bool:
PropertyDataType.INT.value,
PropertyDataType.LONG.value,
):
- return isinstance(value, int)
+ # bool is an int subclass in Python, but True is not a valid
number here
+ return isinstance(value, int) and not isinstance(value, bool)
if data_type in (PropertyDataType.FLOAT.value,
PropertyDataType.DOUBLE.value):
return isinstance(value, float)
- if data_type in (PropertyDataType.TEXT.value,
PropertyDataType.UUID.value):
+ if data_type in (
+ PropertyDataType.TEXT.value,
+ PropertyDataType.UUID.value,
+ PropertyDataType.BLOB.value,
+ ):
+ # BLOB is created as TEXT (see _set_property_data_type), so it
validates as text
return isinstance(value, str)
# TODO: check ok below
if data_type == PropertyDataType.DATE.value: # the format should be
"yyyy-MM-dd"
import re
- return isinstance(value, str) and re.match(r"^\d{4}-\d{2}-\d{2}$",
value)
+ return isinstance(value, str) and
bool(re.match(r"^\d{4}-\d{2}-\d{2}$", value))
raise ValueError(f"Unknown/Unsupported data type: {data_type}")
+
+ def _coerce_property_value(self, data_type: str, cardinality: str, value)
-> Tuple[Any, bool]:
+ """Losslessly coerce a property value to its declared type and
cardinality.
+
+ Returns ``(coerced_value, True)`` when the value is already valid or
convertible
+ without losing information (numeric strings to numbers, numbers to
text, scalars
+ to single-element collections), otherwise ``(None, False)`` so the
caller can
+ report the value instead of silently dropping it.
+ """
+ if cardinality in (PropertyCardinality.LIST.value,
PropertyCardinality.SET.value):
+ items = value if isinstance(value, list) else [value]
+ coerced_items = []
+ for item in items:
+ coerced, ok = self._coerce_single_value(data_type, item)
+ if not ok:
+ return None, False
+ coerced_items.append(coerced)
+ return coerced_items, True
+ return self._coerce_single_value(data_type, value)
+
+ def _coerce_single_value(self, data_type: str, value) -> Tuple[Any, bool]:
+ try:
+ if self._check_property_data_type(data_type,
PropertyCardinality.SINGLE.value, value):
+ return value, True
+ candidates = self._lossless_candidates(data_type, value)
+ for candidate in candidates:
+ if self._check_property_data_type(data_type,
PropertyCardinality.SINGLE.value, candidate):
+ return candidate, True
+ except ValueError:
+ return None, False # unknown data type: the caller reports the
value
+ return None, False
+
+ @staticmethod
+ def _lossless_candidates(data_type: str, value) -> List[Any]:
+ """Values that carry the same information as ``value`` in the declared
type."""
+ if isinstance(value, bool):
+ return []
+ if data_type in (PropertyDataType.TEXT.value,
PropertyDataType.UUID.value, PropertyDataType.BLOB.value):
+ return [str(value)] if isinstance(value, (int, float)) else []
+ if data_type in (
+ PropertyDataType.BYTE.value,
+ PropertyDataType.INT.value,
+ PropertyDataType.LONG.value,
+ ):
+ if isinstance(value, float):
+ return [int(value)] if value.is_integer() else []
+ if isinstance(value, str):
+ try:
+ return [int(value)]
+ except ValueError:
+ return []
+ return []
+ if data_type in (PropertyDataType.FLOAT.value,
PropertyDataType.DOUBLE.value):
+ if isinstance(value, (int, float, str)):
+ try:
+ return [float(value)]
Review Comment:
Agreed — pushed (ca011d2). Root cause: the FLOAT/DOUBLE branch accepted
`float(value)` without comparing the result to the value that was written, so
IEEE-754 rounding (`9007199254740993` and `"9007199254740993"` ->
`9007199254740992.0`) was reported as a valid conversion while the graph
received a different value.
Structural fix: a conversion is lossless only when the written value and the
converted value denote the same number (`Decimal(str(value)) ==
Decimal(repr(converted))`). Anything else is reported like an un-coercible
value (nullable property dropped, non-nullable skips the vertex with a reason)
instead of silently rounded. Out-of-range inputs (`10**400`, `"1e400"`) no
longer raise `OverflowError` or write `inf` — they report as un-coercible as
well. Exactly representable boundaries still coerce (`2**53` ->
`9007199254740992.0`, `"3.50"` -> `3.5`).
Test: `test_coerce_property_value_rejects_lossy_double_conversions` covers
the large-integer and numeric-string forms from the report, out-of-range values
for DOUBLE and FLOAT, and the exact-representability boundary. Red without the
fix (rounded values returned as valid; `10**400` crashed on `OverflowError`).
##########
hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py:
##########
@@ -162,15 +198,35 @@ def load_into_graph(self, vertices, edges, schema): #
pylint: disable=too-many-
else:
result =
self._handle_graph_creation(self.client.graph().add_vertex, input_label,
input_properties)
if result is None:
- raise ValueError(f"Failed to create vertex '{input_label}'
with properties {input_properties}")
+ log.error(
+ "Failed to create vertex '%s' with properties %s, skip it
& need check it again",
+ input_label,
+ input_properties,
+ )
+ skipped_vertices += 1
+ skip_reports.append(f"{mapping_id or input_label}: server
rejected the vertex")
+ if mapping_id:
+ skipped_ids.add(mapping_id)
+ continue
Review Comment:
Fixed — pushed (ca011d2). Root cause: `load_into_graph` only logged the
skips and `run()` returned `data` untouched, so the flow/API result carried no
record of the rejected vertex; and `WkFlowState.assign_from_json` drops keys
that are not state members, so a report attached in `run()` would not have
reached the result either.
Structural fix: `load_into_graph` now returns a structured `import_summary`
— `imported_vertices`/`imported_edges` counts, `skipped_vertices` as `[{id,
reason}]` (unknown label / missing primary key / non-nullable coercion failure
/ server rejection) and `skipped_edges` as `[{label, outV, inV, reason}]`,
including the dependent edges of skipped vertices — and `run()` attaches it to
the data returned into the pipeline. `WkFlowState` gains an `import_summary`
member (with `setup()` reset) so the report survives `assign_from_json` and
appears in the flow/API result via `ImportGraphDataFlow.post_deal`.
Test: `test_import_flow_reports_skipped_vertex_and_dependent_edges` runs the
real `ImportGraphDataFlow` end to end (server creates one vertex, rejects the
other) and asserts the result names `person:Bob` with its reason and the
dependent `knows` edge (label/outV/inV + reason). Red without the fix
(`import_summary` missing from the result).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]