bitflicker64 commented on code in PR #372:
URL: https://github.com/apache/hugegraph-ai/pull/372#discussion_r4281595283
##########
hugegraph-llm/docs/extraction-runtime.md:
##########
@@ -0,0 +1,188 @@
+# Reviewed Property-Graph Extraction
+
+The first version adds optional model review and repair to the existing
+`POST /graph/extract` path. Requests without `review_options`, or with it set
to
+`null`, keep the existing extraction behavior. Sending an object enables
review;
+`{}` selects the defaults.
+
+The reviewed path uses the existing configured chat model for extraction,
review,
+and repair. It validates the graph against the supplied schema and returns a
+result for every chunk. It does not write graphs to HugeGraph.
+
+For OpenAI and LiteLLM, our client wrappers omit `max_tokens` from the SDK call
+in all three stages. SDK provider adapters and services may still apply default
+output limits or hard output and context limits. Existing requests without
review
+and other model uses retain their configured token limit.
+
+## Setup and a runnable example
+
+Use Python 3.10 or 3.11 and run from the repository root:
+
+```bash
+uv sync --python 3.11 --extra llm --extra dev
+```
+
+Configure a supported chat model using the existing
+[LLM configuration](../config.md). The example uses the same
+`get_chat_llm(llm_settings, limit_output=False, raise_on_truncation=True)`
+client factory as the extraction node, so it requires a reachable configured
+model and any credentials needed by that model. An inline schema avoids a
+HugeGraph Server connection.
+
+Run the [Python example](examples/reviewed_property_graph.py):
+
+```bash
+uv run --no-sync python hugegraph-llm/docs/examples/reviewed_property_graph.py
+```
+
+It runs two text chunks concurrently using the complete inline schema in
+[reviewed-extraction.json](examples/reviewed-extraction.json), creates a
separate
+client for each chunk, and prints all chunk results. Inspect their `status`,
+`issues`, and call counts; a successful process exit alone does not mean every
+chunk was accepted. Live model outputs and review decisions can vary.
+
+To use the HTTP entry, start the existing service:
+
+```bash
+uv run --no-sync python -m hugegraph_llm.demo.rag_demo.app
+```
+
+Then send the same example request from the repository root:
+
+```bash
+curl -X POST http://127.0.0.1:8001/graph/extract \
+ -H 'Content-Type: application/json' \
+ --data-binary @hugegraph-llm/docs/examples/reviewed-extraction.json
+```
+
+On Windows PowerShell, use `curl.exe` and place the command on one line. The
+request contains `texts`, an inline `schema`, and:
+
+```json
+"review_options": {"max_reviews": 2, "max_fixes": 1, "max_workers": 2}
+```
+
+The supplied schema defines person and company vertices with `name` primary
keys,
+and `works_for` edges from person to company. It does not need to be installed
in
+HugeGraph. Existing named-schema requests still use `client_config`.
+
+## Processing and budgets
+
+Each chunk follows:
+
+```text
+extract -> validate -> review
+ ^ |
+ | +-- pass -> final
+ | +-- block -> blocked
+ +-- repair <- fix
+```
+
+Validation checks graph structure, labels, properties, schema-provided types
and
+required fields, vertex identity, and relation endpoints. Invalid entries are
+retained so the model can repair them. Review compares the current graph with
the
+original text and schema for unsupported facts, wrong properties, relation
+direction, and clear omissions. The existing extraction prompt is reused;
review
+and repair prompts live in `PromptConfig` alongside it.
+
+Repair returns a complete replacement graph. The engine discards old issues,
+validates the replacement, and reviews it again when budget remains. Model or
+JSON parsing errors fail that chunk instead of being converted into an empty
+successful graph. An actual empty graph can pass if validation and review
accept
+it.
+
+| Option | Default | Accepted range | Meaning |
+| --- | --- | --- | --- |
+| `max_reviews` | 2 | 0–5 | Maximum review attempts per chunk. |
+| `max_fixes` | 1 | 0–3 | Maximum repair attempts per chunk. |
+| `max_workers` | 4 | 1–8 | Maximum chunks being prepared or executed
concurrently. |
+
+All options require integers; booleans, fractional numbers, and numeric strings
+are rejected. Unknown keys, such as `max_review`, and out-of-range values
return
+HTTP 422 before extraction starts. Zero reviews or repairs disables that stage;
Review Comment:
Important: `max_reviews: 0` does not disable review. It makes acceptance
impossible. In `engine.py` (lines 76-77), a chunk that passes validation with
no review budget left returns `candidate` with `review_budget_exhausted`, and
`ExtractNode._extract_with_review` merges only `final` graphs. So any request
with `{"max_reviews": 0}` still pays for one extraction call per chunk (plus
any repairs), and it always returns `status: "failed"` with an empty `result`.
`test_no_final_chunks_returns_business_failure_with_http_200` and the
pooled-pipeline test assert exactly this. The Chinese architecture doc gets it
right (its flowchart sends "no review budget" to `candidate`), but this
sentence tells English readers the stage is simply skipped. Please either
reject 0 with `ge=1` on `max_reviews` in `contracts.py`, or reword this
sentence to say that with zero reviews no chunk can become `final` and the
merged result is always empty. If you choose `ge=1`, also update the `0` cases
in `test_co
ntracts.py`, `test_engine.py`, `test_graph_extract_api.py` and the integration
tests, and the `0–5` ranges in both docs.
##########
hugegraph-llm/src/hugegraph_llm/models/llms/openai.py:
##########
@@ -67,13 +69,15 @@ def generate(
completions = self.client.chat.completions.create(
model=self.model,
temperature=self.temperature,
- max_tokens=self.max_tokens,
+ **({"max_tokens": self.max_tokens} if self.max_tokens is not
None else {}),
messages=messages,
)
if not completions.choices:
raise RuntimeError(f"Empty choices in LLM response:
{str(completions)[:200]}")
if completions.usage:
log.info("Token usage: %s",
completions.usage.model_dump_json())
+ if self.raise_on_truncation and
completions.choices[0].finish_reason == "length":
+ raise LLMOutputTruncatedError()
return completions.choices[0].message.content
# catch context length / do not retry
except openai.BadRequestError as e:
Review Comment:
Minor: on the reviewed path, a provider rejection is reported as a JSON
parse failure. This handler turns `BadRequestError` into the string `Error:
...`. That exception is how OpenAI-compatible servers report a context-length
overflow. `AuthenticationError` just below gets the same treatment.
`PropertyGraphBundle` passes the string to `_parse_json`. I reproduced this
with a mocked SDK that raises on the review call: the chunk ends `failed`,
stage `review`, reason `review_failed: JSONDecodeError`. Review and repair
prompts embed the text, the full schema and the graph, so they are the calls
most likely to hit the context limit. Please re-raise both errors when
`raise_on_truncation` is set (or under a separate flag passed only by the
reviewed path), so `chunk_results[].reason` names the provider error. Add a
case for it to `test_output_limits.py`. The `agenerate` handlers can get the
same change for consistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
