imbajin opened a new issue, #3346:
URL: https://github.com/apache/hugegraph/issues/3346

   ## Goal
   
   Track long-term, evidence-driven performance work across HugeGraph traversal 
and its shared execution/collection paths. Start with Kout as a reproducible 
case, then extend to other representative operations when measurements reveal 
shared bottlenecks.
   
   Optimize end-to-end efficiency under realistic workloads, rather than 
greedily improving one method, microbenchmark, or configuration at the expense 
of correctness, memory, latency, or concurrent throughput.
   
   ## Starting evidence
   
   [🔗 Server #3342](https://github.com/apache/hugegraph/pull/3342) fixes lost 
Kout results by serializing shared result-set updates and distinct-limit 
admission. Preserve this correctness guarantee.
   
   [Bounded measurements and 
limitations](https://github.com/apache/hugegraph/pull/3342#pullrequestreview-5514009398)
 show workload-dependent trade-offs between internal parallelism and concurrent 
requests. They do not establish a production regression caused by the fix, 
isolate monitor cost, or establish production-wide performance acceptance. No 
narrowed-lock optimization has been benchmarked.
   
   This is a long-term task, not an additional 1.8 release gate. Candidate 
optimizations below are hypotheses to evaluate, not commitments to implement.
   
   ## TODO
   
   - [ ] Establish a reusable, correctness-checked baseline and retain 
reproducible commands, data generation, raw results, environment details, and 
exact tested revisions.
   - [ ] Profile representative end-to-end workloads before prioritizing 
changes.
   - [ ] Rank opportunities by measured user impact, resource cost, 
implementation risk, and maintenance burden.
   - [ ] Implement justified optimizations separately, with before/after 
comparisons against a correct baseline.
   - [ ] Add stable regression coverage and document deployment/concurrency 
trade-offs.
   - [ ] Expand coverage to related traversals and shared infrastructure as 
evidence warrants.
   
   ### Candidate investigations
   
   - [ ] Internal worker counts, executor reuse/admission, and resource budgets 
across concurrent requests.
   - [ ] Bounded/lazy edge-query and iterator preparation to avoid unnecessary 
work after limits are satisfied.
   - [ ] Primitive IdSet bulk union to reduce numeric-ID wrapping during layer 
merges.
   - [ ] Empty-frontier termination and avoidable allocation/cancellation 
overhead.
   - [ ] Critical-section scope or aggregation strategies only where profiling 
justifies them; preserve exact distinct limits, deduplication, visibility, 
resource cleanup, and off-heap ID safety.
   
   ## Measurement and acceptance
   
   - [ ] Cover single and concurrent requests; several worker settings 
including the actual default; numeric/string IDs; unique/duplicate/convergent 
graphs; finite/unlimited limits; nearest modes; and representative graph sizes.
   - [ ] Include relevant backends and full API/service paths, not only 
isolated consumers. Distinguish warmed caches from genuinely cold storage.
   - [ ] Record throughput, latency distributions, CPU, allocation/GC, memory, 
and lock/scheduling observations. Use sufficient independent runs, warm-up, 
sustained durations, and variability reporting; short repeated samples are not 
independent experiments or production tail-latency proof.
   - [ ] Verify complete results where deterministic, exact counts and legal 
results for bounded queries, plus cancellation, capacity and resource-lifecycle 
behavior.
   - [ ] Agree workload-specific acceptance budgets before claiming success. 
Check for displaced costs and regressions in other supported workloads.
   - [ ] Change one causal factor at a time where practical. Report 
inconclusive results honestly; do not use unsafe old concurrent behavior as a 
performance target.
   
   Keep findings and follow-up PRs linked here so improvements accumulate into 
a coherent performance baseline rather than isolated tuning.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to