dosubot[bot] commented on issue #619:
URL:
https://github.com/apache/incubator-hugegraph-toolchain/issues/619#issuecomment-2324793405
<!-- Greeting -->
Hello, @MorgenFreeMan! I'm here to help you with any bugs, questions, or
contributions you may have. Let's tackle this issue together!
<!-- Answer -->
The `OutOfMemoryError: Java heap space` issue you're encountering with the
HugeGraph Loader tool can be addressed by adjusting the memory settings and
optimizing the loader parameters. Here are some steps you can take:
1. **Increase Java Heap Space**:
Ensure that the `JAVA_OPTIONS` environment variable is set to allocate
more heap space. You can set this before running the loader script:
```bash
export JAVA_OPTIONS="-Xms28G -Xmx30G"
```
2. **Adjust Loader Parameters**:
The settings for `batch-insert-threads`, `max-conn`, and
`max-conn-per-route` should be optimized based on the number of logical cores
(CPUs) available. For an 8-core machine, the recommended settings are:
- `batch-insert-threads`: 8
- `max-conn`: 32
- `max-conn-per-route`: 16
Update your command to reflect these values:
```bash
sh hugegraph-loader.sh -g hugegraph_bqw_test -s
/opt/data/hugegraph/apache-hugegraph-toolchain-incubating-1.3.0/apache-hugegraph-loader-incubating-1.3.0/example/prod/schema.groovy
-f
/opt/data/hugegraph/apache-hugegraph-toolchain-incubating-1.3.0/apache-hugegraph-loader-incubating-1.3.0/example/prod/struct-instance_releation.json
-h 127.0.0.1 -p 58080 --clear-all-data false --batch-insert-threads 8
--max-conn 32 --max-conn-per-route 16
```
3. **Check for SLF4J Bindings**:
The multiple SLF4J bindings warning indicates potential conflicts. Ensure
that only one SLF4J binding is present in your classpath to avoid unexpected
behavior.
4. **Optimize Batch Size**:
The `batch_size` parameter in your `struct.json` is set to 500. You might
want to experiment with smaller batch sizes to reduce memory consumption:
```json
"batch_size": 200
```
By following these steps, you should be able to mitigate the
`OutOfMemoryError` and improve the performance of the HugeGraph Loader tool
<sup>[[1]](https://github.com/apache/incubator-hugegraph-doc/blob/master/content/en/docs/quickstart/hugegraph-loader.md)</sup><sup>[[2]](https://github.com/apache/incubator-hugegraph-toolchain/blob/master/hugegraph-loader/src/main/java/org/apache/hugegraph/loader/executor/LoadOptions.java)</sup>.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
