Hey guys, We have upgraded to Oak 1.6.5 (AEM 6.3) and we have noticed that the offline compaction is a lot slower than it was initially in AEM 6.1. While we had compaction of 5-15 mins for 15GB at first, it now takes half an hour. Is there any performance tweaking / parameters we can use to speed it up? Below you can find the script we use to do daily offline compaction and a log file of one of those times. Usually it takes away one GB per day. Seeing as we have the possibility to take down the system once a night, we don't have to solely rely on online compaction.
Another question I have with this, would it be ok that we remove the .bak files
automatically after the compaction has ran? Or why do they stay in the folder?
Script:
```
#!/bin/bash
# AEM config
AEM_PATH="/opt/aem/author"
AEM_USER="aem-user"
CRX_QUICKSTART_PATH="${AEM_PATH}/crx-quickstart"
REPOSITORY_PATH="${CRX_QUICKSTART_PATH}/repository"
SEGMENTSTORE_PATH="${REPOSITORY_PATH}/segmentstore"
AEM_STATUS="${CRX_QUICKSTART_PATH}/bin/status"
AEM_START="${CRX_QUICKSTART_PATH}/bin/start"
AEM_STOP="${CRX_QUICKSTART_PATH}/bin/stop"
# OAK Run config
OAK_RUN_JAR="/opt/aem_dependencies/aem/oak-run-1.6.5.jar"
HEAPSIZE="-Xmx12228M"
UPDATE_LIMIT="-Dupdate.limit=10000"
COMPRESS_INTERVAL="-Dcompress-interval=100000"
COMPACTION_PROGRESS_LOG="-Dcompaction-progress-log=150000"
MEMORY_MAPPED="-Dtar.memoryMapped=true"
OAK_RUN_PARAMETERS="${HEAPSIZE} ${UPDATE_LIMIT} ${COMPRESS_INTERVAL}
${COMPACTION_PROGRESS_LOG} ${MEMORY_MAPPED}"
OAK_RUN="java ${OAK_RUN_PARAMETERS} -jar ${OAK_RUN_JAR}"
if ! [ "$(whoami)" = "${AEM_USER}" ]
then
echo "Error: Maintenance task needs to be run as AEM user (${AEM_USER})."
exit 1
fi
echo "==> Starting maintenance ($(date))"
echo "==> Make sure AEM is not running"
if ${AEM_STATUS}
then
echo "==> Stopping AEM.."
${AEM_STOP}
fi
echo "==> Cleanup unrefrenced checkpoints"
${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} || exit 1
${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} rm-unreferenced || exit 1
echo "==> Compact TAR segmentstore"
${OAK_RUN} compact ${SEGMENTSTORE_PATH} || exit 1
echo "==> Starting AEM"
${AEM_START}
echo "==> Maintenance completed ($(date))"
exit 0
```
Logs:
```
==> Stopping AEM..
05.03.2018 04:00:01.716 *INFO * [main] Setting sling.home=. (command line)
05.03.2018 04:00:01.743 *INFO * [main] Sent 'stop' to /127.0.0.1:45726: OK
Application not running
==> Cleanup unrefrenced checkpoints
Apache Jackrabbit Oak 1.6.5
Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
- 3f22ad0f-3b78-4764-a697-c323f6d95180 created 2018-03-05 04:26:30.67 expires
2020-11-29 04:26:30.67
- 5caca09a-9d6b-426d-8166-e967ca1fbf7e created 2018-03-05 04:26:30.567 expires
2020-11-29 04:26:30.567
Found 2 checkpoints
Apache Jackrabbit Oak 1.6.5
Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
Referenced checkpoint from /:async@async is 3f22ad0f-3b78-4764-a697-c323f6d95180
Referenced checkpoint from /:async@fulltext-async is
5caca09a-9d6b-426d-8166-e967ca1fbf7e
Removed 0 checkpoints in 44ms.
==> Compact TAR segmentstore
Apache Jackrabbit Oak 1.6.5
Compacting /opt/aem/author/crx-quickstart/repository/segmentstore
With default access mode
before
Sun Mar 04 04:48:27 CET 2018, data05195a.tar
Sun Mar 04 04:51:46 CET 2018, data05213a.tar
Sun Mar 04 04:37:16 CET 2018, data05173a.tar
Sun Mar 04 04:51:22 CET 2018, data05211a.tar
Sun Mar 04 04:43:13 CET 2018, data05182a.tar
Sun Mar 04 04:55:57 CET 2018, data05226a.tar
Mon Mar 05 03:00:57 CET 2018, data05229a.tar
Sun Mar 04 04:36:03 CET 2018, data05171a.tar
Sun Mar 04 04:52:42 CET 2018, data05218a.tar
Sun Mar 04 04:34:42 CET 2018, data05169a.tar
Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
Sun Mar 04 04:34:07 CET 2018, data05168a.tar
Sun Mar 04 04:49:31 CET 2018, data05201a.tar
Sun Mar 04 04:53:31 CET 2018, data05222a.tar
Sun Mar 04 06:11:04 CET 2018, data05228a.tar
Sun Mar 04 04:46:31 CET 2018, data05188a.tar
Sun Mar 04 04:50:08 CET 2018, data05204a.tar
Sun Mar 04 04:48:36 CET 2018, data05196a.tar
Thu Dec 14 07:14:54 CET 2017, repo.lock
Sun Mar 04 04:43:54 CET 2018, data05183a.tar
Sun Mar 04 04:49:08 CET 2018, data05199a.tar
Sun Mar 04 04:49:43 CET 2018, data05202a.tar
Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
Sun Mar 04 04:41:19 CET 2018, data05179a.tar
Mon Mar 05 04:26:34 CET 2018, journal.log
Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
Sun Mar 04 04:53:55 CET 2018, data05224a.tar
Sun Mar 04 04:49:55 CET 2018, data05203a.tar
Sun Mar 04 04:53:03 CET 2018, data05220a.tar
Mon Mar 05 04:26:45 CET 2018, manifest
Sun Mar 04 04:49:20 CET 2018, data05200a.tar
Sun Mar 04 04:42:30 CET 2018, data05181a.tar
Sun Mar 04 04:45:37 CET 2018, data05186a.tar
Sun Mar 04 04:55:57 CET 2018, data05227a.tar
Sun Mar 04 04:48:59 CET 2018, data05198a.tar
Sun Mar 04 04:37:39 CET 2018, data05174a.tar
Sun Mar 04 04:52:19 CET 2018, data05216a.tar
Sun Mar 04 04:47:43 CET 2018, data05191a.tar
Sun Mar 04 04:48:47 CET 2018, data05197a.tar
Sun Mar 04 04:45:01 CET 2018, data05185a.tar
Sun Mar 04 04:38:20 CET 2018, data05175a.tar
Sun Mar 04 04:50:42 CET 2018, data05207a.tar
Sun Mar 04 04:40:31 CET 2018, data05178a.tar
Sun Mar 04 04:52:30 CET 2018, data05217a.tar
Sun Mar 04 04:46:56 CET 2018, data05189a.tar
Sun Mar 04 04:39:05 CET 2018, data05176a.tar
Sun Mar 04 04:35:12 CET 2018, data05170a.tar
Sun Mar 04 04:50:33 CET 2018, data05206a.tar
Sun Mar 04 04:47:22 CET 2018, data05190a.tar
Sun Mar 04 04:51:34 CET 2018, data05212a.tar
Sun Mar 04 04:54:25 CET 2018, data05225a.tar
Sun Mar 04 04:36:55 CET 2018, data05172a.tar
Sun Mar 04 04:50:20 CET 2018, data05205a.tar
Sun Mar 04 04:51:58 CET 2018, data05214a.tar
Mon Mar 05 04:26:41 CET 2018, data05231a.tar
Mon Mar 05 04:19:14 CET 2018, data05230a.tar
Sun Mar 04 04:53:43 CET 2018, data05223a.tar
Sun Mar 04 04:44:31 CET 2018, data05184a.tar
Sun Mar 04 04:47:53 CET 2018, data05192a.tar
Sun Mar 04 04:39:47 CET 2018, data05177a.tar
Sun Mar 04 04:48:17 CET 2018, data05194a.tar
Sun Mar 04 04:52:09 CET 2018, data05215a.tar
Sun Mar 04 04:48:05 CET 2018, data05193a.tar
Sun Mar 04 04:52:52 CET 2018, data05219a.tar
Sun Mar 04 04:53:20 CET 2018, data05221a.tar
Sun Mar 04 04:50:52 CET 2018, data05208a.tar
Sun Mar 04 04:46:07 CET 2018, data05187a.tar
Sun Mar 04 04:51:12 CET 2018, data05210a.tar
Sun Mar 04 04:56:04 CET 2018, gc.log
Sun Mar 04 04:33:34 CET 2018, data05167a.tar
Sun Mar 04 04:41:47 CET 2018, data05180a.tar
Sun Mar 04 04:51:02 CET 2018, data05209a.tar
size 17.1 GB (17094875037 bytes)
-> compacting
04:26:46.520 [main] WARN o.a.jackrabbit.oak.segment.Compactor - Deprecated
argument 'compress-interval', please use 'oak.segment.compaction.cacheSize'
instead.
-> cleaning up
-> writing new journal.log: bf675b66-e8e6-4bf4-a982-dc7316323e9c:859 root
1520222043417
after
Mon Mar 05 04:51:23 CET 2018, data05285a.tar
Mon Mar 05 04:36:53 CET 2018, data05244a.tar
Mon Mar 05 04:48:54 CET 2018, data05276a.tar
Mon Mar 05 04:43:10 CET 2018, data05254a.tar
Mon Mar 05 04:53:55 CET 2018, data05291a.tar
Mon Mar 05 04:48:31 CET 2018, data05274a.tar
Mon Mar 05 04:50:47 CET 2018, data05283a.tar
Mon Mar 05 04:40:27 CET 2018, data05249a.tar
Mon Mar 05 04:52:22 CET 2018, data05288a.tar
Mon Mar 05 04:49:30 CET 2018, data05278a.tar
Mon Mar 05 04:28:56 CET 2018, data05233a.tar
Mon Mar 05 04:31:18 CET 2018, data05236a.tar
Mon Mar 05 04:52:02 CET 2018, data05287a.tar
Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
Mon Mar 05 04:47:51 CET 2018, data05272a.tar
Mon Mar 05 04:36:02 CET 2018, data05243a.tar
Mon Mar 05 04:35:14 CET 2018, data05242a.tar
Mon Mar 05 04:42:46 CET 2018, data05253a.tar
Mon Mar 05 04:44:55 CET 2018, data05260a.tar
Mon Mar 05 04:39:07 CET 2018, data05247a.tar
Mon Mar 05 04:46:50 CET 2018, data05268a.tar
Mon Mar 05 04:44:06 CET 2018, data05256a.tar
Mon Mar 05 04:45:33 CET 2018, data05263a.tar
Mon Mar 05 04:53:22 CET 2018, data05290a.tar
Thu Dec 14 07:14:54 CET 2017, repo.lock
Mon Mar 05 04:50:19 CET 2018, data05281a.tar
Mon Mar 05 04:49:13 CET 2018, data05277a.tar
Mon Mar 05 04:45:51 CET 2018, data05264a.tar
Mon Mar 05 04:33:40 CET 2018, data05240a.tar
Mon Mar 05 04:46:08 CET 2018, data05265a.tar
Mon Mar 05 04:32:55 CET 2018, data05239a.tar
Mon Mar 05 04:41:04 CET 2018, data05250a.tar
Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
Mon Mar 05 04:45:18 CET 2018, data05262a.tar
Mon Mar 05 04:30:20 CET 2018, data05235a.tar
Mon Mar 05 04:44:44 CET 2018, data05259a.tar
Mon Mar 05 04:51:43 CET 2018, data05286a.tar
Mon Mar 05 04:54:03 CET 2018, journal.log
Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
Mon Mar 05 04:32:32 CET 2018, data05238a.tar
Mon Mar 05 04:28:18 CET 2018, data05232a.tar
Mon Mar 05 04:45:07 CET 2018, data05261a.tar
Mon Mar 05 04:47:38 CET 2018, data05271a.tar
Mon Mar 05 04:49:48 CET 2018, data05279a.tar
Mon Mar 05 04:53:56 CET 2018, manifest
Mon Mar 05 04:34:30 CET 2018, data05241a.tar
Mon Mar 05 04:46:20 CET 2018, data05266a.tar
Mon Mar 05 04:43:41 CET 2018, data05255a.tar
Mon Mar 05 04:38:21 CET 2018, data05246a.tar
Mon Mar 05 04:48:14 CET 2018, data05273a.tar
Mon Mar 05 04:52:50 CET 2018, data05289a.tar
Mon Mar 05 04:42:19 CET 2018, data05252a.tar
Mon Mar 05 04:41:47 CET 2018, data05251a.tar
Mon Mar 05 04:47:07 CET 2018, data05269a.tar
Mon Mar 05 04:50:34 CET 2018, data05282a.tar
Mon Mar 05 04:47:21 CET 2018, data05270a.tar
Mon Mar 05 04:39:50 CET 2018, data05248a.tar
Mon Mar 05 04:29:40 CET 2018, data05234a.tar
Mon Mar 05 04:44:19 CET 2018, data05257a.tar
Mon Mar 05 04:44:30 CET 2018, data05258a.tar
Mon Mar 05 04:32:13 CET 2018, data05237a.tar
Mon Mar 05 04:48:43 CET 2018, data05275a.tar
Mon Mar 05 04:50:05 CET 2018, data05280a.tar
Mon Mar 05 04:37:30 CET 2018, data05245a.tar
Mon Mar 05 04:51:02 CET 2018, data05284a.tar
Mon Mar 05 04:46:33 CET 2018, data05267a.tar
Mon Mar 05 04:54:03 CET 2018, gc.log
size 16.1 GB (16081145586 bytes)
removed files [data05228a.tar, data05222a.tar, data05167a.tar,
data05184a.tar, data05214a.tar, data05192a.tar, data05200a.tar, data05198a.tar,
data05183a.tar, data05189a.tar, data05205a.tar, data05175a.tar, data05230a.tar,
data05206a.tar, data05191a.tar, data05197a.tar, data05174a.tar, data05177a.tar,
data05188a.tar, data05171a.tar, data05182a.tar, data05169a.tar, data05224a.tar,
data05227a.tar, data05221a.tar, data05213a.tar, data05193a.tar, data05185a.tar,
data05190a.tar, data05216a.tar, data05219a.tar, data05196a.tar, data05199a.tar,
data05195a.tar, data05211a.tar, data05187a.tar, data05203a.tar, data05208a.tar,
data05186a.tar, data05172a.tar, data05180a.tar, data05202a.tar, data05210a.tar,
data05178a.tar, data05209a.tar, data05217a.tar, data05225a.tar, data05194a.tar,
data05207a.tar, data05218a.tar, data05229a.tar, data05220a.tar, data05168a.tar,
data05223a.tar, data05226a.tar, data05179a.tar, data05176a.tar, data05231a.tar,
data05170a.tar, data05215a.tar, data05201a.tar, data05181a.tar, data05173a.tar,
data05204a.tar, data05212a.tar]
added files [data05259a.tar, data05267a.tar, data05245a.tar,
data05237a.tar, data05253a.tar, data05261a.tar, data05283a.tar, data05244a.tar,
data05258a.tar, data05291a.tar, data05236a.tar, data05275a.tar, data05260a.tar,
data05252a.tar, data05289a.tar, data05232a.tar, data05246a.tar, data05243a.tar,
data05238a.tar, data05235a.tar, data05240a.tar, data05279a.tar, data05262a.tar,
data05268a.tar, data05276a.tar, data05251a.tar, data05284a.tar, data05270a.tar,
data05254a.tar, data05273a.tar, data05281a.tar, data05265a.tar, data05249a.tar,
data05257a.tar, data05250a.tar, data05256a.tar, data05264a.tar, data05239a.tar,
data05287a.tar, data05272a.tar, data05242a.tar, data05248a.tar, data05241a.tar,
data05280a.tar, data05247a.tar, data05255a.tar, data05278a.tar, data05233a.tar,
data05286a.tar, data05271a.tar, data05274a.tar, data05288a.tar, data05263a.tar,
data05266a.tar, data05285a.tar, data05290a.tar, data05277a.tar, data05282a.tar,
data05269a.tar, data05234a.tar]
Compaction succeeded in 27.29 min (1637s).
```
Greets,
Roy
signature.asc
Description: Message signed with OpenPGP
