[
https://issues.apache.org/jira/browse/ARTEMIS-2746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Nigro updated ARTEMIS-2746:
-------------------------------------
Description:
While loading the journal, before reloading the messages into each queue, we
store the reference to the message and some additional information
(scheduledDeliveryTime and deliveryCount) in a LinkedHashMap per-queue.
These information are using a AddMessageRecord class that can weigth 28 bytes:
12 object header + 4 for Message + 8 for scheduledDeliveryTime + 4
deliveryCount + 4 padding on a JDK 64 bits with COOPS.
Often both scheduledDeliveryTime and deliveryCount are not used, hence we could
save 12 bytes per-message by using a separate data-structure to hold such
additional records.
Such records could be stored in a separate LongObjectHasMap that would cost
approximately (it depends on the load factor) per-entry:
8 (for the messageID key) + 4 (for the additional record reference) + 16 (for
the record instance) = 28 bytes
In the worst case, given that the additional informations require the same
amount of memory (28 bytes), the only downside would be some lookup cost while
walking/updating both the data-structures, but in the best case (and most
common cases) it means eg
100_000 elements = saved ~2.8 MB
1_000_000 elements = saved ~28 MB
10_000_000 elements = saved ~280 MB
was:
While loading the journal, before reloading the messages into each queue, we
store the reference to the message and some additional information
(scheduledDeliveryTime and deliveryCount) in a LinkedHashMap per-queue.
These information are using a AddMessageRecord class that can weigth 28 bytes:
12 object header + 4 for Message + 8 for scheduledDeliveryTime + 4
deliveryCount + 4 padding on a JDK 64 bits bits with COOPS.
Often both scheduledDeliveryTime and deliveryCount are not used, hence we could
save 12 bytes per-message by using a separate data-structure to hold such
additional records.
Such records could be stored in a separate LongObjectHasMap that would cost
approximately (it depends on the load factor) per-entry:
8 (for the messageID key) + 4 (for the additional record reference) + 16 (for
the record instance) = 28 bytes
In the worst case, given that the additional informations require the same
amount of memory (28 bytes), the only downside would be some lookup cost while
walking/updating both the data-structures, but in the best case (and most
common cases) it means eg
100_000 elements = saved ~2.8 MB
1_000_000 elements = saved ~28 MB
10_000_000 elements = saved ~280 MB
> Reduce memory footprint of journal loading
> ------------------------------------------
>
> Key: ARTEMIS-2746
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2746
> Project: ActiveMQ Artemis
> Issue Type: Improvement
> Components: Broker
> Reporter: Francesco Nigro
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While loading the journal, before reloading the messages into each queue, we
> store the reference to the message and some additional information
> (scheduledDeliveryTime and deliveryCount) in a LinkedHashMap per-queue.
> These information are using a AddMessageRecord class that can weigth 28 bytes:
> 12 object header + 4 for Message + 8 for scheduledDeliveryTime + 4
> deliveryCount + 4 padding on a JDK 64 bits with COOPS.
> Often both scheduledDeliveryTime and deliveryCount are not used, hence we
> could save 12 bytes per-message by using a separate data-structure to hold
> such additional records.
> Such records could be stored in a separate LongObjectHasMap that would cost
> approximately (it depends on the load factor) per-entry:
> 8 (for the messageID key) + 4 (for the additional record reference) + 16 (for
> the record instance) = 28 bytes
> In the worst case, given that the additional informations require the same
> amount of memory (28 bytes), the only downside would be some lookup cost
> while walking/updating both the data-structures, but in the best case (and
> most common cases) it means eg
> 100_000 elements = saved ~2.8 MB
> 1_000_000 elements = saved ~28 MB
> 10_000_000 elements = saved ~280 MB
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
