[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122713#comment-14122713 ] Benedict commented on CASSANDRA-7882: - bq. or this happens with native gcc as well? This will happen with all allocators (they need to manage the memory somehow), but the profile will vary with each allocator and platform bq. should we keep the start & end region-size configurable? I think 1Kb -> 1Mb is more than enough to solve the problem. 1Kb is considerably less than the general overhead for managing a table (might even be sensible to start around 8Kb), so not a meaningful impact, and with logarithmic scaling we're guaranteed good occupancy, so we're only going to have memory pressure if there's genuine memory pressure. I'm on the fence about if the logarithmic scaling should be configurable or just always on. But it's likely sufficient to simply have an on/off config parameter, if any. If you want to have a crack at patching this, please do, and I'll review. We're always keen to expand the contributor base and it isn't a high priority for the core team right now. > Allow disabling slab allocation for off-heap memtables > -- > > Key: CASSANDRA-7882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7882 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jay Patel > Fix For: 2.1.1 > > > CASSANDRA-5935 allows option to disable region-based allocation for on-heap > memtables but there is no option to disable it for off-heap memtables > (memtable_allocation_type: offheap_objects). > Disabling region-based allocation will allow us to pack more tables in the > schema since minimum of 1MB region won't be allocated per table. Downside can > be more fragmentation which should be controllable by using better allocator > like JEMalloc. > How about below option in yaml?: > memtable_allocation_type: unslabbed_offheap_objects > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122706#comment-14122706 ] Jay Patel commented on CASSANDRA-7882: -- Is this overhead from using jemalloc allocator, or this happens with native gcc as well? BTW, I've opened a related ticket CASSANDRA-7883 to allow plugging jemalloc. And, CASSANDRA-7884 to make the region size configurable. I think logarithmically scaling from 1K to 1 MB option can also work. But, should we keep the start & end region-size configurable? Usually, this will be advanced option to tune but very helpful. By default, we can keep 1 MB region size as most use cases will not need to pack many tables. Let me know the best way to handle this, and also you can assign it to me to have it in 2.1.1. We're actually quite dependent on this as currently 1 MB is putting a upper limit on number of tables in the cluster & having multiple under-utilized clusters to support high # of tables is not cost-effective. > Allow disabling slab allocation for off-heap memtables > -- > > Key: CASSANDRA-7882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7882 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jay Patel > Fix For: 2.1.1 > > > CASSANDRA-5935 allows option to disable region-based allocation for on-heap > memtables but there is no option to disable it for off-heap memtables > (memtable_allocation_type: offheap_objects). > Disabling region-based allocation will allow us to pack more tables in the > schema since minimum of 1MB region won't be allocated per table. Downside can > be more fragmentation which should be controllable by using better allocator > like JEMalloc. > How about below option in yaml?: > memtable_allocation_type: unslabbed_offheap_objects > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122609#comment-14122609 ] Benedict commented on CASSANDRA-7882: - There's a slight issue with this approach, which is the one of the main reasons I aborted including this in 2.1, namely that there is no easy way to establish _how much memory_ is allocated by these allocators. i.e., the overheads are opaque, and likely significant. 16-byte+ overheads, with padding to nearest 8/16-bytes is quite likely, but we have absolutely no way of knowing, and it will vary. These overheads are likely to be a significant portion of the total amount allocated, and so we are unlikely to obey the configured memory constraints, which could result in an unstable server. If we want to solve the problem of many tables, there are other potential approaches, such as logarithmically scaling the size of the slab we allocate from, say, 1K for the first allocation against any memtable, up to the current 1Mb. This should prevent the overheads from being painful. > Allow disabling slab allocation for off-heap memtables > -- > > Key: CASSANDRA-7882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7882 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jay Patel > Fix For: 2.1.1 > > > CASSANDRA-5935 allows option to disable region-based allocation for on-heap > memtables but there is no option to disable it for off-heap memtables > (memtable_allocation_type: offheap_objects). > Disabling region-based allocation will allow us to pack more tables in the > schema since minimum of 1MB region won't be allocated per table. Downside can > be more fragmentation which should be controllable by using better allocator > like JEMalloc. > How about below option in yaml?: > memtable_allocation_type: unslabbed_offheap_objects > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7882) Allow disabling slab allocation for off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122602#comment-14122602 ] Jay Patel commented on CASSANDRA-7882: -- I've added code for this in my loca env, and done some testing. Reduction in allocated memory is significant with higher number of tables (1000s to 10s of thousands) in a cluster. By default region-based allocation should be always-on but providing this option will allow flexibility for packing more tables. I'm fine if we should keep it undocumented like CASSANDRA-5935. > Allow disabling slab allocation for off-heap memtables > -- > > Key: CASSANDRA-7882 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7882 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jay Patel > Fix For: 2.1.1 > > > CASSANDRA-5935 allows option to disable region-based allocation for on-heap > memtables but there is no option to disable it for off-heap memtables > (memtable_allocation_type: offheap_objects). > Disabling region-based allocation will allow us to pack more tables in the > schema since minimum of 1MB region won't be allocated per table. Downside can > be more fragmentation which should be controllable by using better allocator > like JEMalloc. > How about below option in yaml?: > memtable_allocation_type: unslabbed_offheap_objects > Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
