[
https://issues.apache.org/jira/browse/IMPALA-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-9413.
-----------------------------------
Resolution: Won't Fix
> Calculation of result set memory usage is wrong when using GCC7 with new ABI
> ----------------------------------------------------------------------------
>
> Key: IMPALA-9413
> URL: https://issues.apache.org/jira/browse/IMPALA-9413
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Priority: Major
> Attachments: stringcapacity.cc
>
>
> GCC5+ uses a new ABI for std::string which has a small string optimization.
> This allows it to avoid an extra memory allocation for strings up to 15
> characters. This means that string.capacity() is 15 while still only using
> sizeof(string), so calculations of memory usage that add sizeof(string) +
> string.capacity are no longer correct. This happens in the query result set:
> [https://github.com/apache/impala/blob/master/be/src/service/query-result-set.cc#L239-L241]
> At the moment, Impala uses GCC 4.9.2, which does not have this optimization,
> so this is only a problem when we switch to the new ABI.
> I have attached a simple c++ file to demonstrate the difference. On
> GCC-4.9.2, the output is:
> {noformat}
> joe@joemcdonnell:~/view2/Impala/stringcapacity$ ./a.out
> init short_string
> [Allocating 30 bytes]
> sizeof(short_string): 8
> short_string.size(): 5
> short_string.capacity(): 5
> init long_string
> [Allocating 54 bytes]
> sizeof(long_string): 8
> long_string.size(): 29
> long_string.capacity(): 29
> {noformat}
> On GCC 5.4.0:
> {noformat}
> init short_string
> sizeof(short_string): 32
> short_string.size(): 5
> short_string.capacity(): 15
> init long_string
> [Allocating 30 bytes]
> sizeof(long_string): 32
> long_string.size(): 29
> long_string.capacity(): 29{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
