[
https://issues.apache.org/jira/browse/DRILL-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4266:
------------------------------------
Attachment:
memComsumption_framework.output_Sun_Jan_17_04_jacques_branch_drill-4131
Attached are results of the same test executed with drill built from branch:
{code}
Generated by Git-Commit-Id-Plugin
#Sun Jan 17 03:49:25 UTC 2016
git.commit.id.abbrev=daa89a6
git.commit.user.email=jacq...@apache.org
git.commit.message.full=DRILL-4131\: Move RPC allocators under Drill's root
allocator & accounting\n\n- Allow settings to be set to ensure RPC reservation
and maximums (currently unset by default). Defaults set in drill-module.conf\n-
Add new metrics to report RPC layer memory consumption.\n- Check for memory
leaks from RPC layer at shutdown.\n
git.commit.id=daa89a68aa416f8718ad005bc57baf4ac58a9c66
git.commit.message.short=DRILL-4131\: Move RPC allocators under Drill's root
allocator & accounting
git.commit.user.name=Jacques Nadeau
git.build.user.name=vmarkman
git.commit.id.describe=0.9.0-560-gdaa89a6
git.build.user.email=vmark...@maprtech.com
git.branch=daa89a68aa416f8718ad005bc57baf4ac58a9c66
git.commit.time=16.01.2016 @ 00\:04\:59 UTC
git.build.time=17.01.2016 @ 03\:49\:25 UTC
git.remote.origin.url=g...@github.com\:jacques-n/drill.git
{code}
> Possible memory leak (fragmentation ?) in rpc layer
> ----------------------------------------------------
>
> Key: DRILL-4266
> URL: https://issues.apache.org/jira/browse/DRILL-4266
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - RPC
> Affects Versions: 1.5.0
> Reporter: Victoria Markman
> Assignee: Jacques Nadeau
> Attachments: drill.log.2016-01-12-16, memComsumption.txt,
> memComsumption_framework.output_Fri_Jan_15_width_per_node=4.log,
> memComsumption_framework.output_Sun_Jan_17_04_jacques_branch_drill-4131,
> test.tar
>
>
> I have executed 5 tests from Advanced/mondrian test suite in a loop overnight.
> My observation is that direct memory steadily grew from 117MB to 1.8GB and
> remained on that level for 14875 iteration of the tests.
> My question is: why do 5 queries that were able to execute with 117MB of
> memory require 1.8GB of memory after 5 hours of execution ?
> Attached:
> * Memory used after each test iteration : memComsumption.txt
> * Log of the framework run: drill.log.2016-01-12-16
> * Tests: test.tar
> Setup:
> {noformat}
> Single node 32 core box.
> DRILL_MAX_DIRECT_MEMORY="4G"
> DRILL_HEAP="1G"
> 0: jdbc:drill:schema=dfs> select * from sys.options where status like
> '%CHANGED%';
> +-----------------------------------+----------+---------+----------+----------+-------------+-----------+------------+
> | name | kind | type | status | num_val
> | string_val | bool_val | float_val |
> +-----------------------------------+----------+---------+----------+----------+-------------+-----------+------------+
> | planner.enable_decimal_data_type | BOOLEAN | SYSTEM | CHANGED | null
> | null | true | null |
> +-----------------------------------+----------+---------+----------+----------+-------------+-----------+------------+
> 1 row selected (1.309 seconds)
> {noformat}
> {noformat}
> Reproduction:
> * tar xvf test.tar into Functional/test directory
> * ./run.sh -s Functional/test -g regression -t 180 -n 5 -i 10000000 -m
> {noformat}
> This is very similar behavior as Hakim and I observed long time ago with
> window functions. Now, that new allocator is in place we rerun this test and
> we see the similar things, and allocator does not seem to think that we have
> a memory leak. Hence the speculation that memory is leaked in RPC layer.
> I'm going to reduce planner.width.max_per_node and see if it has any effect
> on memory allocation (speculating again ...)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
