[
https://issues.apache.org/jira/browse/GOBBLIN-1492?focusedWorklogId=630759&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630759
]
ASF GitHub Bot logged work on GOBBLIN-1492:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 28/Jul/21 20:17
Start Date: 28/Jul/21 20:17
Worklog Time Spent: 10m
Work Description: sv2000 commented on a change in pull request #3335:
URL: https://github.com/apache/gobblin/pull/3335#discussion_r678622001
##########
File path:
gobblin-utility/src/main/java/org/apache/gobblin/util/ConfigUtils.java
##########
@@ -116,7 +116,8 @@ public static Properties configToProperties(Config config,
Optional<String> pref
Config resolvedConfig = config.resolve();
for (Map.Entry<String, ConfigValue> entry : resolvedConfig.entrySet()) {
if (!prefix.isPresent() || entry.getKey().startsWith(prefix.get())) {
- String propKey = desanitizeKey(entry.getKey());
+ // Intern the string so that constant keys are not duplicated to
save memory
+ String propKey = desanitizeKey(entry.getKey()).intern();
Review comment:
@Will-Lo This method is invoked in a number of places and not limited to
Gobblin service. One concern is that the number of keys may not be as small as
you imagine. In production settings, Gobblin jobs load system configs set by
AZ/Hadoop or even the host where a job/container is running. These can be
non-trivial in number. 60k may be sufficient. But what happens if the number of
config keys exceed this value? Will the JVM just crash?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 630759)
Time Spent: 3h (was: 2h 50m)
> Optimize memory usage for Gobblin as a Service
> ----------------------------------------------
>
> Key: GOBBLIN-1492
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1492
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-service
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Gobblin as a Service, under high loads, spends a large amount of time garbage
> collecting. This will track a number of changes relating to optimizing the
> memory usage
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
