[
https://issues.apache.org/jira/browse/VELOCITY-715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Atsushi Isobe updated VELOCITY-715:
-----------------------------------
Description:
org.apache.velocity.runtime.resource.loader.FileResourceLoader#templatePaths
has stored templatename and real file path permanently.So if
VelocityEngine#getTemplate(String) has called by too many files, #templatePaths
have been enlarged.
Bellow my test results.
BEFORE
----------------------------------------
2009-05-08 19:16:54,571 INFO [FileResourceLoaderLeakedObjectTest] count:
000,000 total: 002,696,488
2009-05-08 19:19:23,381 INFO [FileResourceLoaderLeakedObjectTest] count:
065,536 total: 014,689,896
2009-05-08 19:22:03,782 INFO [FileResourceLoaderLeakedObjectTest] count:
131,072 total: 025,374,944
2009-05-08 19:24:14,756 INFO [FileResourceLoaderLeakedObjectTest] count:
196,608 total: 036,282,696
2009-05-08 19:26:37,864 INFO [FileResourceLoaderLeakedObjectTest] count:
262,144 total: 046,776,168
2009-05-08 19:28:55,090 INFO [FileResourceLoaderLeakedObjectTest] count:
327,680 total: 057,119,560
2009-05-08 19:31:37,524 INFO [FileResourceLoaderLeakedObjectTest] count:
393,216 total: 069,858,824
2009-05-08 19:34:10,425 INFO [FileResourceLoaderLeakedObjectTest] count:
458,752 total: 080,019,920
2009-05-08 19:36:52,788 INFO [FileResourceLoaderLeakedObjectTest] count:
524,288 total: 090,093,168
2009-05-08 19:39:13,662 INFO [FileResourceLoaderLeakedObjectTest] count:
589,824 total: 097,385,776
2009-05-08 19:41:54,621 INFO [FileResourceLoaderLeakedObjectTest] count:
655,360 total: 107,387,792
2009-05-08 19:44:31,601 INFO [FileResourceLoaderLeakedObjectTest] count:
720,896 total: 117,451,912
2009-05-08 19:47:08,890 INFO [FileResourceLoaderLeakedObjectTest] count:
786,432 total: 137,151,792
2009-05-08 19:49:44,924 INFO [FileResourceLoaderLeakedObjectTest] count:
851,968 total: 147,277,856
AFTER
----------------------------------------
2009-05-08 19:53:54,463 INFO [FileResourceLoaderLeakedObjectTest] count:
000,000 total: 002,697,096
2009-05-08 19:56:32,389 INFO [FileResourceLoaderLeakedObjectTest] count:
065,536 total: 005,510,968
2009-05-08 19:59:00,548 INFO [FileResourceLoaderLeakedObjectTest] count:
131,072 total: 005,221,008
2009-05-08 20:01:36,114 INFO [FileResourceLoaderLeakedObjectTest] count:
196,608 total: 007,527,856
2009-05-08 20:04:04,294 INFO [FileResourceLoaderLeakedObjectTest] count:
262,144 total: 008,880,192
2009-05-08 20:06:30,697 INFO [FileResourceLoaderLeakedObjectTest] count:
327,680 total: 007,549,696
2009-05-08 20:08:39,257 INFO [FileResourceLoaderLeakedObjectTest] count:
393,216 total: 012,251,648
2009-05-08 20:10:58,604 INFO [FileResourceLoaderLeakedObjectTest] count:
458,752 total: 012,101,344
2009-05-08 20:13:12,704 INFO [FileResourceLoaderLeakedObjectTest] count:
524,288 total: 013,593,888
2009-05-08 20:15:30,824 INFO [FileResourceLoaderLeakedObjectTest] count:
589,824 total: 012,101,344
2009-05-08 20:17:45,941 INFO [FileResourceLoaderLeakedObjectTest] count:
655,360 total: 013,655,456
2009-05-08 20:19:58,246 INFO [FileResourceLoaderLeakedObjectTest] count:
720,896 total: 012,096,672
2009-05-08 20:22:13,234 INFO [FileResourceLoaderLeakedObjectTest] count:
786,432 total: 021,538,536
2009-05-08 20:24:33,981 INFO [FileResourceLoaderLeakedObjectTest] count:
851,968 total: 023,973,896
2009-05-08 20:26:55,324 INFO [FileResourceLoaderLeakedObjectTest] count:
917,504 total: 021,583,728
2009-05-08 20:29:13,956 INFO [FileResourceLoaderLeakedObjectTest] count:
983,040 total: 021,533,464
was:
org.apache.velocity.runtime.resource.loader.FileResourceLoader#templatePaths
has stored templatename and real file path permanently.So if
VelocityEngine#getTemplate(String) has called by too many files, #templatePaths
have been enlarged.
Below my patch and test class.
---------------------------------------------------------------------------
---
./src/java/org/apache/velocity/runtime/resource/loader/FileResourceLoader.java
(revision 772904)
+++
./src/java/org/apache/velocity/runtime/resource/loader/FileResourceLoader.java
(working copy)
@@ -30,6 +30,7 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.WeakHashMap;
import org.apache.commons.collections.ExtendedProperties;
import org.apache.velocity.exception.ResourceNotFoundException;
@@ -61,7 +62,7 @@
* times of the files. This is synchronizedMap
* instance.
*/
- private Map templatePaths = Collections.synchronizedMap(new HashMap());
+ private Map templatePaths = Collections.synchronizedMap(new WeakHashMap());
/** Shall we inspect unicode files to see what encoding they contain?. */
private boolean unicode = false;
@@ -330,8 +331,8 @@
currentFile = testFile;
}
}
- File file = getFile(path, fileName);
- if (currentFile == null || !file.exists())
+ File file = path != null ? getFile(path, fileName) : null;
+ if (currentFile == null || file == null || !file.exists())
{
/*
* noop: if the file is missing now (either the cached
@@ -365,6 +366,10 @@
public long getLastModified(Resource resource)
{
String path = (String) templatePaths.get(resource.getName());
+ if (path == null)
+ {
+ return 0;
+ }
File file = getFile(path, resource.getName());
if (file.canRead())
---------------------------------------------------------------------------
import java.io.File;
import java.io.IOException;
import java.io.StringWriter;
import java.text.DecimalFormat;
import java.util.Properties;
import org.apache.commons.io.FileUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.velocity.Template;
import org.apache.velocity.VelocityContext;
import org.apache.velocity.app.Velocity;
import org.apache.velocity.app.VelocityEngine;
import org.apache.velocity.exception.MethodInvocationException;
import org.apache.velocity.exception.ParseErrorException;
import org.apache.velocity.exception.ResourceNotFoundException;
public class FileResourceLoaderLeakedObjectTest {
private static final String TEMPLATE_ENCODING = "UTF-8";
private static final File TEMPLATE_DIRECTORY = new File("./");
private static final String TEMPLATE_PREFIX = "testTemplate";
private static final String TEMPLATE_SUFFIX = ".vm";
private static final String TEMPLATE_STRING = "foobar";
private static final Log log =
LogFactory.getLog(FileResourceLoaderLeakedObjectTest.class);
public static void main(String[] args) throws Exception {
VelocityEngine ve = new VelocityEngine();
Properties properties = new Properties();
properties.put(Velocity.FILE_RESOURCE_LOADER_PATH,
TEMPLATE_DIRECTORY.getPath());
ve.init(properties);
VelocityContext context = new VelocityContext();
DecimalFormat countFormat = new DecimalFormat("000,000");
DecimalFormat totalFormat = new DecimalFormat("000,000,000");
StringWriter writer = new StringWriter();
File tmp = null;
for (int i = 0; i < 0x100000; i++) {
try {
tmp = File.createTempFile(TEMPLATE_PREFIX,
TEMPLATE_SUFFIX, TEMPLATE_DIRECTORY);
FileUtils.writeStringToFile(tmp,
TEMPLATE_STRING, TEMPLATE_ENCODING);
Template template =
ve.getTemplate(tmp.getPath(), TEMPLATE_ENCODING);
template.merge(context, writer);
if ((i & 0xFFFF) == 0) {
System.gc();
Runtime runtime = Runtime.getRuntime();
log.info(new StringBuilder("count:
").append(countFormat.format(i)).append(" total:
").append(totalFormat.format(runtime.totalMemory() - runtime.freeMemory())));
}
} catch (ResourceNotFoundException e) {
log.warn("", e);
} catch (ParseErrorException e) {
log.warn("", e);
} catch (MethodInvocationException e) {
log.warn("", e);
} catch (IOException e) {
log.warn("", e);
} catch (Exception e) {
log.warn("", e);
} finally {
if (tmp != null && tmp.exists()) {
tmp.delete();
}
}
}
}
}
---------------------------------------------------------------------------
Environment: (was: Sun JRE 6 Update 13)
> Leaked object in FileResourceLoader#templatePaths
> -------------------------------------------------
>
> Key: VELOCITY-715
> URL: https://issues.apache.org/jira/browse/VELOCITY-715
> Project: Velocity
> Issue Type: Bug
> Components: Engine
> Reporter: Atsushi Isobe
> Priority: Minor
> Attachments: FileResourceLoader.patch,
> FileResourceLoaderLeakedObjectTest.java
>
>
> org.apache.velocity.runtime.resource.loader.FileResourceLoader#templatePaths
> has stored templatename and real file path permanently.So if
> VelocityEngine#getTemplate(String) has called by too many files,
> #templatePaths have been enlarged.
> Bellow my test results.
> BEFORE
> ----------------------------------------
> 2009-05-08 19:16:54,571 INFO [FileResourceLoaderLeakedObjectTest] count:
> 000,000 total: 002,696,488
> 2009-05-08 19:19:23,381 INFO [FileResourceLoaderLeakedObjectTest] count:
> 065,536 total: 014,689,896
> 2009-05-08 19:22:03,782 INFO [FileResourceLoaderLeakedObjectTest] count:
> 131,072 total: 025,374,944
> 2009-05-08 19:24:14,756 INFO [FileResourceLoaderLeakedObjectTest] count:
> 196,608 total: 036,282,696
> 2009-05-08 19:26:37,864 INFO [FileResourceLoaderLeakedObjectTest] count:
> 262,144 total: 046,776,168
> 2009-05-08 19:28:55,090 INFO [FileResourceLoaderLeakedObjectTest] count:
> 327,680 total: 057,119,560
> 2009-05-08 19:31:37,524 INFO [FileResourceLoaderLeakedObjectTest] count:
> 393,216 total: 069,858,824
> 2009-05-08 19:34:10,425 INFO [FileResourceLoaderLeakedObjectTest] count:
> 458,752 total: 080,019,920
> 2009-05-08 19:36:52,788 INFO [FileResourceLoaderLeakedObjectTest] count:
> 524,288 total: 090,093,168
> 2009-05-08 19:39:13,662 INFO [FileResourceLoaderLeakedObjectTest] count:
> 589,824 total: 097,385,776
> 2009-05-08 19:41:54,621 INFO [FileResourceLoaderLeakedObjectTest] count:
> 655,360 total: 107,387,792
> 2009-05-08 19:44:31,601 INFO [FileResourceLoaderLeakedObjectTest] count:
> 720,896 total: 117,451,912
> 2009-05-08 19:47:08,890 INFO [FileResourceLoaderLeakedObjectTest] count:
> 786,432 total: 137,151,792
> 2009-05-08 19:49:44,924 INFO [FileResourceLoaderLeakedObjectTest] count:
> 851,968 total: 147,277,856
> AFTER
> ----------------------------------------
> 2009-05-08 19:53:54,463 INFO [FileResourceLoaderLeakedObjectTest] count:
> 000,000 total: 002,697,096
> 2009-05-08 19:56:32,389 INFO [FileResourceLoaderLeakedObjectTest] count:
> 065,536 total: 005,510,968
> 2009-05-08 19:59:00,548 INFO [FileResourceLoaderLeakedObjectTest] count:
> 131,072 total: 005,221,008
> 2009-05-08 20:01:36,114 INFO [FileResourceLoaderLeakedObjectTest] count:
> 196,608 total: 007,527,856
> 2009-05-08 20:04:04,294 INFO [FileResourceLoaderLeakedObjectTest] count:
> 262,144 total: 008,880,192
> 2009-05-08 20:06:30,697 INFO [FileResourceLoaderLeakedObjectTest] count:
> 327,680 total: 007,549,696
> 2009-05-08 20:08:39,257 INFO [FileResourceLoaderLeakedObjectTest] count:
> 393,216 total: 012,251,648
> 2009-05-08 20:10:58,604 INFO [FileResourceLoaderLeakedObjectTest] count:
> 458,752 total: 012,101,344
> 2009-05-08 20:13:12,704 INFO [FileResourceLoaderLeakedObjectTest] count:
> 524,288 total: 013,593,888
> 2009-05-08 20:15:30,824 INFO [FileResourceLoaderLeakedObjectTest] count:
> 589,824 total: 012,101,344
> 2009-05-08 20:17:45,941 INFO [FileResourceLoaderLeakedObjectTest] count:
> 655,360 total: 013,655,456
> 2009-05-08 20:19:58,246 INFO [FileResourceLoaderLeakedObjectTest] count:
> 720,896 total: 012,096,672
> 2009-05-08 20:22:13,234 INFO [FileResourceLoaderLeakedObjectTest] count:
> 786,432 total: 021,538,536
> 2009-05-08 20:24:33,981 INFO [FileResourceLoaderLeakedObjectTest] count:
> 851,968 total: 023,973,896
> 2009-05-08 20:26:55,324 INFO [FileResourceLoaderLeakedObjectTest] count:
> 917,504 total: 021,583,728
> 2009-05-08 20:29:13,956 INFO [FileResourceLoaderLeakedObjectTest] count:
> 983,040 total: 021,533,464
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
