The function uridecode used by werc performs poorly on large POSTed data, which
may be encountered in situations such as editing a 8k+ file in dirdir. The
short patch I am including causes a drastic improvement in this case while not
having a significant worsening of the worst-case. On typical text input
through dirdir, I went from noticeable processing times for 16k tests and nginx
timeouts on 64k text, to no noticeable slowdowns on 16k and reasonable
processing time on the 63k document.
What this patch does is pre-check the input for the sequence "%0A" and splits
the input into lines at that point. This works around the performance issues
uridecode has on long input for typical text data. Creating a version of this
that would improve worst-case, instead of just the expected-case, would require
a more careful approach to finding appropriate split points.
This patch has been in production for a few months in the were instance I
modify with no ill effects, and revealed an issue with crashes in dirdir when
handling 121k input documents.
A note about why I use awk in place of sed in the patch: the variant of sed
preferred by werc truncates the input line at 8k characters, awk doesn't.
===
diff -Naur old/werc-1.4.0/bin/cgilib.rc new/werc-1.4.0/bin/cgilib.rc
--- old/werc-1.4.0/bin/cgilib.rc 2011-04-23 11:53:45.000000000 -0400
+++ new/werc-1.4.0/bin/cgilib.rc 2011-04-23 11:55:12.000000000 -0400
@@ -65,7 +65,7 @@
}
fn urldecode {
-awk '
+awk '{ gsub("%0[aA]", "%0A\n"); print $0 }' | awk '
BEGIN {
hextab ["0"] = 0; hextab ["8"] = 8;
hextab ["1"] = 1; hextab ["9"] = 9;
===
--
Seneca Cunningham
[email protected]
--
You received this message because you are subscribed to the Google Groups
"werc" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/werc9?hl=en.
