Tom, I figured out the problem with the "memory bloat" when I used your
alternative ns_getform to parse a large file upload temp file.
The problem is that if I call ns_queryget after calling your ns_getform, that
causes aolserver to re-parse the large file upload, and to do it in the old
memory-inefficient way.
I'm not sure if there's a way for your version of ns_getform to tell the
aolserver internals that the parsing of the form is done, so subsequent
ns_queryget calls don't cause a re-parse.
At any rate, there's an easy workaround, which is to read things right out the
ns_set that your ns_getform populates.
-john
On Nov 23, 2009, at 8:47 PM, John Buckman wrote:
> Tom, there is some sort of weird interaction effect on aolserver when doing
> tcl stuff and there is a large file as the temporary file. Some cases cause
> the nsd process to bloat to the size of the memory. I wasn't able to figure
> out why. I wasn't able to fix your ns_getform to not bloat.
>
> However, I was able to copy Vlad's ns_parseformfile proc for use on aolserver
> from naviserver, and it doesn't bloat.
>
> Here is the code for handling a large file uploaded as a file, rather than in
> memory:
>
> set type [ns_set iget [ns_conn headers] content-type]
> set form [ns_set create]
> ns_parseformfile $form $type
> array set formdata [ns_set array $form]
> puts "final array: [array get formdata]"
>
>
> proc ns_parseformfile { form contentType } {
>
> set fp [ns_conn contentchannel]
>
> if { ![regexp -nocase {boundary=(.*)$} $contentType 1 b] } {
> puts "Warning: no MIME boundary"
> return
> }
>
> fconfigure $fp -encoding binary -translation binary
> set boundary "--$b"
>
> while { ![eof $fp] } {
> # skip past the next boundary line
> if { ![string match $boundary* [string trim [gets $fp]]] } {
> continue
> }
>
> # fetch the disposition line and field name
> set disposition [string trim [gets $fp]]
> if { ![string length $disposition] } {
> break
> }
>
> set disposition [split $disposition \;]
> set name [string trim [lindex [split [lindex $disposition 1] =] 1] \"]
>
> # fetch and save any field headers (usually just content-type for files)
>
> while { ![eof $fp] } {
> set line [string trim [gets $fp]]
> if { ![string length $line] } {
> break
> }
> set header [split $line :]
> set key [string tolower [string trim [lindex $header 0]]]
> set value [string trim [lindex $header 1]]
>
> ns_set put $form $name.$key $value
> }
>
> if { [llength $disposition] == 3 } {
> # uploaded file -- save the original filename as the value
> set filename [string trim [lindex [split [lindex $disposition 2] =]
> 1] \"]
> ns_set put $form $name $filename
>
> # read lines of data until another boundary is found
> set start [tell $fp]
> set end $start
>
> while { ![eof $fp] } {
> if { [string match $boundary* [string trim [gets $fp]]] } {
> break
> }
> set end [tell $fp]
> }
> set length [expr $end - $start - 2]
>
> # create a temp file for the content, which will be deleted
> # when the connection close. ns_openexcl can fail, hence why
> # we keep spinning
>
> set tmp ""
> while { $tmp == "" } {
> set tmpfile [ns_tmpnam]
> set tmp [ns_openexcl $tmpfile]
> }
>
> catch {fconfigure $tmp -encoding binary -translation binary}
>
> if { $length > 0 } {
> seek $fp $start
> ns_cpfp $fp $tmp $length
> }
>
> close $tmp
> seek $fp $end
> ns_set put $form $name.tmpfile $tmpfile
>
> if { [ns_conn isconnected] } {
> ns_atclose "ns_unlink -nocomplain $tmpfile"
> }
>
> } else {
> # ordinary field - read lines until next boundary
> set first 1
> set value ""
> set start [tell $fp]
>
> while { [gets $fp line] >= 0 } {
> set line [string trimright $line \r]
> if { [string match $boundary* $line] } {
> break
> }
> if { $first } {
> set first 0
> } else {
> append value \n
> }
> append value $line
> set start [tell $fp]
> }
> seek $fp $start
> ns_set put $form $name $value
> }
> }
> close $fp
> }
>
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
<[email protected]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
