On Jul 31, 2:52 am, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
> 2009/7/31 gert <gert.cuyk...@gmail.com>:
>
> > On Jul 31, 2:19 am, Graham Dumpleton <graham.dumple...@gmail.com>
> > wrote:
> >> 2009/7/31 gert <gert.cuyk...@gmail.com>:
>
> >> > On Jul 31, 1:04 am, Graham Dumpleton <graham.dumple...@gmail.com>
> >> > wrote:
> >> >> 2009/7/31 gert <gert.cuyk...@gmail.com>:
>
> >> >> > On Jul 28, 5:50 am, Graham Dumpleton <graham.dumple...@gmail.com>
> >> >> > wrote:
> >> >> >> 2009/7/28 gert <gert.cuyk...@gmail.com>:
>
> >> >> >> > On Jul 26, 1:59 am, gert <gert.cuyk...@gmail.com> wrote:
> >> >> >> >> On Jul 26, 1:25 am, Graham Dumpleton <graham.dumple...@gmail.com>
> >> >> >> >> wrote:
>
> >> >> >> >> > 2009/7/25 gert <gert.cuyk...@gmail.com>:
>
> >> >> >> >> > > On Jul 25, 3:42 am, Graham Dumpleton
> >> >> >> >> > > <graham.dumple...@gmail.com>
> >> >> >> >> > > wrote:
> >> >> >> >> > >> 2009/7/25 gert <gert.cuyk...@gmail.com>:
>
> >> >> >> >> > >> > On Jul 24, 1:22 am, Graham Dumpleton
> >> >> >> >> > >> > <graham.dumple...@gmail.com>
> >> >> >> >> > >> > wrote:
> >> >> >> >> > >> >> 2009/7/24 gert <gert.cuyk...@gmail.com>:
> >> >> >> >> > >> >> > On Jul 23, 9:27 pm, gert <gert.cuyk...@gmail.com> wrote:
> >> >> >> >> > >> >> >> On Jul 23, 7:28 pm, gert <gert.cuyk...@gmail.com>
> >> >> >> >> > >> >> >> wrote:
>
> >> >> >> >> > >> >> >> > wsgi file downloads
> >> >> >> >> > >> >> >> > ------------------------------
> >> >> >> >> > >> >> >> > class FileWrapper(object):
> >> >> >> >> > >> >> >> > def __init__(self, fp, blksize=8192):
> >> >> >> >> > >> >> >> > self.fp = fp
> >> >> >> >> > >> >> >> > self.blksize=blksize
> >> >> >> >> > >> >> >> > def __getitem__(self, key):
> >> >> >> >> > >> >> >> > data = self.fp.read(self.blksize)
> >> >> >> >> > >> >> >> > if data: return data
> >> >> >> >> > >> >> >> > raise IndexError
> >> >> >> >> > >> >> >> > def close(self):
> >> >> >> >> > >> >> >> > self.fp.close()
>
> >> >> >> >> > >> >> >> > def application(environ, response):
> >> >> >> >> > >> >> >> > import os
> >> >> >> >> > >> >> >> >
> >> >> >> >> > >> >> >> > f=open(os.path.join(os.path.dirname(__file__),'../www/bin/
> >> >> >> >> > >> >> >> > picture.png'), 'rb')
> >> >> >> >> > >> >> >> > l = os.fstat(f.fileno()).st_size
> >> >> >> >> > >> >> >> > response('200 OK', [('Content-type',
> >> >> >> >> > >> >> >> > 'image/png'), ('Content-
> >> >> >> >> > >> >> >> > Length', str(l))])
> >> >> >> >> > >> >> >> > if 'wsgi.file_wrapper' in environ: return
> >> >> >> >> > >> >> >> > environ
> >> >> >> >> > >> >> >> > ['wsgi.file_wrapper'](f, 8192)
> >> >> >> >> > >> >> >> > else: return FileWrapper(f, 8192)
>
> >> >> >> >> > >> >> >> > wsgi file uploads
> >> >> >> >> > >> >> >> > -------------------------
> >> >> >> >> > >> >> >> > from re import search,DOTALL
>
> >> >> >> >> > >> >> >> > def application(environ, response):
> >> >> >> >> > >> >> >> > t = environ['wsgi.input'].read(int(environ
> >> >> >> >> > >> >> >> > ['CONTENT_LENGTH'])).decode('latin1')
> >> >> >> >> > >> >> >> > b = environ['CONTENT_TYPE'].split('boundary=')[1]
> >> >> >> >> > >> >> >> > p = search(b+r'.*?Content-Type:
> >> >> >> >> > >> >> >> > application/octet-stream\r\n\r\n
> >> >> >> >> > >> >> >> > (.*?)\r\n--'+b,t,DOTALL).group(1)
> >> >> >> >> > >> >> >> > p = p.encode('latin1')
> >> >> >> >> > >> >> >> > response('200 OK', [('Content-type',
> >> >> >> >> > >> >> >> > 'text/plain;charset=utf-8'),
> >> >> >> >> > >> >> >> > ('Content-Length', str(len(j))) ])
> >> >> >> >> > >> >> >> > return ['complete']
>
> >> >> >> >> > >> >> >> > How would you chunk up the reading part to save
> >> >> >> >> > >> >> >> > memory ?
>
> >> >> >> >> > >> >> >> def application(environ, response):
> >> >> >> >> > >> >> >> with open('/usr/httpd/logs/wsgiTemp') as f:
> >> >> >> >> > >> >> >> while True:
> >> >> >> >> > >> >> >> chunk = environ['wsgi.input'].read(8192)
> >> >> >> >> > >> >> >> if not chunk: break
> >> >> >> >> > >> >> >> f.write(chunk)
> >> >> >> >> > >> >> >> response('200 OK')
> >> >> >> >> > >> >> >> return ['complete']
>
> >> >> >> >> > >> >> >> hmm it keeps on telling me its not writeble aldo it is
> >> >> >> >> > >> >> >> chmod 777
>
> >> >> >> >> > >> >> > woops :)
>
> >> >> >> >> > >> >> > def application(environ, response):
> >> >> >> >> > >> >> > with open('/usr/httpd/logs/wsgiTemp','w') as f:
> >> >> >> >> > >> >> > while True:
> >> >> >> >> > >> >> > chunk =
> >> >> >> >> > >> >> > environ['wsgi.input'].read(8192).decode('latin1')
> >> >> >> >> > >> >> > if not chunk: break
> >> >> >> >> > >> >> > f.write(chunk)
> >> >> >> >> > >> >> > response('200 OK',[])
> >> >> >> >> > >> >> > return ['complete']
>
> >> >> >> >> > >> >> That is not portable WSGI code. It may work on mod_wsgi
> >> >> >> >> > >> >> but not on
> >> >> >> >> > >> >> other WSGI servers. This is because WSGI specification
> >> >> >> >> > >> >> requires that
> >> >> >> >> > >> >> you not read more than CONTENT_LENGTH bytes from input
> >> >> >> >> > >> >> stream. It only
> >> >> >> >> > >> >> works for mod_wsgi because it returns a proper EOF
> >> >> >> >> > >> >> sentinel when input
> >> >> >> >> > >> >> is exhausted, but WSGI specification doesn't specifically
> >> >> >> >> > >> >> require
> >> >> >> >> > >> >> that. Where the code can thus fail is where client
> >> >> >> >> > >> >> requests keep alive
> >> >> >> >> > >> >> for connection.
>
> >> >> >> >> > >> > How do I make sure a chunk doesn't cut the boundary in
> >> >> >> >> > >> > half ? So i can
> >> >> >> >> > >> > figure out what chunks are binary data ?
>
> >> >> >> >> > >> You have to internally buffer data within the component.
> >> >> >> >> > >> This is what
> >> >> >> >> > >> cgi.FieldStorage does, so perhaps have a look at it.
>
> >> >> >> >> > >> Better still have a look at equivalent to cgi.FieldStorage
> >> >> >> >> > >> in Werkzeug
> >> >> >> >> > >> as it is WSGI 1.0 compliant where as cgi.FieldStorage isn't.
>
> >> >> >> >> > > Can mod_wsgi handle PUT request ? Or do i need a separate
> >> >> >> >> > > apache
> >> >> >> >> > > module for this ?
> >> >> >> >> > > Seems PUT is the only possible way to resume file uploads ?
>
> >> >> >> >> > All HTTP methods except maybe CONNECT and TRACE should get
> >> >> >> >> > through to mod_wsgi.
>
> >> >> >> >> > Whether resumable uploads is possible is probably more a
> >> >> >> >> > feature of
> >> >> >> >> > your WSGI application and cannot see why it would be restricted
> >> >> >> >> > to PUT
> >> >> >> >> > except by convention. Of course your client needs to support
> >> >> >> >> > uploads
> >> >> >> >> > for byte ranges.
>
> >> >> >> >> I tested it with curl -T bigfile
> >> >> >> >> http://gert:pas...@192.168.2.17/upload.wsgi
> >> >> >> >> and indeed it just dumps the file on my disk, which is good news
> >> >> >> >> :)
>
> >> >> >> >> The I could have guest it news is that curl -C bigfile
> >> >> >> >> http://gert:pas...@192.168.2.17/upload.wsgi results into a empty
> >> >> >> >> file.
> >> >> >> >> So curl is expecting something back before it sends anything,
> >> >> >> >> which
> >> >> >> >> makes it a bit complicated, actually allot. Has anybody already
> >> >> >> >> made
> >> >> >> >> this kind of wsgi for curl handshaking ?
>
> >> >> >> > It seems that you have to tell curl where it needs to start
> >> >> >> > uploading
> >> >> >> > manually, there is no such thing as server communication to do so
> >> >> >> > automatically. Example
> >> >> >> > curl -C 1024 bigfile http://gert:pas...@192.168.2.17/upload.wsgi
>
> >> >> >> > That makes me wonder if the download wsgi.file_wrapper can actually
> >> >> >> > resume downloads, I always took it for granted it could do so ?
> >> >> >> > And if
> >> >> >> > not where does the offset get stored in the headers ?
>
> >> >> >> Would expect that that would also require WSGI application to be
> >> >> >> knowledgeable about byte range requests. So, WSGI application would
> >> >> >> need to see it is a request for partial data, open the file, seek to
> >> >> >> start position, plus set status and response headers indicating
> >> >> >> partial response, and set Content-Length to amount of data to be
> >> >> >> returned from the seek position in file.
>
> >> >> >> Note that this code wouldn't be portable as mod_wsgi is I believe
> >> >> >> only
> >> >> >> implementation which pays attention to Content-Length when
> >> >> >> wsgi.file_wrapper is used to ensure only part of a file is returned.
>
> >> >> >> Can't help much more than that as don't know about how HTTP handles
> >> >> >> partial byte range requests etc.
>
> >> >> >http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.16
>
> >> >> > So how do i tell the filewrapper to do this ?
>
> >> >> > Content-Range: bytes 21010-47021/47022
> >> >> > Content-Length: 26012
>
> >> >> I am not sure what all the values in the Content-Range mean so you
> >> >> will have to explain. And no I don't have time to go read the RFC.
>
> >> >> If it is returning just a single range within the file, then do the
> >> >> following.
>
> >> >> Open file. Seek to position 21010. Set returned Content-Length header
> >> >> to length from that seek point onwards that want to return, possible
> >> >> 26012, and then return wsgi.file_wrapper() for open file object.
>
> >> > I hate RFC's they always make it sound so complicated
>
> >> > Content-Range: bytes 21010-47021/47022
> >> > Content-Length: 26012
>
> >> > html 1.1 only support bytes so all numbers represent bytes
> >> > (first number is te start byte) - (second the end byte) / (third
> >> > number is the total byte count of the complete file or * if unkown,
> >> > example 0 1 2 = 3 bytes)
> >> > content length is the number of bytes that wil get sent.
>
> >> > take note that this could be used to download different parts from
> >> > several servers
>
> >> > So how do i tell the environ['wsgi.file_wrapper'](f, 8192) that f is
> >> > just the 500 bytes in the middle of a file ?
>
> >> I have already told you that twice. :-(
>
> >> What don't you understand about:
>
> >> """
> >> Open file. Seek to position 21010. Set returned Content-Length header
> >> to length from that seek point onwards that want to return, possible
> >> 26012, and then return wsgi.file_wrapper() for open file object.
> >> """
>
> >> I suggest you go read about the seek() method file objects in Python
> >> documentation.
>
> > I know but your first question was
> > "I am not sure what all the values in the Content-Range mean so you
> > will have to explain."
> > So i did what you asked me to do, explaining :)
>
> I was only referring to your question:
>
> """
> So how do i tell the environ['wsgi.file_wrapper'](f, 8192) that f is
> just the 500 bytes in the middle of a file ?
> """
That would be a explanation question of the previous question :)
> and I had already answered that twice.
You are going to love my next "GET someURL\r\n Range:
bytes=4096-8191,0-4095" question :)
and how the hell do i generate a boundary --46228a661764c4210
> Code example below:
>
> import os
>
> def application(environ, start_response):
> status = '200 OK'
> handle = file('/etc/passwd', 'rb')
>
> # Seek to start position 10 bytes into file.
> handle.seek(10, os.SEEK_SET)
>
> # Set content length so only 6 bytes from set position of file is sent.
> response_headers = [('Content-type', 'text/plain'),('Content-Length',
> '6')]
>
> start_response(status, response_headers)
>
> return environ['wsgi.file_wrapper'](handle)
>
> Remember that this likely will not work on other WSGI implementations
> as I believe that only mod_wsgi pays attentions to response
> Content-Length and ensures that applications only return as much data
> as they indicate should be.
>
ok thanks going to experiment with it a bit need to look up what
os.SEEK_SET does.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To post to this group, send email to modwsgi@googlegroups.com
To unsubscribe from this group, send email to
modwsgi+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en
-~----------~----~----~----~------~----~------~--~---
