Re: [sword-app-tech] deposit.isMultipart() and multipart.dat (sword2-server-1.0-classes.jar, MultipartStream$MalformedStreamException)
Hi Phil, Yeah, multipart is critically broken across all implementations. The Apache Commons/Java approach doesn't work at all (notwithstanding Eddie's workaround), and the python implementation is highly variable (depending on framework used), but none of them truly work. It is my plan and hope to back all of the multipart OUT of the sword spec for a future version (like a 2.1), so I strongly recommend not using multipart deposit. Instead do a POST of an Atom Entry and a PUT of the Media Resource in two distinct HTTP requests. Cheers, Richard On 10 July 2013 22:42, Philip Durbin philip_dur...@harvard.edu wrote: Very helpful, thanks, Eddie. I see one of your commit comments is this... Best as I can tell, commons-fileupload doesn't actually handle multipart/related correctly. This is currently a naive hack that reads everything into memory, but it at least parses multipart/related correctly (if you have the heap space). I wonder what you think about this comment from http://irclog.greptilian.com/sourcefu/2013-07-10#i_10152 ... semiosis: side note, servlet 3 supports multipart, no need for apache commons multipart anymore Phil On Wed, Jul 10, 2013 at 5:28 PM, Edwin Shin edwin.s...@yourmediashelf.com wrote: Philip, If this is related to, ahem, multipart-related support, that simply didn't work in swordapp (at least last year). It was one of the issues I addressed in my fork, see: https://github.com/mediashelf/sword2-server/commit/5458968f7d4e73c76bd35c18f6d4102d99663db3 https://github.com/mediashelf/sword2-server/commit/1f6c677c36b409cb929370df31d28e162cf56632 -Eddie On Jul 10, 2013, at 5:05 PM, Philip Durbin philip_dur...@harvard.edu wrote: I'm still using sword2-server-1.0-classes.jar and got a strange exception today when trying to make deposit.isMultipart() true with the proper arguments to curl. I'm able to reproduce the exception in my Vagrant environment so I just added this issue: MalformedStreamException with Content-Type: multipart/related · Issue #2 · dvn/swordpoc - https://github.com/dvn/swordpoc/issues/2 Anyway, I'm still investigating this but I thought I'd bring it up while it's on my mind. Before working on this too hard I'll probably return to cases where deposit.isBinaryOnly() or deposit.isEntryOnly() is true... seems like I'll want to get multipart working eventually though... Phil p.s. Here's what I wrote in that issue #2: While looking at isMultipart() in https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/Deposit.java I attempted to use curl to upload multipart.dat from https://github.com/swordapp/Simple-Sword-Server/blob/master/tests/resources/multipart.dat with these commands... wget https://raw.github.com/swordapp/Simple-Sword-Server/master/tests/resources/multipart.dat curl --insecure --data-binary @multipart.dat -H 'Content-Type: multipart/related; boundary0670350989==' -H MIME-Version: 1.0 https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 ... and got this exception: [#|2013-07-10T21:39:34.677+0200|WARNING|glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=19;_ThreadName=Thread-2;|StandardWrapperValve[collection]: PWC1406: Servlet.service() for servlet collection threw exception org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Header section has more than 10240 bytes (maybe it is not properly terminated) at org.apache.commons.fileupload.MultipartStream.readHeaders(MultipartStream.java:542) at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:976) at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.init(FileUploadBase.java:942) at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331) at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:349) at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126) at org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) at org.swordapp.server.SwordAPIEndpoint.addDepositPropertiesFromMultipart(SwordAPIEndpoint.java:235) at org.swordapp.server.CollectionAPI.post(CollectionAPI.java:152) at org.swordapp.server.servlets.CollectionServletDefault.doPost(CollectionServletDefault.java:48) at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)
Re: [sword-app-tech] deposit.isMultipart() and multipart.dat (sword2-server-1.0-classes.jar, MultipartStream$MalformedStreamException)
Very helpful, thanks, Eddie. I see one of your commit comments is this... Best as I can tell, commons-fileupload doesn't actually handle multipart/related correctly. This is currently a naive hack that reads everything into memory, but it at least parses multipart/related correctly (if you have the heap space). I wonder what you think about this comment from http://irclog.greptilian.com/sourcefu/2013-07-10#i_10152 ... semiosis: side note, servlet 3 supports multipart, no need for apache commons multipart anymore Phil On Wed, Jul 10, 2013 at 5:28 PM, Edwin Shin edwin.s...@yourmediashelf.com wrote: Philip, If this is related to, ahem, multipart-related support, that simply didn't work in swordapp (at least last year). It was one of the issues I addressed in my fork, see: https://github.com/mediashelf/sword2-server/commit/5458968f7d4e73c76bd35c18f6d4102d99663db3 https://github.com/mediashelf/sword2-server/commit/1f6c677c36b409cb929370df31d28e162cf56632 -Eddie On Jul 10, 2013, at 5:05 PM, Philip Durbin philip_dur...@harvard.edu wrote: I'm still using sword2-server-1.0-classes.jar and got a strange exception today when trying to make deposit.isMultipart() true with the proper arguments to curl. I'm able to reproduce the exception in my Vagrant environment so I just added this issue: MalformedStreamException with Content-Type: multipart/related · Issue #2 · dvn/swordpoc - https://github.com/dvn/swordpoc/issues/2 Anyway, I'm still investigating this but I thought I'd bring it up while it's on my mind. Before working on this too hard I'll probably return to cases where deposit.isBinaryOnly() or deposit.isEntryOnly() is true... seems like I'll want to get multipart working eventually though... Phil p.s. Here's what I wrote in that issue #2: While looking at isMultipart() in https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/Deposit.java I attempted to use curl to upload multipart.dat from https://github.com/swordapp/Simple-Sword-Server/blob/master/tests/resources/multipart.dat with these commands... wget https://raw.github.com/swordapp/Simple-Sword-Server/master/tests/resources/multipart.dat curl --insecure --data-binary @multipart.dat -H 'Content-Type: multipart/related; boundary0670350989==' -H MIME-Version: 1.0 https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 ... and got this exception: [#|2013-07-10T21:39:34.677+0200|WARNING|glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=19;_ThreadName=Thread-2;|StandardWrapperValve[collection]: PWC1406: Servlet.service() for servlet collection threw exception org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Header section has more than 10240 bytes (maybe it is not properly terminated) at org.apache.commons.fileupload.MultipartStream.readHeaders(MultipartStream.java:542) at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:976) at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.init(FileUploadBase.java:942) at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331) at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:349) at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126) at org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) at org.swordapp.server.SwordAPIEndpoint.addDepositPropertiesFromMultipart(SwordAPIEndpoint.java:235) at org.swordapp.server.CollectionAPI.post(CollectionAPI.java:152) at org.swordapp.server.servlets.CollectionServletDefault.doPost(CollectionServletDefault.java:48) at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161) at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231) at
Re: [sword-app-tech] deposit.isMultipart() and multipart.dat (sword2-server-1.0-classes.jar, MultipartStream$MalformedStreamException)
Philip, If this is related to, ahem, multipart-related support, that simply didn't work in swordapp (at least last year). It was one of the issues I addressed in my fork, see: https://github.com/mediashelf/sword2-server/commit/5458968f7d4e73c76bd35c18f6d4102d99663db3 https://github.com/mediashelf/sword2-server/commit/1f6c677c36b409cb929370df31d28e162cf56632 -Eddie On Jul 10, 2013, at 5:05 PM, Philip Durbin philip_dur...@harvard.edu wrote: I'm still using sword2-server-1.0-classes.jar and got a strange exception today when trying to make deposit.isMultipart() true with the proper arguments to curl. I'm able to reproduce the exception in my Vagrant environment so I just added this issue: MalformedStreamException with Content-Type: multipart/related · Issue #2 · dvn/swordpoc - https://github.com/dvn/swordpoc/issues/2 Anyway, I'm still investigating this but I thought I'd bring it up while it's on my mind. Before working on this too hard I'll probably return to cases where deposit.isBinaryOnly() or deposit.isEntryOnly() is true... seems like I'll want to get multipart working eventually though... Phil p.s. Here's what I wrote in that issue #2: While looking at isMultipart() in https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/Deposit.java I attempted to use curl to upload multipart.dat from https://github.com/swordapp/Simple-Sword-Server/blob/master/tests/resources/multipart.dat with these commands... wget https://raw.github.com/swordapp/Simple-Sword-Server/master/tests/resources/multipart.dat curl --insecure --data-binary @multipart.dat -H 'Content-Type: multipart/related; boundary0670350989==' -H MIME-Version: 1.0 https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 ... and got this exception: [#|2013-07-10T21:39:34.677+0200|WARNING|glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=19;_ThreadName=Thread-2;|StandardWrapperValve[collection]: PWC1406: Servlet.service() for servlet collection threw exception org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Header section has more than 10240 bytes (maybe it is not properly terminated) at org.apache.commons.fileupload.MultipartStream.readHeaders(MultipartStream.java:542) at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:976) at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.init(FileUploadBase.java:942) at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331) at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:349) at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126) at org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) at org.swordapp.server.SwordAPIEndpoint.addDepositPropertiesFromMultipart(SwordAPIEndpoint.java:235) at org.swordapp.server.CollectionAPI.post(CollectionAPI.java:152) at org.swordapp.server.servlets.CollectionServletDefault.doPost(CollectionServletDefault.java:48) at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161) at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231) at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317) at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195) at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860) at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757) at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056) at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229) at
