Ok, so even though my development environment and GAE don't want to allow big files I can do it by deploying the project in own server just by setting the limits so that they fit my needs. Thank you for the help!
On Wed, Jul 14, 2010 at 1:16 PM, Manuel Carrasco Moñino <[email protected]>wrote: > There are many constrains in GAE, when I was playing with it, I had > problems with request size, request time, etc. For instance on those > times the request could not take more than 30 seconds, and the request > size was 512KB. > > I know things have changed now, and the limits are different when you > use a free or a billing account. Also there are some differences > running the app in development and in production modes. > > I think if you are unable to upload files larger than 10MB there > should be a limit. > > -Manolo > > > > On Wed, Jul 14, 2010 at 9:11 AM, Mika Tikkanen > <[email protected]> wrote: > > I'm not using blobstore, probably, I don't even know what it is :) From > > earlier code you can see that I'm just reading the input to output at the > > server side. So there is no storing of the files, just reading back to > > client. > > > > On Wed, Jul 14, 2010 at 9:23 AM, Manuel Carrasco Moñino < > [email protected]> > > wrote: > >> > >> If you are using appengine things are different because there are many > >> constrains. Are you using blobstore? > >> > >> On Tue, Jul 13, 2010 at 3:00 PM, Mika Tikkanen > >> <[email protected]> wrote: > >> > So how do I change that value for google appengine? Or for my > >> > development > >> > environment in eclipse? > >> > > >> > On Tue, Jul 13, 2010 at 12:03 PM, Manuel Carrasco Moñino > >> > <[email protected]> > >> > wrote: > >> >> > >> >> It seems an issue with the maximum configured limit for post requests > >> >> at your server side. > >> >> > >> >> -Manolo > >> >> > >> >> On Tue, Jul 13, 2010 at 10:43 AM, Mika Tikkanen > >> >> <[email protected]> wrote: > >> >> > On Mon, Jul 12, 2010 at 7:26 PM, Jeff Chimene <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> On 07/11/2010 11:17 AM, Roope wrote: > >> >> >> > Hi > >> >> >> > > >> >> >> > I'm currently developing some bioinformatics tools and I want to > >> >> >> > make > >> >> >> > them as web apps. > >> >> >> > > >> >> >> > The thing is that user needs to input files that might be up to > >> >> >> > 4Gb > >> >> >> > but usually the biggest is just about 250Mb. > >> >> >> > > >> >> >> > There are two main use-cases: > >> >> >> > 1. Steaming the file and taking just some small parts of it to > >> >> >> > string > >> >> >> > 2. Reading the whole file and making object from each line in > file > >> >> >> > > >> >> >> > I would like to do all the file processing in client side, but I > >> >> >> > understood that it is not yet possible using gwt? > >> >> >> > >> >> >> In general the answer is yes, but not because of GWT. Until > >> >> >> wide-spread > >> >> >> browser support of HTML 5 which will provide better local file > >> >> >> handling. > >> >> > > >> >> > > >> >> > So GWT wont support it yet and there might be some browsers that do > >> >> > support > >> >> > it. So I could use those browser that do support it and use > >> >> > some JavaScript code? > >> >> > > >> >> >> > >> >> >> > If I keep the server local it is feasible to upload some 250mb > >> >> >> > files > >> >> >> > and process them at the server side, but I have no success so > far > >> >> >> > in > >> >> >> > this, even with 10mb file. > >> >> >> > >> >> >> You might provide some background on the "... no success so > far..." > >> >> >> path. > >> >> > > >> >> > I managed to upload some over 1MB files but the over 10MB didn't > >> >> > upload. > >> >> > Here is the server side code, copied most of it and the rest of the > >> >> > code > >> >> > I > >> >> > show from some example how to do this or from forums. > >> >> > import org.apache.commons.fileupload.FileItemIterator; > >> >> > import org.apache.commons.fileupload.servlet.ServletFileUpload; > >> >> > import java.io.BufferedInputStream; > >> >> > import java.io.IOException; > >> >> > import java.io.InputStream; > >> >> > import java.io.OutputStream; > >> >> > import javax.servlet.ServletException; > >> >> > import javax.servlet.http.HttpServlet; > >> >> > import javax.servlet.http.HttpServletRequest; > >> >> > import javax.servlet.http.HttpServletResponse; > >> >> > public class FileUpload extends HttpServlet { > >> >> > public void doPost(HttpServletRequest req, HttpServletResponse res) > >> >> > throws ServletException, IOException { > >> >> > try { > >> >> > ServletFileUpload upload = new ServletFileUpload(); > >> >> > res.setContentType("text/plain"); > >> >> > FileItemIterator iterator = upload.getItemIterator(req); > >> >> > while (iterator.hasNext()) { > >> >> > copy(iterator.next().openStream(), res.getOutputStream()); > >> >> > } > >> >> > } catch (Exception ex) { > >> >> > throw new ServletException(ex); > >> >> > } > >> >> > } > >> >> > public static void copy(InputStream is, OutputStream os) throws > >> >> > IOException > >> >> > { > >> >> > byte buffer[] = new byte[8192]; > >> >> > int bytesRead; > >> >> > BufferedInputStream bis = new BufferedInputStream(is); > >> >> > while ((bytesRead = bis.read(buffer)) != -1) { > >> >> > os.write(buffer, 0, bytesRead); > >> >> > } > >> >> > is.close(); > >> >> > os.flush(); > >> >> > os.close(); > >> >> > } > >> >> > } > >> >> > Here is the the part from web.xml needed(replace * with your own > >> >> > stuff) > >> >> > <servlet> > >> >> > <servlet-name>fileUploaderServler</servlet-name> > >> >> > <servlet-class>com.*.*.server.FileUpload</servlet-class> > >> >> > </servlet> > >> >> > <servlet-mapping> > >> >> > <servlet-name>fileUploaderServler</servlet-name> > >> >> > <url-pattern>/*/fileupload</url-pattern> > >> >> > </servlet-mapping> > >> >> > ...and here is the code for the form that uses the fileupload > >> >> > private static FormPanel getForm(final DialogBox dialogBox,final > >> >> > TextArea > >> >> > ta) { > >> >> > final FormPanel form = new FormPanel(); > >> >> > form.setAction(GWT.getModuleBaseURL() + "fileupload"); > >> >> > // Because we're going to add a FileUpload widget, we'll need to > set > >> >> > the > >> >> > // form to use the POST method, and multipart MIME encoding. > >> >> > form.setEncoding(FormPanel.ENCODING_MULTIPART); > >> >> > form.setMethod(FormPanel.METHOD_POST); > >> >> > // Create a panel to hold all of the form widgets. > >> >> > final VerticalPanel panel = new VerticalPanel(); > >> >> > form.setWidget(panel); > >> >> > // Create a FileUpload widget. > >> >> > final FileUpload upload = new FileUpload(); > >> >> > upload.setName("uploadFormElement"); > >> >> > panel.add(upload); > >> >> > HorizontalPanel horizontal = new HorizontalPanel(); > >> >> > // Add a 'submit' button. > >> >> > horizontal.add(new Button("Submit", new ClickHandler() { > >> >> > public void onClick(ClickEvent event) { > >> >> > form.submit(); > >> >> > } > >> >> > })); > >> >> > // Add a 'cancel' button. > >> >> > horizontal.add(new Button("Cancel", new ClickHandler() { > >> >> > public void onClick(ClickEvent event) { > >> >> > dialogBox.hide(); > >> >> > } > >> >> > })); > >> >> > panel.add(horizontal); > >> >> > // Lets add handlers > >> >> > form.addSubmitHandler(new SubmitHandler() { > >> >> > @Override > >> >> > public void onSubmit(SubmitEvent event) { > >> >> > if (upload.getFilename().length() == 0) { > >> >> > Window.alert("Must select a valid file"); > >> >> > event.cancel(); > >> >> > } > >> >> > } > >> >> > }); > >> >> > form.addSubmitCompleteHandler(new SubmitCompleteHandler() { > >> >> > public void onSubmitComplete(SubmitCompleteEvent event) { > >> >> > ta.setText(event.getResults()); > >> >> > dialogBox.hide(); > >> >> > } > >> >> > }); > >> >> > return form; > >> >> > } > >> >> > > >> >> >> > >> >> >> For files of this size, I'd consider creating a file share on the > >> >> >> server > >> >> >> or using FTP. I'm guessing the data collection occurs on the PC? > If > >> >> >> so, > >> >> >> you won't be able to process the files locally until HTML 5 and > >> >> >> local > >> >> >> file support. Depending on your environment, you might be able to > >> >> >> remote-mount a disk device and write directly to that device from > >> >> >> the > >> >> >> data collection source. > >> >> >> > >> >> >> > So what would you suggest me to do? Besides dumping the web > >> >> >> > application idea and making some java applet. > >> >> >> > >> >> >> You want to move the data to the server, construct a server app > that > >> >> >> retrieves data slices on demand and sends them to the client for > >> >> >> rendering. You can certainly do the Java app on the server to > >> >> >> retrieve/preprocess the data, then render it on the client using > >> >> >> various > >> >> >> graphing packages. Search this list for pointers to SVG, or > graphing > >> >> >> libraries. > >> >> > > >> >> > I will probably make java applet that can do the parsing of the > file > >> >> > for > >> >> > the > >> >> > user and the the result, the smaller files, will be uploaded to > >> >> > server > >> >> > and > >> >> > then the string or required objects from that file are returned. > This > >> >> > way it > >> >> > is done at the "cleint side" but without including the web > >> >> > application > >> >> > in > >> >> > the process. After GWT and the browsers update to support reading > >> >> > file > >> >> > at > >> >> > client side I will dump the java tool. > >> >> > Is there way to check how big the file is that the user is trying > to > >> >> > send? > >> >> > Besides at the server side.. > >> >> > Thank you for the help! > >> >> > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > >> >> > Groups > >> >> > "Google Web Toolkit" group. > >> >> > To post to this group, send email to > >> >> > [email protected]. > >> >> > To unsubscribe from this group, send email to > >> >> > [email protected]<google-web-toolkit%[email protected]> > . > >> >> > For more options, visit this group at > >> >> > http://groups.google.com/group/google-web-toolkit?hl=en. > >> >> > > >> >> > >> >> -- > >> >> You received this message because you are subscribed to the Google > >> >> Groups > >> >> "Google Web Toolkit" group. > >> >> To post to this group, send email to > >> >> [email protected]. > >> >> To unsubscribe from this group, send email to > >> >> [email protected]<google-web-toolkit%[email protected]> > . > >> >> For more options, visit this group at > >> >> http://groups.google.com/group/google-web-toolkit?hl=en. > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google Web Toolkit" group. > >> > To post to this group, send email to > >> > [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]<google-web-toolkit%[email protected]> > . > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-web-toolkit?hl=en. > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google Web Toolkit" group. > >> To post to this group, send email to > [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<google-web-toolkit%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-web-toolkit?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google Web Toolkit" group. > > To post to this group, send email to [email protected] > . > > To unsubscribe from this group, send email to > > [email protected]<google-web-toolkit%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/google-web-toolkit?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google Web Toolkit" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-web-toolkit%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-web-toolkit?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
