Hi Anand, Processing is offline, but the order of magnitude will be more like days than 10 minutes, so I guess backends are the way to go. Basically the application will perform statistical analysis on large datasets of news items and media data. Right now, we run this analysis on hardware we own, but we'd like to make the basic toolset available to the scientific community.
As for SciPy being in Fortran, does this mean that there are no plans at all to make SciPy available on Appengine? Or only that it will take some more time to do so? Regards, Hans Then Op maandag 23 juli 2012 14:47:19 UTC+2 schreef Anand Mistry het volgende: > > If the processing is considered "offline" and not latency sensitive, then > you can put that work onto a task queue and take up to 10 minutes. No need > for a backend. > > As for SciPy, Brandon is exactly right. SciPy is written entirely in > Fortran whereas the other third-party libraries we have (NumPy, PIL, lxml) > are written in C. For NLTK, do you really need SciPy? According to > nltk.org, only NumPy is a dependency (optional). > > On Monday, 23 July 2012 09:33:35 UTC+10, Hans Then wrote: >> >> Ah. That is very good news. We have no real timing requirements. I think >> we will setup a backend instance for long running requests, to get around >> the 60s limit. >> >> On Mon, Jul 23, 2012 at 1:24 AM, Drake <[email protected]> wrote: >> >>> I think we have everything you need. We are working on some speed >>> improvements because unfortunately the 60s limit doesn’t let you work with >>> very big documents. >>> >>> Do you know how many words at a time you need to put through?**** >>> >>> ** ** >>> >>> ** ** >>> >>> *From:* [email protected] [mailto: >>> [email protected]] *On Behalf Of *Hans Then >>> *Sent:* Sunday, July 22, 2012 3:51 PM >>> >>> *To:* [email protected] >>> *Subject:* Re: [google-appengine] Re: numpy and scipy question**** >>> >>> ** ** >>> >>> I am mainly using the classify and the cluster packages.**** >>> >>> ** ** >>> >>> Thanks for considering this to be included.**** >>> >>> ** ** >>> >>> Hans >>> >>> Op zondag 22 juli 2012 19:18:25 UTC+2 schreef Brandon Wirtz het volgende: >>> **** >>> >>> I am building an API that does much of what NLTK does. Which parts are >>> you looking to use? Likely making API calls to my optimized version would >>> cost less than actually running the full version, (which includes a lot of >>> things that won’t work on GAE (like the cool drawing of the sentence >>> diagram) >>> >>> If the parts you need don’t take too much effort to add, I’ll put them >>> on my teams task list for the API release in 2 weeks.**** >>> >>> **** >>> >>> -Brandon**** >>> >>> **** >>> >>> *From:* [email protected] [mailto: >>> [email protected]] *On Behalf Of *Hans Then >>> *Sent:* Sunday, July 22, 2012 7:04 AM >>> *To:* [email protected] >>> *Subject:* Re: [google-appengine] Re: numpy and scipy question**** >>> >>> **** >>> >>> The application I develop uses libraries such as gensim and NLTK, which >>> depend on SciPy. Currently my plan is to run it on Heroku, but I would much >>> prefer to use appengine.**** >>> >>> **** >>> >>> Hans**** >>> >>> On Sun, Jul 22, 2012 at 11:24 AM, Drake <[email protected]> wrote:**** >>> >>> I don’t know then. I thought NumPy was all C… >>> >>> Any how I use Numpy but not SciPy. Apparently Google caters to my every >>> whim and so they gave me what I needed. J >>> >>> Depending what you need Matrix.py, and Euclid.py get you many of the >>> most common things you would use Numpy and Scipy for.**** >>> >>> **** >>> >>> **** >>> >>> *From:* [email protected] [mailto: >>> [email protected]] *On Behalf Of *Hans Then >>> *Sent:* Sunday, July 22, 2012 1:34 AM >>> *To:* [email protected] >>> *Subject:* Re: [google-appengine] Re: numpy and scipy question**** >>> >>> **** >>> >>> NumPy is also Fortran. **** >>> >>> **** >>> >>> But my confusion is that if SciPy will not be supported, then why >>> support NumPy? NumPy and SciPy are hardly ever used separately.**** >>> >>> **** >>> >>> Hans >>> >>> Op zondag 22 juli 2012 09:11:00 UTC+2 schreef Brandon Wirtz het volgende: >>> **** >>> >>> SciPy is mostly in Fortran. So I don’t give you good odds.**** >>> >>> **** >>> >>> *From:* [email protected] [mailto: >>> [email protected]] *On Behalf Of *Hans Then >>> *Sent:* Sunday, July 22, 2012 12:05 AM >>> *To:* [email protected] >>> *Subject:* [google-appengine] Re: numpy and scipy question**** >>> >>> **** >>> >>> Hi Anand,**** >>> >>> **** >>> >>> Are there plans to also support SciPy in the future?**** >>> >>> **** >>> >>> Hans Then >>> >>> Op woensdag 25 januari 2012 00:34:53 UTC+1 schreef Anand Mistry het >>> volgende:**** >>> >>> NumPy is available on the Python 2.7 runtime, but SciPy is not.**** >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msg/google-appengine/-/-WRRRE6BhD8J. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en.**** >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msg/google-appengine/-/iM54xnkAilIJ. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en.**** >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en.**** >>> >>> **** >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en.**** >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msg/google-appengine/-/ESn4rOHx3a8J. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en.**** >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/k9iy_VHClxwJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
