Hi Tomas,
Sorry for the delay in my reply. The requests are serial but occur in rapid
succession which was causing a lot of memory use, the current limit I have
set is 20 requests per minute. The manner in which you are currently
querying is not optimal, we do have a method for querying in bulk,
available here http://www.hmdb.ca/spectra/ms/search
Another possible solution would be to download the masses of the compounds
and corresponding HMDB IDs and then run the query in your program. I
suspect this would greatly speed up getting your program for your users as
well. If you are concerned with always having the latest compounds
available it would still be more efficient if you downloaded all the IDs
and compound masses at the start of every MzMine session. I could make a
download link available for you that contains this information in a tab
delimited format, the file would be around 287 kb when zipped so it would
be a very quick download.
Would any of these options work for you?
Mike
On Thu, Nov 29, 2012 at 6:47 PM, Tomas Pluskal <plus...@oist.jp> wrote:
> Hi Michael,
>
> Thanks for contacting us. Let's work together to resolve this issue.
>
> In MZmine, the user may choose a single detected peak (=mass value), or a
> peak list full of detected masses, and submit them to HMDB search.
> I suppose a single mass search will never exceed your access limits, so
> the problem might arise when the whole peak list is selected.
> A single peak list may contain potentially thousands of mass values,
> depending on the analytical method and parameters used for peak detection.
>
> MZmine searches for the masses in HMDB one-by-one in a serial fashion -
> each new request is sent after the previous request results are received.
> There are no multiple parallel connections to HMDB from a single MZmine
> instance.
>
> The search process works as follows:
> 1) For each mass that is searched, MZmine will first access this URL:
> http://www.hmdb.ca/chemquery/mass?&query_from=XXX&query_to=YYY to find
> compounds within certain tolerance range of that mass
> 2) For each of the compounds found, MZmine then downloads the details page
> at this URL: http://www.hmdb.ca/metabolites/HMDB12345 (only the actual
> HTML result is downloaded, no images or scripts etc). There is a limit for
> how many compounds MZmine will download from a single mass search. I think
> most users set this limit to 10 or 100.
> 3) After all the individual results are downloaded, MZmine will proceed to
> the next mass search (step 1)
>
> As you can see from the above description, a single search operation on a
> peak list containing thousands of entries might generate tens of thousands
> of page requests to HMDB. On the other hand, these requests are generated
> serially, so a single MZmine user would never overload your web server. The
> problem might occur if there are many MZmine users performing searches at
> the same time.
>
> What is the current access limit you imposed on the HMDB website?
> Do you have any suggestion how to optimize the search process?
>
> Best regards,
>
> Tomas
>
>
> On Nov 30, 2012, at 2:48 AM, Michael Wilson <michael.wil...@ualberta.ca>
> wrote:
>
> > Hello,
> >
> > I am one of the maintainers/developers of HMDB, we recently had to limit
> the number of hits per minute to HMDB as some users were sending huge
> numbers of requests for an extended period of time which was affecting
> performance of the site. Any hits that exceed the rate limit will get 503
> errors. I have recently received a complaint from a user of your MzMine
> software that the software is throwing an error due to a 503 error being
> returned by HMDB implying it is exceeding the access limit.
> >
> > I would like to modify the limit to enable your software to keep
> working, but need some information from your team to enable me to do so. At
> what rate does MzMine grab pages from HMDB? Also how many do you grab at a
> time? An easy solution would be for me to increase the allowed burst size
> or allowed rate to a high enough number to allow the MzMine software to
> continue to function properly.
> >
> > Kind regards,
> >
> > Mike
> >
> > PS here is the message from the user:
> >
> > I am currently using mzMine, and searching for metabolite identification
> using the HMDB. During its search, the following message appears:
> > 'Could not connect to HMDB: java.io.IOException:
> > Server returned HTTP response code: 503 for URL
> > http://www.hmdb.ca/metabolites/HMDB11601(null:-1)'
> > Is there anything that I could do to rectify this problem and return all
> metabolite possibilities?
> >
> > --
> > Michael Wilson
> > Bioinformatic Systems Analyst
> > michael.wil...@ualberta.ca
> > 780.492.2315 (office)
> > 780.289.4026 (mobile)
> >
> > University of Alberta
> > Department of Computing Science
> > 116 St. and 85 Ave.
> > Edmonton, Alberta T6G 2R3
> >
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Keep yourself connected to Go Parallel:
> > VERIFY Test and improve your parallel project with help from experts
> > and peers.
> http://goparallel.sourceforge.net_______________________________________________
> > Mzmine-devel mailing list
> > Mzmine-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/mzmine-devel
>
> ===============================================
> Tomáš Pluskal
> G0 Cell Unit, Okinawa Institute of Science and Technology Graduate
> University
> 1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
> WWW: https://groups.oist.jp/g0
> TEL: +81-98-966-8684
> Fax: +81-98-966-2890
>
>
--
Michael Wilson
Bioinformatic Systems Analyst
michael.wil...@ualberta.ca
780.492.2315 (office)
780.289.4026 (mobile)
University of Alberta
Department of Computing Science
116 St. and 85 Ave.
Edmonton, Alberta T6G 2R3
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
