Hi Michael, Thanks for contacting us. Let's work together to resolve this issue.
In MZmine, the user may choose a single detected peak (=mass value), or a peak list full of detected masses, and submit them to HMDB search. I suppose a single mass search will never exceed your access limits, so the problem might arise when the whole peak list is selected. A single peak list may contain potentially thousands of mass values, depending on the analytical method and parameters used for peak detection. MZmine searches for the masses in HMDB one-by-one in a serial fashion - each new request is sent after the previous request results are received. There are no multiple parallel connections to HMDB from a single MZmine instance. The search process works as follows: 1) For each mass that is searched, MZmine will first access this URL: http://www.hmdb.ca/chemquery/mass?&query_from=XXX&query_to=YYY to find compounds within certain tolerance range of that mass 2) For each of the compounds found, MZmine then downloads the details page at this URL: http://www.hmdb.ca/metabolites/HMDB12345 (only the actual HTML result is downloaded, no images or scripts etc). There is a limit for how many compounds MZmine will download from a single mass search. I think most users set this limit to 10 or 100. 3) After all the individual results are downloaded, MZmine will proceed to the next mass search (step 1) As you can see from the above description, a single search operation on a peak list containing thousands of entries might generate tens of thousands of page requests to HMDB. On the other hand, these requests are generated serially, so a single MZmine user would never overload your web server. The problem might occur if there are many MZmine users performing searches at the same time. What is the current access limit you imposed on the HMDB website? Do you have any suggestion how to optimize the search process? Best regards, Tomas On Nov 30, 2012, at 2:48 AM, Michael Wilson <michael.wil...@ualberta.ca> wrote: > Hello, > > I am one of the maintainers/developers of HMDB, we recently had to limit the > number of hits per minute to HMDB as some users were sending huge numbers of > requests for an extended period of time which was affecting performance of > the site. Any hits that exceed the rate limit will get 503 errors. I have > recently received a complaint from a user of your MzMine software that the > software is throwing an error due to a 503 error being returned by HMDB > implying it is exceeding the access limit. > > I would like to modify the limit to enable your software to keep working, but > need some information from your team to enable me to do so. At what rate does > MzMine grab pages from HMDB? Also how many do you grab at a time? An easy > solution would be for me to increase the allowed burst size or allowed rate > to a high enough number to allow the MzMine software to continue to function > properly. > > Kind regards, > > Mike > > PS here is the message from the user: > > I am currently using mzMine, and searching for metabolite identification > using the HMDB. During its search, the following message appears: > 'Could not connect to HMDB: java.io.IOException: > Server returned HTTP response code: 503 for URL > http://www.hmdb.ca/metabolites/HMDB11601(null:-1)' > Is there anything that I could do to rectify this problem and return all > metabolite possibilities? > > -- > Michael Wilson > Bioinformatic Systems Analyst > michael.wil...@ualberta.ca > 780.492.2315 (office) > 780.289.4026 (mobile) > > University of Alberta > Department of Computing Science > 116 St. and 85 Ave. > Edmonton, Alberta T6G 2R3 > > > > > ------------------------------------------------------------------------------ > Keep yourself connected to Go Parallel: > VERIFY Test and improve your parallel project with help from experts > and peers. > http://goparallel.sourceforge.net_______________________________________________ > Mzmine-devel mailing list > Mzmine-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/mzmine-devel =============================================== Tomáš Pluskal G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University 1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan WWW: https://groups.oist.jp/g0 TEL: +81-98-966-8684 Fax: +81-98-966-2890 ------------------------------------------------------------------------------ Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net _______________________________________________ Mzmine-devel mailing list Mzmine-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mzmine-devel
