Hi Michael,
I am also sorry for the late reply. I can update the MZmine module to divide
the whole peak list searches into batches of e.g 100 masses. In that case if
the user searches a peaklist of 2000 entries, it will perform 20 searches of
100 masses (serially, of course). Will that be acceptable for you?
Please give me some time to implement this. I think I can release an updated
version of MZmine within January.
The idea to download the local copy of the database is an interesting one, it
would greatly speed up the searches, indeed. But this requires more substantial
redesign of the search module in MZmine, so it will take more time. I would
like to switch to the bulk search first, and then work on this idea later.
Best regards,
Tomas
On Dec 10, 2012, at 2:31 PM, Michael Wilson
<michael.wil...@ualberta.ca<mailto:michael.wil...@ualberta.ca>> wrote:
Hi Tomas,
Sorry for the delay in my reply. The requests are serial but occur in rapid
succession which was causing a lot of memory use, the current limit I have set
is 20 requests per minute. The manner in which you are currently querying is
not optimal, we do have a method for querying in bulk, available here
http://www.hmdb.ca/spectra/ms/search
Another possible solution would be to download the masses of the compounds and
corresponding HMDB IDs and then run the query in your program. I suspect this
would greatly speed up getting your program for your users as well. If you are
concerned with always having the latest compounds available it would still be
more efficient if you downloaded all the IDs and compound masses at the start
of every MzMine session. I could make a download link available for you that
contains this information in a tab delimited format, the file would be around
287 kb when zipped so it would be a very quick download.
Would any of these options work for you?
Mike
On Thu, Nov 29, 2012 at 6:47 PM, Tomas Pluskal
<plus...@oist.jp<mailto:plus...@oist.jp>> wrote:
Hi Michael,
Thanks for contacting us. Let's work together to resolve this issue.
In MZmine, the user may choose a single detected peak (=mass value), or a peak
list full of detected masses, and submit them to HMDB search.
I suppose a single mass search will never exceed your access limits, so the
problem might arise when the whole peak list is selected.
A single peak list may contain potentially thousands of mass values, depending
on the analytical method and parameters used for peak detection.
MZmine searches for the masses in HMDB one-by-one in a serial fashion - each
new request is sent after the previous request results are received. There are
no multiple parallel connections to HMDB from a single MZmine instance.
The search process works as follows:
1) For each mass that is searched, MZmine will first access this URL:
http://www.hmdb.ca/chemquery/mass?&query_from=XXX&query_to=YYY to find
compounds within certain tolerance range of that mass
2) For each of the compounds found, MZmine then downloads the details page at
this URL: http://www.hmdb.ca/metabolites/HMDB12345 (only the actual HTML result
is downloaded, no images or scripts etc). There is a limit for how many
compounds MZmine will download from a single mass search. I think most users
set this limit to 10 or 100.
3) After all the individual results are downloaded, MZmine will proceed to the
next mass search (step 1)
As you can see from the above description, a single search operation on a peak
list containing thousands of entries might generate tens of thousands of page
requests to HMDB. On the other hand, these requests are generated serially, so
a single MZmine user would never overload your web server. The problem might
occur if there are many MZmine users performing searches at the same time.
What is the current access limit you imposed on the HMDB website?
Do you have any suggestion how to optimize the search process?
Best regards,
Tomas
On Nov 30, 2012, at 2:48 AM, Michael Wilson
<michael.wil...@ualberta.ca<mailto:michael.wil...@ualberta.ca>> wrote:
> Hello,
>
> I am one of the maintainers/developers of HMDB, we recently had to limit the
> number of hits per minute to HMDB as some users were sending huge numbers of
> requests for an extended period of time which was affecting performance of
> the site. Any hits that exceed the rate limit will get 503 errors. I have
> recently received a complaint from a user of your MzMine software that the
> software is throwing an error due to a 503 error being returned by HMDB
> implying it is exceeding the access limit.
>
> I would like to modify the limit to enable your software to keep working, but
> need some information from your team to enable me to do so. At what rate does
> MzMine grab pages from HMDB? Also how many do you grab at a time? An easy
> solution would be for me to increase the allowed burst size or allowed rate
> to a high enough number to allow the MzMine software to continue to function
> properly.
>
> Kind regards,
>
> Mike
>
> PS here is the message from the user:
>
> I am currently using mzMine, and searching for metabolite identification
> using the HMDB. During its search, the following message appears:
> 'Could not connect to HMDB: java.io.IOException:
> Server returned HTTP response code: 503 for URL
> http://www.hmdb.ca/metabolites/HMDB11601(null:-1)'
> Is there anything that I could do to rectify this problem and return all
> metabolite possibilities?
>
> --
> Michael Wilson
> Bioinformatic Systems Analyst
> michael.wil...@ualberta.ca<mailto:michael.wil...@ualberta.ca>
> 780.492.2315<tel:780.492.2315> (office)
> 780.289.4026<tel:780.289.4026> (mobile)
>
> University of Alberta
> Department of Computing Science
> 116 St. and 85 Ave.
> Edmonton, Alberta T6G 2R3
>
>
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> VERIFY Test and improve your parallel project with help from experts
> and peers.
> http://goparallel.sourceforge.net_______________________________________________<http://goparallel.sourceforge.net_______________________________________________/>
> Mzmine-devel mailing list
> Mzmine-devel@lists.sourceforge.net<mailto:Mzmine-devel@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/mzmine-devel
===============================================
Tomáš Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684<tel:%2B81-98-966-8684>
Fax: +81-98-966-2890<tel:%2B81-98-966-2890>
--
Michael Wilson
Bioinformatic Systems Analyst
michael.wil...@ualberta.ca<mailto:michael.wil...@ualberta.ca>
780.492.2315 (office)
780.289.4026 (mobile)
University of Alberta
Department of Computing Science
116 St. and 85 Ave.
Edmonton, Alberta T6G 2R3
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
===============================================
Tomas Pluskal
G0 Cell Unit, Okinawa Institute of Science and Technology Graduate University
1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
WWW: https://groups.oist.jp/g0
TEL: +81-98-966-8684
Fax: +81-98-966-2890
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
