Hi Tomas,
Happy New Year! Let me know if you run into any problems using the bulk
search tool.
Mike
On Mon, Dec 17, 2012 at 10:32 PM, Tomas Pluskal <plus...@oist.jp> wrote:
> Hi Michael,
>
> I am also sorry for the late reply. I can update the MZmine module to
> divide the whole peak list searches into batches of e.g 100 masses. In that
> case if the user searches a peaklist of 2000 entries, it will perform 20
> searches of 100 masses (serially, of course). Will that be acceptable for
> you?
>
> Please give me some time to implement this. I think I can release an
> updated version of MZmine within January.
>
> The idea to download the local copy of the database is an interesting
> one, it would greatly speed up the searches, indeed. But this requires more
> substantial redesign of the search module in MZmine, so it will take more
> time. I would like to switch to the bulk search first, and then work on
> this idea later.
>
> Best regards,
>
> Tomas
>
>
>
> On Dec 10, 2012, at 2:31 PM, Michael Wilson <michael.wil...@ualberta.ca>
> wrote:
>
> Hi Tomas,
>
> Sorry for the delay in my reply. The requests are serial but occur in
> rapid succession which was causing a lot of memory use, the current limit I
> have set is 20 requests per minute. The manner in which you are currently
> querying is not optimal, we do have a method for querying in bulk,
> available here http://www.hmdb.ca/spectra/ms/search
>
>
> Another possible solution would be to download the masses of the
> compounds and corresponding HMDB IDs and then run the query in your
> program. I suspect this would greatly speed up getting your program for
> your users as well. If you are concerned with always having the latest
> compounds available it would still be more efficient if you downloaded all
> the IDs and compound masses at the start of every MzMine session. I could
> make a download link available for you that contains this information in a
> tab delimited format, the file would be around 287 kb when zipped so it
> would be a very quick download.
>
> Would any of these options work for you?
>
> Mike
>
>
>
>
> On Thu, Nov 29, 2012 at 6:47 PM, Tomas Pluskal <plus...@oist.jp> wrote:
>
>> Hi Michael,
>>
>> Thanks for contacting us. Let's work together to resolve this issue.
>>
>> In MZmine, the user may choose a single detected peak (=mass value), or a
>> peak list full of detected masses, and submit them to HMDB search.
>> I suppose a single mass search will never exceed your access limits, so
>> the problem might arise when the whole peak list is selected.
>> A single peak list may contain potentially thousands of mass values,
>> depending on the analytical method and parameters used for peak detection.
>>
>> MZmine searches for the masses in HMDB one-by-one in a serial fashion -
>> each new request is sent after the previous request results are received.
>> There are no multiple parallel connections to HMDB from a single MZmine
>> instance.
>>
>> The search process works as follows:
>> 1) For each mass that is searched, MZmine will first access this URL:
>> http://www.hmdb.ca/chemquery/mass?&query_from=XXX&query_to=YYY to find
>> compounds within certain tolerance range of that mass
>> 2) For each of the compounds found, MZmine then downloads the details
>> page at this URL: http://www.hmdb.ca/metabolites/HMDB12345 (only the
>> actual HTML result is downloaded, no images or scripts etc). There is a
>> limit for how many compounds MZmine will download from a single mass
>> search. I think most users set this limit to 10 or 100.
>> 3) After all the individual results are downloaded, MZmine will proceed
>> to the next mass search (step 1)
>>
>> As you can see from the above description, a single search operation on a
>> peak list containing thousands of entries might generate tens of thousands
>> of page requests to HMDB. On the other hand, these requests are generated
>> serially, so a single MZmine user would never overload your web server. The
>> problem might occur if there are many MZmine users performing searches at
>> the same time.
>>
>> What is the current access limit you imposed on the HMDB website?
>> Do you have any suggestion how to optimize the search process?
>>
>> Best regards,
>>
>> Tomas
>>
>>
>> On Nov 30, 2012, at 2:48 AM, Michael Wilson <michael.wil...@ualberta.ca>
>> wrote:
>>
>> > Hello,
>> >
>> > I am one of the maintainers/developers of HMDB, we recently had to
>> limit the number of hits per minute to HMDB as some users were sending huge
>> numbers of requests for an extended period of time which was affecting
>> performance of the site. Any hits that exceed the rate limit will get 503
>> errors. I have recently received a complaint from a user of your MzMine
>> software that the software is throwing an error due to a 503 error being
>> returned by HMDB implying it is exceeding the access limit.
>> >
>> > I would like to modify the limit to enable your software to keep
>> working, but need some information from your team to enable me to do so. At
>> what rate does MzMine grab pages from HMDB? Also how many do you grab at a
>> time? An easy solution would be for me to increase the allowed burst size
>> or allowed rate to a high enough number to allow the MzMine software to
>> continue to function properly.
>> >
>> > Kind regards,
>> >
>> > Mike
>> >
>> > PS here is the message from the user:
>> >
>> > I am currently using mzMine, and searching for metabolite
>> identification using the HMDB. During its search, the following message
>> appears:
>> > 'Could not connect to HMDB: java.io.IOException:
>> > Server returned HTTP response code: 503 for URL
>> > http://www.hmdb.ca/metabolites/HMDB11601(null:-1)'
>> > Is there anything that I could do to rectify this problem and return
>> all metabolite possibilities?
>> >
>> > --
>> > Michael Wilson
>> > Bioinformatic Systems Analyst
>> > michael.wil...@ualberta.ca
>> > 780.492.2315 (office)
>> > 780.289.4026 (mobile)
>> >
>> > University of Alberta
>> > Department of Computing Science
>> > 116 St. and 85 Ave.
>> > Edmonton, Alberta T6G 2R3
>> >
>> >
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Keep yourself connected to Go Parallel:
>> > VERIFY Test and improve your parallel project with help from experts
>> > and peers.
>> http://goparallel.sourceforge.net_______________________________________________
>> > Mzmine-devel mailing list
>> > Mzmine-devel@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/mzmine-devel
>>
>> ===============================================
>> Tomáš Pluskal
>> G0 Cell Unit, Okinawa Institute of Science and Technology Graduate
>> University
>> 1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
>> WWW: https://groups.oist.jp/g0
>> TEL: +81-98-966-8684
>> Fax: +81-98-966-2890
>>
>>
>
>
> --
> Michael Wilson
> Bioinformatic Systems Analyst
> michael.wil...@ualberta.ca
> 780.492.2315 (office)
> 780.289.4026 (mobile)
>
> University of Alberta
> Department of Computing Science
> 116 St. and 85 Ave.
> Edmonton, Alberta T6G 2R3
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
>
> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
>
> Mzmine-devel mailing list
> Mzmine-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mzmine-devel
>
>
> ===============================================
> Tomas Pluskal
>
> G0 Cell Unit, Okinawa Institute of Science and Technology Graduate
> University
> 1919-1 Tancha, Onna-son, Okinawa 904-0495, Japan
> WWW: https://groups.oist.jp/g0
> TEL: +81-98-966-8684
> Fax: +81-98-966-2890
>
>
--
Michael Wilson
Bioinformatic Systems Analyst
michael.wil...@ualberta.ca
780.492.2315 (office)
780.289.4026 (mobile)
University of Alberta
Department of Computing Science
116 St. and 85 Ave.
Edmonton, Alberta T6G 2R3
------------------------------------------------------------------------------
Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
much more. Get web development skills now with LearnDevNow -
350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122812
_______________________________________________
Mzmine-devel mailing list
Mzmine-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mzmine-devel
