It looks like I let the cat out of the bag on this one... Some discussion among the community would be good to establish what we want from a bulk loader. For example, I would like all objects created and linked, from samples (with annotations) right up to raw bioassays (grouped into an experiment as Micha does, ideally). Obviously to do this you need to specify the relationships between all the samples, annotations, dyes, hybs, data files, but that could easily end up being as much work as manually loading the data ;-)
Therefore you have to make compromises; the main one being that you treat every sample/extract/etc in exactly the same way (same protocols, same amounts, same labels, same array design). This would probably work for most people - but let's discuss it anyway... One issue with the bulk loader I wrote for BASE 1 is that if your experiment involves a dye swap for one biological replicate, the bulk loader will label each extract with both dyes for *all* replicates (leaving you with unused labelled extracts after linking to the hybs). One quite low cost way to define your experiment for a bulk loader is Tab2MAGE (soon to be MAGE-TAB, see http://www.biomedcentral.com/1471-2105/7/489/abstract ) - although it feels strange to say this because I have been *exporting* Tab2MAGE from BASE 1.2 rather than importing it. The input files for our bulk loader are specified here: http://wiki.vectorbase.org/vectorbase/index.php/MicroarrayExperiments (look for the example file links) but note that the BASE-specific parameters (protocols, raw file parsers, array designs, etc) are configured by the user in an interactive process before the upload. Our bulk loader is also 2-channel only... I have a feeling that the Tab2MAGE parser could be adapted into a BASE 2 bulk loader somehow, although I think all Tab2MAGE code is Perl. I think MAGE-TAB is something we could get the average bench scientist to provide (with a little help MGEDifying the annotations). So far every experiment I have loaded has come with some kind of spreadsheet (each one in a different format of course), so the leap to MAGE-TAB is not so great. Ok, enough from me... Micha Bayer writes: > > > > -----Original Message----- > > From: [EMAIL PROTECTED] [mailto:basedb-users- > > [EMAIL PROTECTED] On Behalf Of Nicklas Nordborg > > Sent: 29 November 2006 14:40 > > To: BASE ML > > Subject: Re: [base] Offline files > > > > Jari Häkkinen wrote: > > > Hi all, > > > > > > Just a thought, I think the best choice to trigger a large import like > > > this is to create a plug-in instead of adding extra tabs to the file > > > browser. The plug-in should start by requesting the zip file/directory > > > to import. If the zip file is not already uploaded the user will have an > > > opportunity to upload it. After choosing the zip file/directory the > > > plug-in could start its import and creation of necessary items. > > > > > > The above is fairly straightforward already today and the new code would > > > start at import/creation of necessary items. > > > > > > This will yield very little (or no) changes to the core and the plug-in > > > should probably be started from the experiment listing page. This will > > > of course change the experiment list jsp (a import tab is needed). > > > > No change is needed to the jsp. The new plugin will be detected and an > > "Import" button will appear. > > > > /Nicklas > > So what interface would I have to implement for this to happen, if I were to > turn my code into a plugin? > > If it's not too much work I might consider doing this, that would be much > better. > > Cheers > > Micha > > ================================== > Dr Micha M Bayer > Bioinformatics Specialist > Genetics Programme > The Scottish Crop Research Institute > Invergowrie > Dundee > DD2 5DA > Scotland, UK > Telephone +44(0)1382 562731 ext. 2309 > Fax +44(0)1382 562426 > http://www.scri.sari.ac.uk/MichaBayer.htm > ============================= > > > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > DISCLAIMER: > > This email is from the Scottish Crop Research Institute, but the views > expressed by the sender are not necessarily the views of SCRI and its > subsidiaries. This email and any files transmitted with it are confidential > to the intended recipient at the e-mail address to which it has been > addressed. It may not be disclosed or used by any other than that addressee. > If you are not the intended recipient you are requested to preserve this > confidentiality and you must not use, disclose, copy, print or rely on this > e-mail in any way. Please notify [EMAIL PROTECTED] quoting the > name of the sender and delete the email from your system. > > Although SCRI has taken reasonable precautions to ensure no viruses are > present in this email, neither the Institute nor the sender accepts any > responsibility for any viruses, and it is your responsibility to scan the > email > and the attachments (if any). > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > The BASE general discussion mailing list > basedb-users@lists.sourceforge.net > unsubscribe: send a mail with subject "unsubscribe" to > [EMAIL PROTECTED] -- Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | Division of Cell and Molecular Biology | Imperial College London | Phone +442075941945 | Email [EMAIL PROTECTED] ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject "unsubscribe" to [EMAIL PROTECTED]
