Philippe Rocca-Serra writes: > Hi Everyone, > > As Dominic pointed out, we are working anyway on an import into BASE2 > using tab2mage format in the short time (see ticket 338), and once > MAGE-TAB specifications are stable move on to support this format too.
excellent! > There is a MAGE-TAB jamboree happening in Philadelphia next week, so I > will count on Helen to update us on the outcome of this meeting so we > can possibly expedite the process. will there be Java code to parse MAGE-TAB? I guess that would speed up the development of the BASE importer. > The tab2mage import will prove no doubt extremely useful to exchange > data and to get data from repositories as pointed out > So everything is in the pipeline and we jsut need a little bit a > patience, days have only 24 hours > > cheers > > Philippe > > -- > **************************************************************** > Philippe Rocca-Serra (Ph.D) > Nutri/Toxicogenomics Internal Coordinator > EBI- The European Bioinformatics Institute > EMBL Outstation Hinxton > > Wellcome Trust Genome Campus > CB10 1SD Cambridge, UK > > email: [EMAIL PROTECTED] > Tel: +44 (0) 1223 492 553 > Fax: +44 (0) 1223 494 468 > http://www.ebi.ac.uk/microarray/Projects/tox-nutri/index.html > > **************************************************************** > > > > > > Helen Parkinson wrote: > > Hi, > > > > this is an interesting problem and one we have been dealing with at > > ArrayExpress as well. For really large experiments the only thing that > > that has worked for submissions is a spreadsheet - currently we use > > tab2mage for this purpose. *tab2mage*.sourceforge.net/ > > > > > > You might think about an importer that takes a spreadsheet, or partial > > spreadsheet plus files in an archive as input, e.g. MAGETAB, this would > > allow people to pull in other people's data downloaded from public > > repositories, and allow biologists to format up their own data easily. > > And as spreadsheet export from BASE is planned the import would also be > > nice, round tripping is also a good test of format and application > > > > cheers > > > > Helen > > > > Jari Häkkinen wrote: > > > >> There is a possibility to get access to the subversion repository and > >> trac environment just for this kind of stuff. Check out > >> http://baseplugins.thep.lu.se Trac environment allows for discussions > >> about the importer through the ticketing system. > >> > >> > >> Jari > >> > >> > >> Micha Bayer wrote: > >> > >> > >>> That sounds like a hell of a lot more work than what I have done so > >>> far.... :-) > >>> > >>> My first shot at this has been quite basic and it only creates new > >>> RawBioAssays and a single Experiment that these belong to. It could of > >>> course be extended to do more stuff, but my time allowance for this is > >>> limited and I don't think I'll be able to spend a lot more time on this > >>> than I already have. > >>> > >>> I think the upshot of all this discussion has been: > >>> > >>> 1. people need a batch importer (desperately, in fact), and > >>> 2. a plugin is much better and will (hopefully) be possible > >>> > >>> With that in mind, I'll go away and look into turning my stuff into a > >>> plugin now, and when I have got the basic version of that working (i.e. > >>> file upload, RawBioAssay and Experiment creation only, as above) I'll > >>> make the code available to the community somehow and someone else can > >>> pick it up from there if they want to make it fancier. I hope that's > >>> okay -- even with no extra effort from someone else it'll still mean > >>> that a hell of a lot of effort can be saved. > >>> > >>> Perhaps the dev team can point me in the right direction initially. My > >>> colleague Iain here is just working on a file importer plugin that > >>> handles Excel files and his code starts off like this: > >>> > >>> > >>> public class ExcelFileImporter extends AbstractPlugin implements > >>> InteractivePlugin, AutoDetectingImporter > >>> { > >>> private static final Set<GuiContext> guiContexts = > >>> Collections.singleton(new GuiContext(Item.RAWBIOASSAY, > >>> GuiContext.Type.ITEM)); > >>> > >>> > >>> So am I right in thinking that if I change this to the following... > >>> > >>> > >>> public class ZipFileImporter extends AbstractPlugin implements > >>> InteractivePlugin, AutoDetectingImporter > >>> { > >>> private static final Set<GuiContext> guiContexts = > >>> Collections.singleton(new GuiContext(Item.EXPERIMENT, > >>> GuiContext.Type.ITEM)); > >>> > >>> > >>> ....then I'll get an import tab created in the Experiment list view > >>> which will let me import the zip file using my plugin? > >>> > >>> We're planning to make the Excel plugin available for everyone too BTW, > >>> once it's finished. > >>> > >>> Cheers > >>> Micha > >>> > >>> ================================== > >>> Dr Micha M Bayer > >>> Bioinformatics Specialist > >>> Genetics Programme > >>> The Scottish Crop Research Institute > >>> Invergowrie > >>> Dundee > >>> DD2 5DA > >>> Scotland, UK > >>> Telephone +44(0)1382 562731 ext. 2309 > >>> Fax +44(0)1382 562426 > >>> http://www.scri.sari.ac.uk/MichaBayer.htm > >>> ============================= > >>> > >>> > >>> > >>> > >>>> -----Original Message----- > >>>> From: [EMAIL PROTECTED] [mailto:basedb-users- > >>>> [EMAIL PROTECTED] On Behalf Of Bob MacCallum > >>>> Sent: 29 November 2006 18:36 > >>>> To: BASE ML > >>>> Subject: Re: [base] Offline files > >>>> > >>>> > >>>> It looks like I let the cat out of the bag on this one... > >>>> > >>>> Some discussion among the community would be good to establish what we > >>>> want > >>>> from a bulk loader. For example, I would like all objects created and > >>>> linked, > >>>> from samples (with annotations) right up to raw bioassays (grouped into > >>>> an > >>>> experiment as Micha does, ideally). Obviously to do this you need to > >>>> specify > >>>> the relationships between all the samples, annotations, dyes, hybs, data > >>>> files, but that could easily end up being as much work as manually > >>>> loading > >>>> the > >>>> data ;-) > >>>> > >>>> Therefore you have to make compromises; the main one being that you > >>>> treat > >>>> every sample/extract/etc in exactly the same way (same protocols, same > >>>> amounts, same labels, same array design). This would probably work for > >>>> most > >>>> people - but let's discuss it anyway... > >>>> > >>>> One issue with the bulk loader I wrote for BASE 1 is that if your > >>>> experiment > >>>> involves a dye swap for one biological replicate, the bulk loader will > >>>> label > >>>> each extract with both dyes for *all* replicates (leaving you with > >>>> unused > >>>> labelled extracts after linking to the hybs). > >>>> > >>>> One quite low cost way to define your experiment for a bulk loader is > >>>> Tab2MAGE > >>>> (soon to be MAGE-TAB, see > >>>> http://www.biomedcentral.com/1471-2105/7/489/abstract ) - although it > >>>> feels > >>>> strange to say this because I have been *exporting* Tab2MAGE from BASE > >>>> 1.2 > >>>> rather than importing it. The input files for our bulk loader are > >>>> specified > >>>> here: > >>>> http://wiki.vectorbase.org/vectorbase/index.php/MicroarrayExperiments > >>>> (look for the example file links) but note that the BASE-specific > >>>> parameters > >>>> (protocols, raw file parsers, array designs, etc) are configured by the > >>>> user > >>>> in an interactive process before the upload. Our bulk loader is also > >>>> 2-channel only... > >>>> > >>>> I have a feeling that the Tab2MAGE parser could be adapted into a BASE 2 > >>>> bulk > >>>> loader somehow, although I think all Tab2MAGE code is Perl. > >>>> > >>>> I think MAGE-TAB is something we could get the average bench scientist > >>>> to > >>>> provide (with a little help MGEDifying the annotations). So far every > >>>> experiment I have loaded has come with some kind of spreadsheet (each > >>>> one > >>>> in a > >>>> different format of course), so the leap to MAGE-TAB is not so great. > >>>> > >>>> Ok, enough from me... > >>>> > >>>> > >>>> > >>>> > >>>> Micha Bayer writes: > >>>> > > >>>> > > >>>> > > -----Original Message----- > >>>> > > From: [EMAIL PROTECTED] [mailto:basedb- > >>>> users- > >>>> > > [EMAIL PROTECTED] On Behalf Of Nicklas Nordborg > >>>> > > Sent: 29 November 2006 14:40 > >>>> > > To: BASE ML > >>>> > > Subject: Re: [base] Offline files > >>>> > > > >>>> > > Jari Häkkinen wrote: > >>>> > > > Hi all, > >>>> > > > > >>>> > > > Just a thought, I think the best choice to trigger a large import > >>>> like > >>>> > > > this is to create a plug-in instead of adding extra tabs to the > >>>> file > >>>> > > > browser. The plug-in should start by requesting the zip > >>>> file/directory > >>>> > > > to import. If the zip file is not already uploaded the user will > >>>> have an > >>>> > > > opportunity to upload it. After choosing the zip file/directory > >>>> the > >>>> > > > plug-in could start its import and creation of necessary items. > >>>> > > > > >>>> > > > The above is fairly straightforward already today and the new > >>>> code > >>>> would > >>>> > > > start at import/creation of necessary items. > >>>> > > > > >>>> > > > This will yield very little (or no) changes to the core and the > >>>> plug-in > >>>> > > > should probably be started from the experiment listing page. This > >>>> will > >>>> > > > of course change the experiment list jsp (a import tab is > >>>> needed). > >>>> > > > >>>> > > No change is needed to the jsp. The new plugin will be detected and > >>>> an > >>>> > > "Import" button will appear. > >>>> > > > >>>> > > /Nicklas > >>>> > > >>>> > So what interface would I have to implement for this to happen, if I > >>>> were to turn my code into a plugin? > >>>> > > >>>> > If it's not too much work I might consider doing this, that would be > >>>> much better. > >>>> > > >>>> > Cheers > >>>> > > >>>> > Micha > >>>> > > >>>> > ================================== > >>>> > Dr Micha M Bayer > >>>> > Bioinformatics Specialist > >>>> > Genetics Programme > >>>> > The Scottish Crop Research Institute > >>>> > Invergowrie > >>>> > Dundee > >>>> > DD2 5DA > >>>> > Scotland, UK > >>>> > Telephone +44(0)1382 562731 ext. 2309 > >>>> > Fax +44(0)1382 562426 > >>>> > http://www.scri.sari.ac.uk/MichaBayer.htm > >>>> > ============================= > >>>> > > >>>> > > >>>> > > >>>> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > >>>> _ _ > >>>> _ _ > >>>> > > >>>> > DISCLAIMER: > >>>> > > >>>> > This email is from the Scottish Crop Research Institute, but the > >>>> views > >>>> > expressed by the sender are not necessarily the views of SCRI and its > >>>> > subsidiaries. This email and any files transmitted with it are > >>>> confidential > >>>> > to the intended recipient at the e-mail address to which it has been > >>>> > addressed. It may not be disclosed or used by any other than that > >>>> addressee. > >>>> > If you are not the intended recipient you are requested to preserve > >>>> this > >>>> > confidentiality and you must not use, disclose, copy, print or rely > >>>> on > >>>> this > >>>> > e-mail in any way. Please notify [EMAIL PROTECTED] quoting the > >>>> > name of the sender and delete the email from your system. > >>>> > > >>>> > Although SCRI has taken reasonable precautions to ensure no viruses > >>>> are > >>>> > present in this email, neither the Institute nor the sender accepts > >>>> any > >>>> > responsibility for any viruses, and it is your responsibility to scan > >>>> the email > >>>> > and the attachments (if any). > >>>> > > >>>> > > >>>> > > >>>> ----------------------------------------------------------------------- > >>>> -- > >>>> > Take Surveys. Earn Cash. Influence the Future of IT > >>>> > Join SourceForge.net's Techsay panel and you'll get the chance to > >>>> share > >>>> your > >>>> > opinions on IT & business topics through brief surveys - and earn > >>>> cash > >>>> > > >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > >>>> > _______________________________________________ > >>>> > The BASE general discussion mailing list > >>>> > basedb-users@lists.sourceforge.net > >>>> > unsubscribe: send a mail with subject "unsubscribe" to > >>>> > [EMAIL PROTECTED] > >>>> > >>>> -- > >>>> Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | > >>>> Division of Cell and Molecular Biology | Imperial College London | > >>>> Phone +442075941945 | Email [EMAIL PROTECTED] > >>>> > >>>> ------------------------------------------------------------------------- > >>>> Take Surveys. Earn Cash. Influence the Future of IT > >>>> Join SourceForge.net's Techsay panel and you'll get the chance to share > >>>> your > >>>> opinions on IT & business topics through brief surveys - and earn cash > >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > >>>> _______________________________________________ > >>>> The BASE general discussion mailing list > >>>> basedb-users@lists.sourceforge.net > >>>> unsubscribe: send a mail with subject "unsubscribe" to > >>>> [EMAIL PROTECTED] > >>>> > >>>> > >>> ------------------------------------------------------------------------- > >>> Take Surveys. Earn Cash. Influence the Future of IT > >>> Join SourceForge.net's Techsay panel and you'll get the chance to share > >>> your > >>> opinions on IT & business topics through brief surveys - and earn cash > >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > >>> _______________________________________________ > >>> The BASE general discussion mailing list > >>> basedb-users@lists.sourceforge.net > >>> unsubscribe: send a mail with subject "unsubscribe" to > >>> [EMAIL PROTECTED] > >>> > >>> > >>> > >> > >> > > > > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > The BASE general discussion mailing list > basedb-users@lists.sourceforge.net > unsubscribe: send a mail with subject "unsubscribe" to > [EMAIL PROTECTED] -- Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | Division of Cell and Molecular Biology | Imperial College London | Phone +442075941945 | Email [EMAIL PROTECTED] ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject "unsubscribe" to [EMAIL PROTECTED]
