[galaxy-dev] 回复： 回复： 回复： Speed up the galaxy
Hi Alex Now is an another problem. I now deal with two fastq files, there are Illumina enconding 1.8 and pair-end, so I don't need to groom. But the fact is, I need to use the filter by quality and Fastq interlacer and without groom thses two tools can not regonize the files. Any idea to solve this problem? 发件人： Bossers, Alex alex.boss...@wur.nl 收件人： 泽 蔡 caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu 发送日期： 2012年12月5日, 星期三, 4:02 上午 主题: RE: 回复： [galaxy-dev] 回复： Speed up the galaxy It seems old Illumina encoding 1.5 So yes some tools requiring the fastqsanger would need grooming. If you are up to programming you can seriously speed this up by using a precalculated transfer or hash table. That way you do not have to do any calculation but just translate each quality line using generic regexp/grep/sed like tools or $seq =~ t/STARTSCORES/SANGERSCORES/g in perl. The wiki table could be an alternative option. You still have to set the upload type to fastsanger likely since it will probably sniff the header and see its fastq but not 1.8+ encoding. Good luck! Alex Van: 泽 蔡 [caizexi...@yahoo.com.cn] Verzonden: dinsdag 4 december 2012 16:41 To: Bossers, Alex Cc: galaxy-dev@lists.bx.psu.edu Onderwerp: 回复： [galaxy-dev] 回复： Speed up the galaxy Hi Alex I look the page of wikipedia, but I have a little confused. We sequenced with solexa. I paste a read of my data, can you tell me whether I need run Groom? The read is like this:@HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1 AGAAGTACATCGCGATGCCGTTNCCNNCGAAGGCGATAGNNNACAAGNCCAAATGNTTCTNCATCNNNCNCGAGNNGNCGAGGNCGCCGTGCGACCCTGC +HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1 Ya^a`edddeddc\c`a`dc]\Ba^BBZ]ZZ`ZZZ]a]]BBB^[`\UB_V[V\`ZBSZX^ 发件人： Bossers, Alex alex.boss...@wur.nl 收件人： 泽 蔡 caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu; Nate Coraor n...@bx.psu.edu 发送日期： 2012年12月4日, 星期二, 下午 11:21 主题: RE: [galaxy-dev] 回复： Speed up the galaxy Hi I presume the best way to optimise your current problem is to evaluate whether you really need to groom your data!? If its old data presumably yes, but if it is recent data in Illumina 1.8+ encoding(http://en.wikipedia.org/wiki/FASTQ_format) it is not necessary speedup 100% :-) Groom would take long at our servers as well but due to the new Illumina format we didn’t bother to optimise it further by parallelisation for instance... Alex Van: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens ? ? Verzonden: dinsdag 4 december 2012 16:08 Aan: Nate Coraor CC: galaxy-dev@lists.bx.psu.edu Onderwerp: [galaxy-dev] 回复： Speed up the galaxy Hi， I read the page of the document. But I don't think there is anything I need. Now, I installed a local instance of galaxy, and I just run it on a PC. I need to deal with large data, if I just use the default configure of galaxy, every task would take long time. For example I ran the FASTQ Groomer with a large file, the precedure is so slow, and galaxy would not use the potential of my machine. So I want to know how can I get galaxy ran faster. I already can upload files quickly and I just need to know how to ran tools quickly. 发件人： Nate Coraor n...@bx.psu.edumailto:n...@bx.psu.edu 收件人： 泽 蔡 caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn 抄送： galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu 发送日期： 2012年12月4日, 星期二, 下午 9:38 主题: Re: [galaxy-dev] Speed up the galaxy On Dec 4, 2012, at 4:36 AM, 泽 蔡 wrote: Hi all, How can I speed up the galaxy? Like how to use more cores and memeries. Hi, This is a pretty broad question. However, I would recommend that you start at: http://usegalaxy.org/production --nate ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] 回复： 回复： 回复： Speed up the galaxy
You can just change the format of the data from fastq to fastqsanger if you're sure about the error format (use the pencil, then datatype tab) note: fastqsanger !=fastqcsanger Brad On Dec 7, 2012, at 3:20 AM, 泽 蔡 caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn wrote: Hi Alex Now is an another problem. I now deal with two fastq files, there are Illumina enconding 1.8 and pair-end, so I don't need to groom. But the fact is, I need to use the filter by quality and Fastq interlacer and without groom thses two tools can not regonize the files. Any idea to solve this problem? 发件人： Bossers, Alex alex.boss...@wur.nlmailto:alex.boss...@wur.nl 收件人： 泽 蔡 caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu 发送日期： 2012年12月5日, 星期三, 4:02 上午 主题: RE: 回复： [galaxy-dev] 回复： Speed up the galaxy It seems old Illumina encoding 1.5 So yes some tools requiring the fastqsanger would need grooming. If you are up to programming you can seriously speed this up by using a precalculated transfer or hash table. That way you do not have to do any calculation but just translate each quality line using generic regexp/grep/sed like tools or $seq =~ t/STARTSCORES/SANGERSCORES/g in perl. The wiki table could be an alternative option. You still have to set the upload type to fastsanger likely since it will probably sniff the header and see its fastq but not 1.8+ encoding. Good luck! Alex Van: 泽 蔡 [caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn] Verzonden: dinsdag 4 december 2012 16:41 To: Bossers, Alex Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Onderwerp: 回复： [galaxy-dev] 回复： Speed up the galaxy Hi Alex I look the page of wikipedia, but I have a little confused. We sequenced with solexa. I paste a read of my data, can you tell me whether I need run Groom? The read is like this:@HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1 AGAAGTACATCGCGATGCCGTTNCCNNCGAAGGCGATAGNNNACAAGNCCAAATGNTTCTNCATCNNNCNCGAGNNGNCGAGGNCGCCGTGCGACCCTGC +HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1 Ya^a`edddeddc\c`a`dc]\Ba^BBZ]ZZ`ZZZ]a]]BBB^[`\UB_V[V\`ZBSZX^ 发件人： Bossers, Alex alex.boss...@wur.nlmailto:alex.boss...@wur.nl 收件人： 泽 蔡 caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu; Nate Coraor n...@bx.psu.edumailto:n...@bx.psu.edu 发送日期： 2012年12月4日, 星期二, 下午 11:21 主题: RE: [galaxy-dev] 回复： Speed up the galaxy Hi I presume the best way to optimise your current problem is to evaluate whether you really need to groom your data!? If its old data presumably yes, but if it is recent data in Illumina 1.8+ encoding(http://en.wikipedia.org/wiki/FASTQ_format) it is not necessary speedup 100% :-) Groom would take long at our servers as well but due to the new Illumina format we didn’t bother to optimise it further by parallelisation for instance... Alex Van: galaxy-dev-boun...@lists.bx.psu.edumailto:galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edumailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens ? ? Verzonden: dinsdag 4 december 2012 16:08 Aan: Nate Coraor CC: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu Onderwerp: [galaxy-dev] 回复： Speed up the galaxy Hi， I read the page of the document. But I don't think there is anything I need. Now, I installed a local instance of galaxy, and I just run it on a PC. I need to deal with large data, if I just use the default configure of galaxy, every task would take long time. For example I ran the FASTQ Groomer with a large file, the precedure is so slow, and galaxy would not use the potential of my machine. So I want to know how can I get galaxy ran faster. I already can upload files quickly and I just need to know how to ran tools quickly. 发件人： Nate Coraor n...@bx.psu.edumailto:n...@bx.psu.edumailto:n...@bx.psu.edumailto:n...@bx.psu.edu 收件人： 泽 蔡 caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn 抄送： galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu 发送日期： 2012年12月4日, 星期二, 下午 9:38 主题: Re: [galaxy-dev] Speed up the galaxy On Dec 4, 2012, at 4:36 AM, 泽 蔡 wrote: Hi all, How can I speed up the galaxy? Like how to use more cores and memeries. Hi, This is a pretty broad question. However, I would recommend that you start at: http://usegalaxy.org/production --nate ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and
[galaxy-dev] 回复： 回复： 回复： 回复： Speed up the galaxy
Hi Brad, I'm sorry, I can not quite follow you. The pencil and the datatype tab are tools in galaxy? 发件人： Langhorst, Brad langho...@neb.com 收件人： 泽 蔡 caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edu Dev galaxy-dev@lists.bx.psu.edu 发送日期： 2012年12月7日, 星期五, 7:53 下午 主题: Re: [galaxy-dev] 回复： 回复： 回复： Speed up the galaxy You can just change the format of the data from fastq to fastqsanger if you're sure about the error format (use the pencil, then datatype tab) note: fastqsanger !=fastqcsanger Brad On Dec 7, 2012, at 3:20 AM, 泽 蔡 caizexi...@yahoo.com.cn wrote: Hi Alex Now is an another problem. I now deal with two fastq files, there are Illumina enconding 1.8 and pair-end, so I don't need to groom. But the fact is, I need to use the filter by quality and Fastq interlacer and without groom thses two tools can not regonize the files. Any idea to solve this problem? 发件人： Bossers, Alex alex.boss...@wur.nl 收件人： 泽 蔡 caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu 发送日期： 2012年12月5日, 星期三, 4:02 上午 主题: RE: 回复： [galaxy-dev] 回复： Speed up the galaxy It seems old Illumina encoding 1.5 So yes some tools requiring the fastqsanger would need grooming. If you are up to programming you can seriously speed this up by using a precalculated transfer or hash table. That way you do not have to do any calculation but just translate each quality line using generic regexp/grep/sed like tools or $seq =~ t/STARTSCORES/SANGERSCORES/g in perl. The wiki table could be an alternative option. You still have to set the upload type to fastsanger likely since it will probably sniff the header and see its fastq but not 1.8+ encoding. Good luck! Alex Van: 泽 蔡 [caizexi...@yahoo.com.cn] Verzonden: dinsdag 4 december 2012 16:41 To: Bossers, Alex Cc: mailto:galaxy-dev@lists.bx.psu.edu Onderwerp: 回复： [galaxy-dev] 回复： Speed up the galaxy Hi Alex I look the page of wikipedia, but I have a little confused. We sequenced with solexa. I paste a read of my data, can you tell me whether I need run Groom? The read is like this:@HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1 AGAAGTACATCGCGATGCCGTTNCCNNCGAAGGCGATAGNNNACAAGNCCAAATGNTTCTNCATCNNNCNCGAGNNGNCGAGGNCGCCGTGCGACCCTGC +HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1 Ya^a`edddeddc\c`a`dc]\Ba^BBZ]ZZ`ZZZ]a]]BBB^[`\UB_V[V\`ZBSZX^ 发件人： Bossers, Alex alex.boss...@wur.nl 收件人： 泽 蔡 caizexi...@yahoo.com.cn 抄送： galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu; Nate Coraor n...@bx.psu.edu 发送日期： 2012年12月4日, 星期二, 下午 11:21 主题: RE: [galaxy-dev] 回复： Speed up the galaxy Hi I presume the best way to optimise your current problem is to evaluate whether you really need to groom your data!? If its old data presumably yes, but if it is recent data in Illumina 1.8+ encoding(http://en.wikipedia.org/wiki/FASTQ_format) it is not necessary speedup 100% :-) Groom would take long at our servers as well but due to the new Illumina format we didn’t bother to optimise it further by parallelisation for instance... Alex Van: mailto:galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens ? ? Verzonden: dinsdag 4 december 2012 16:08 Aan: Nate Coraor CC: mailto:galaxy-dev@lists.bx.psu.edu Onderwerp: [galaxy-dev] 回复： Speed up the galaxy Hi， I read the page of the document. But I don't think there is anything I need. Now, I installed a local instance of galaxy, and I just run it on a PC. I need to deal with large data, if I just use the default configure of galaxy, every task would take long time. For example I ran the FASTQ Groomer with a large file, the precedure is so slow, and galaxy would not use the potential of my machine. So I want to know how can I get galaxy ran faster. I already can upload files quickly and I just need to know how to ran tools quickly. 发件人： Nate Coraor n...@bx.psu.edumailto:n...@bx.psu.edu 收件人： 泽 蔡 caizexi...@yahoo.com.cnmailto:caizexi...@yahoo.com.cn 抄送： galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu galaxy-...@bx.psu.edumailto:galaxy-...@bx.psu.edu 发送日期： 2012年12月4日, 星期二, 下午 9:38 主题: Re: [galaxy-dev] Speed up the galaxy On Dec 4, 2012, at 4:36 AM, 泽 蔡 wrote: Hi all, How can I speed up the galaxy? Like how to use more cores and memeries. Hi, This is a pretty broad question. However, I would recommend that you start at: http://usegalaxy.org/production --nate ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and
Re: [galaxy-dev] binary data types get corrupted on upload
Hi Ido, Is .sqlite a common compressed file type (i.e. zip, gzip, or bz2) just with a different extension? If it is, I think upload.py is always going to try to decompress it. That that has long been a behavior I would like to be able to disable on a per file type basis. If this is the problem and the galaxy team agrees to accept a patch for address the issue, I would be happy to try to implement the fix. -John On Fri, Dec 7, 2012 at 8:39 AM, Ido Tamir ta...@imp.ac.at wrote: from http://wiki.galaxyproject.org/Admin/Datatypes/Adding%20Datatypes I thought that adding subclass=True in datatypes_conf.xml would be enough to register my new type, but it seems like I stil have to add an entry in binary.py for it to be recognized. But its still corrupted. thank you very much, ido -- Tool: Upload File Name: hg19.sqlite Created:Dec 07, 2012 Filesize: 11.1 MB Dbkey: hg19 Format: sqlite ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] error when looking at tool error
If I click on the small bug icon for a dataset in error state, I get this
message in the log:
URL: http://bbc.mdc-berlin.de/galaxy/dataset/errors?id=e96c6ce43a3d239a
File '/data/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line
143 in __call__
app_iter = self.application(environ, start_response)
File '/data/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/data/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in
__call__
body = method( trans, **kwargs )
File
'/data/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/dataset.py',
line 171 in errors
hda = trans.sa_session.query( model.HistoryDatasetAssociation ).get( id )
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 507 in get
return self._get(key, ident)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1500 in _get
return q.all()[0]
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1267 in all
return list(self)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1361 in __iter__
return self._execute_and_instances(context)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1364 in _execute_and_instances
result = self.session.execute(querycontext.statement, params=self._params,
mapper=self._mapper_zero_or_none())
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py',
line 755 in execute
clause, params or {})
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 824 in execute
return Connection.executors[c](self, object, multiparams, params)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 874 in _execute_clauseelement
return self.__execute_context(context)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 896 in __execute_context
self._cursor_execute(context.cursor, context.statement,
context.parameters[0], context=context)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 950 in _cursor_execute
self._handle_dbapi_exception(e, statement, parameters, cursor, context)
File '/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 931 in _handle_dbapi_exception
raise exc.DBAPIError.instance(statement, parameters, e,
connection_invalidated=is_disconnect)
DataError: (DataError) invalid input syntax for integer: e96c6ce43a3d239a
'SELECT history_dataset_association.id AS history_dataset_association_id,
history_dataset_association.history_id AS history_dataset_association_history_id,
history_dataset_association.dataset_id AS history_dataset_association_dataset_id,
history_dataset_association.create_time AS history_dataset_association_create_time,
history_dataset_association.update_time AS history_dataset_association_update_time,
history_dataset_association.state AS history_dataset_association_state,
history_dataset_association.copied_from_history_dataset_association_id AS
history_dataset_association_copied_from_history_dataset_a_1,
history_dataset_association.copied_from_library_dataset_dataset_association_id AS
history_dataset_association_copied_from_library_dataset_d_2, history_dataset_association.hid AS
history_dataset_association_hid, history_dataset_association.name AS
history_dataset_association_name, history_dataset_association.info AS
history_dataset_association_info, history_dataset_association.blurb AS
history_dataset_association_blurb, history_dataset_association.peek AS
history_dataset_association_peek, history_dataset_association.tool_version AS
history_dataset_association_tool_version, history_dataset_association.extension AS
history_dataset_association_extension, history_dataset_association.metadata AS
history_dataset_association_metadata, history_dataset_association.parent_id AS
history_dataset_association_parent_id, history_dataset_association.designation AS
history_dataset_association_designation, history_dataset_association.deleted AS
history_dataset_association_deleted, history_dataset_association.purged AS
history_dataset_association_purged, history_dataset_association.visible AS
history_dataset_association_visible, dataset_1.id AS dataset_1_id, dataset_1.create_time AS
dataset_1_create_time, dataset_1.update_time AS dataset_1_update_time, dataset_1.state AS
dataset_1_state,
Re: [galaxy-dev] binary data types get corrupted on upload
The specification does not mention zip, gzip or bz2 [1] and I am pretty confident, that it does not in addition compresses its content with zip etc.. It starts with a header SQLite format 3\000 which is visible in my file with less. ido [1]http://www.sqlite.org/fileformat.html On Dec 7, 2012, at 3:50 PM, John Chilton wrote: Hi Ido, Is .sqlite a common compressed file type (i.e. zip, gzip, or bz2) just with a different extension? If it is, I think upload.py is always going to try to decompress it. That that has long been a behavior I would like to be able to disable on a per file type basis. If this is the problem and the galaxy team agrees to accept a patch for address the issue, I would be happy to try to implement the fix. -John On Fri, Dec 7, 2012 at 8:39 AM, Ido Tamir ta...@imp.ac.at wrote: from http://wiki.galaxyproject.org/Admin/Datatypes/Adding%20Datatypes I thought that adding subclass=True in datatypes_conf.xml would be enough to register my new type, but it seems like I stil have to add an entry in binary.py for it to be recognized. But its still corrupted. thank you very much, ido -- Tool: Upload File Name: hg19.sqlite Created:Dec 07, 2012 Filesize: 11.1 MB Dbkey: hg19 Format: sqlite ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How to Install FastQC?
Thanks Thon. What is the TOOLSHED option? As far as FASTQC, I downloaded and extracted it, but now what? Do I dump all of those files and directories in tool-data/shared/jars? wget http://www.bioinformatics.babraham.ac.uk/projects/fastqc/fastqc_v0.10.1.zip unzip fastqc_v0.10.1.zip ls FastQC/ Contaminants fastqc fastqc_icon.ico Help INSTALL.txt jbzip2-0.9.jar LICENSE.txt README.txt RELEASE_NOTES.txt run_fastqc.bat sam-1.32.jar Templates uk Thanks, Greg On Thu, Dec 6, 2012 at 6:40 PM, Anthonius deBoer thondeb...@me.com wrote: Hi Greg, Indeed you need to install many many tools to make galaxy run. You may be better off getting the cloud version, which has many tools already installed. Getting GALAXY to run with all the tools that are provided is not a trivial task... It gets MUCH better with the TOOLSHED option, since this can install many of the tools you may need... the old way of galaxy was to rely on many of the tools to be available on the path so they magically run, but things like GATK and FASTQC etc. that are run from JAVA jars, need to be installed manually into that directory tool-data/shared/jars Thon On Dec 06, 2012, at 11:36 AM, greg margeem...@gmail.com wrote: I'm guessing FastQC isn't installed by default? This page give these instructions: http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies FastQC (0.10.0)NGS: QC and manipulationFastQC from Babraham. The main fastqc perl script must be executable from tool-data/shared/jars/FastQC/fastqc But I don't understand what it means. Do I need to download something, change permissions? I don't have anything inside the jars folder. Thanks, Greg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] newbie uploader
On Dec 6, 2012, at 7:16 PM, Smith, Therese wrote: Dear Sir or Madam, We are using DOS prompt ftp, we connected, we sent (8Gbytes), and we received a succcessful completion message from ftp, we saw the file appearing on your page as having been ftp'd to you, however, we received an upload error message, namely: 34: s_8_1_sequence.txt.gz error An error occurred running this job: Traceback (most recent call last): File /galaxy/home/g2main/galaxy_main/tools/data_source/upload.py, line 384, in__main__() File /galaxy/home/g2main/galaxy_main/tools/data_source/upload.py, line 373, in __main__ add_file( dataset We are trying again, with binary channel in ftp,and also would be glad of your advice. Our Galaxy login is: abdul.ban...@uconn.edu Hi Therese, If you could send back an email after your file has finished uploading, but not run the upload tool on it, we can check to make sure it's okay on our filesystem before you attempt to import, and also test importing it ourselves. Thanks, --nate Sincerely, Rouf and Therese ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] error when looking at tool error
On Dec 7, 2012, at 9:54 AM, Andreas Kuntzagk wrote:
If I click on the small bug icon for a dataset in error state, I get this
message in the log:
URL: http://bbc.mdc-berlin.de/galaxy/dataset/errors?id=e96c6ce43a3d239a
File
'/data/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
line 143 in __call__
app_iter = self.application(environ, start_response)
File '/data/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/data/galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in
__call__
body = method( trans, **kwargs )
File
'/data/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/dataset.py',
line 171 in errors
hda = trans.sa_session.query( model.HistoryDatasetAssociation ).get( id )
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 507 in get
return self._get(key, ident)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1500 in _get
return q.all()[0]
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1267 in all
return list(self)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1361 in __iter__
return self._execute_and_instances(context)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/query.py',
line 1364 in _execute_and_instances
result = self.session.execute(querycontext.statement, params=self._params,
mapper=self._mapper_zero_or_none())
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py',
line 755 in execute
clause, params or {})
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 824 in execute
return Connection.executors[c](self, object, multiparams, params)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 874 in _execute_clauseelement
return self.__execute_context(context)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 896 in __execute_context
self._cursor_execute(context.cursor, context.statement,
context.parameters[0], context=context)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 950 in _cursor_execute
self._handle_dbapi_exception(e, statement, parameters, cursor, context)
File
'/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py',
line 931 in _handle_dbapi_exception
raise exc.DBAPIError.instance(statement, parameters, e,
connection_invalidated=is_disconnect)
DataError: (DataError) invalid input syntax for integer: e96c6ce43a3d239a
'SELECT history_dataset_association.id AS history_dataset_association_id,
history_dataset_association.history_id AS
history_dataset_association_history_id,
history_dataset_association.dataset_id AS
history_dataset_association_dataset_id,
history_dataset_association.create_time AS
history_dataset_association_create_time,
history_dataset_association.update_time AS
history_dataset_association_update_time, history_dataset_association.state AS
history_dataset_association_state,
history_dataset_association.copied_from_history_dataset_association_id AS
history_dataset_association_copied_from_history_dataset_a_1,
history_dataset_association.copied_from_library_dataset_dataset_association_id
AS history_dataset_association_copied_from_library_dataset_d_2,
history_dataset_association.hid AS history_dataset_association_hid,
history_dataset_association.name AS history_dataset_association_name,
history_dataset_association.info AS history_dataset_association_info,
history_dataset_associa!
tion.blurb AS history_dataset_association_blurb,
history_dataset_association.peek AS history_dataset_association_peek,
history_dataset_association.tool_version AS
history_dataset_association_tool_version, history_dataset_association.extension
AS history_dataset_association_extension, history_dataset_association.metadata
AS history_dataset_association_metadata, history_dataset_association.parent_id
AS history_dataset_association_parent_id,
history_dataset_association.designation AS
history_dataset_association_designation, history_dataset_association.deleted AS
history_dataset_association_deleted, history_dataset_association.purged AS
history_dataset_association_purged, history_dataset_association.visible AS
history_dataset_association_visible, dataset_1.id AS dataset_1_id,
Re: [galaxy-dev] Defunct munge processes using Torque PBS
Hi Nate. I do understand that this is not a bug directly stemming from the Galaxy code base. Munge is really just a tool to pass user credentials between systems during job submission to the PBS server. Galaxy is spooling jobs through the PBS job runner, which presumably indirectly calls munge through jobs submission to the PBS server. I'm just not sure why the munge process is sometimes becoming corrupt. This is an issue since I rapidly reach my max number of threads for the Galaxy user on my head node. At this point I guess I'll try downloading the latest stable version of Torque and build RPMs. I have been using what is in EPEL for RHEL6. Thanks for the reply, and any other thoughts are still appreciated! On Dec 7, 2012, at 10:51 AM, Nate Coraor n...@bx.psu.edu wrote: On Dec 6, 2012, at 2:34 PM, Matthew Shirley wrote: I am fairly new to PBS management, so I can't rule out some misconfiguration, but I have a strange issue when running Galaxy with the PBS job runner. It seems that munge spawns a bunch of defunct processes after running Galaxy on my cluster: `ps axjf`: 1 25992 25991 25991 ? -1 Sl 7 8:48 python ./scripts/paster.py serve universe_wsgi.ini --daemon 25992 26032 25991 25991 ? -1 Z7 0:00 \_ [munge] defunct 25992 26034 25991 25991 ? -1 Z7 0:00 \_ [munge] defunct 25992 26036 25991 25991 ? -1 Z7 0:00 \_ [munge] defunct Now, these processes are being spawned by Galaxy, and I can't figure out why. Can anyone provide some insight or clues about where to start debugging this? Thanks, Hi Matt, I'm not sure what munge is, it's not something provided with Galaxy. Googling suggests it might be an authentication tool used in some HPC environments. Without having any familiarity with it, I can't say what process in Galaxy would be interacting with it, especially since that interaction must occur implicitly somewhere down the chain of normal Galaxy operations. --nate Matt ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Cannot see data when selecting Display Data in Browser
I'm seeing some weird behavior in our local galaxy instance and am wondering if anyone has seen anything similar. I have a custom tool that I've written that generates a tab-delimted file. When I click on the item name in the history, I can see the contents in the preview window. However, when I click on the eye (Display Data in Browser), it comes up empty. Same thing if I download the file. However, if I go to the database directory to examine the file on disk, the content is exactly what I would expect. The permissions on the file are galaxy user/group and the file has permissions of 644. I don't see any reason why I shouldn't be able to view or download this file through galaxy. Any ideas? Thanks, Dave --- Dave Walton Manager Scientific Computing The Jackson Laboratory Bar Harbor, Maine dave.wal...@jax.org The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Cannot see data when selecting Display Data in Browser
Hey Dave, What revision are you running locally? And, just to confirm, in galaxy the file is recognized as a 'tabular' file type? -Dannon On Dec 7, 2012, at 1:25 PM, Dave Walton dave.wal...@jax.org wrote: I'm seeing some weird behavior in our local galaxy instance and am wondering if anyone has seen anything similar. I have a custom tool that I've written that generates a tab-delimted file. When I click on the item name in the history, I can see the contents in the preview window. However, when I click on the eye (Display Data in Browser), it comes up empty. Same thing if I download the file. However, if I go to the database directory to examine the file on disk, the content is exactly what I would expect. The permissions on the file are galaxy user/group and the file has permissions of 644. I don't see any reason why I shouldn't be able to view or download this file through galaxy. Any ideas? Thanks, Dave --- Dave Walton Manager Scientific Computing The Jackson Laboratory Bar Harbor, Maine dave.wal...@jax.org The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] postgres connection
Galaxy dev:
I'm installing a local galaxy instance (Dec 4, 2012 vintage) and it is running
well under mySQL, but I'm having trouble getting it to work with a postgres
database.
I've done the following to create the db and provide permissions:
$ whoami
galaxy
$ createdb galaxy_prod4
$ psql galaxy_prod4
psql (8.4.13)
Type help for help.
galaxy_prod4=# CREATE ROLE galaxy_prod_role4 WITH PASSWORD 'milkweed';
CREATE ROLE
galaxy_prod4=# GRANT ALL PRIVILEGES ON DATABASE galaxy_prod4 to
galaxy_prod_role4;
GRANT
galaxy_prod4=# ALTER ROLE galaxy_prod_role4 LOGIN;
ALTER ROLE
galaxy_prod4=# \du
List of roles
Role name | Attributes | Member of
---+--+---
galaxy| Superuser| {}
: Create role
: Create DB
galaxy_prod_role | Cannot login | {}
galaxy_prod_role4 | | {}
mydb_role | Cannot login | {}
mydb_user | | {}
postgres | Superuser| {}
: Create role
: Create DB
galaxy_prod4=# \l
List of databases
Name | Owner | Encoding | Collation |Ctype| Access
privileges
--+--+--+-+-+--
galaxy_prod3 | galaxy | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/galaxy
:
galaxy=CTc/galaxy
:
galaxy_prod_role=CTc/galaxy
galaxy_prod4 | galaxy | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/galaxy
:
galaxy=CTc/galaxy
:
galaxy_prod_role4=CTc/galaxy
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0| postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres
:
postgres=CTc/postgres
template1| postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres
:
postgres=CTc/postgres
(5 rows)
Looks promising, right?
My universe_wsgi.ini has the following entry:
database_connection =
postgres:///galaxy_prod_role4:milkweed@127.0.0.1:5432/galaxy_prod4
and started the application, but it failed with:
OperationalError: (OperationalError) FATAL: database
galaxy_prod_role4:milkweed@127.0.0.1mailto:milkweed@127.0.0.1:5432/galaxy_prod4
does not exist
commenting the database_connection line out allows galaxy to start, using mySQL
I presume. Any ideas? This is my first time configuring something like this.
Thank you!
Chris
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
