Hey, Really intensive database operations including datasest collections but other things too like multi-running tools or workflows over many individual datasets for instance - can very easily overwhelm the default sqlite database. This is frustrating and shouldn't happen - but it does unfortunately. I would recommend using a postgres database when testing out dataset collections. The good news is that it is easier than ever to get a fully fledged production-quality server thanks to Bjoern's docker server (https://github.com/bgruening/docker-galaxy-stable) - it comes bundled with Postgres and Slurm so it should be able to handle the collection operations. If you need to run Galaxy on a non-containerized server (for instance because that is where the software is) more information on setting up Galaxy can be found here https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer.
Here is a Trello card to track progress on the database optimization efforts if you are interested https://trello.com/c/UPLsMKQI. Very sorry. -John -John On Mon, Jan 26, 2015 at 9:35 AM, Torsten Houwaart <houwa...@informatik.uni-freiburg.de> wrote: > Hello Galaxy Devs, > > I was using data collections (for the first time) for a new workflow of ours > and I ran into this problem. There was no complaint by the workflow-editor > and I could start the workflow but then <see below> happened. > If you need more information about the workflow or otherwise let me know. > > Best, > Torsten H. > > > job traceback: > Traceback (most recent call last): > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", > line 565, in finish_job > job_state.job_wrapper.finish( stdout, stderr, exit_code ) > File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line > 1250, in finish > self.sa_session.flush() > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/scoping.py", > line 114, in do > return getattr(self.registry(), name)(*args, **kwargs) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py", > line 1718, in flush > self._flush(objects) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py", > line 1789, in _flush > flush_context.execute() > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/unitofwork.py", > line 331, in execute > rec.execute(self) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/unitofwork.py", > line 475, in execute > uow > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/persistence.py", > line 59, in save_obj > mapper, table, update) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/persistence.py", > line 485, in _emit_update_statements > execute(statement, params) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", > line 1449, in execute > params) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", > line 1584, in _execute_clauseelement > compiled_sql, distilled_params > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", > line 1698, in _execute_context > context) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", > line 1691, in _execute_context > context) > File > "/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/default.py", > line 331, in do_execute > cursor.execute(statement, parameters) > DBAPIError: (TransactionRollbackError) deadlock detected > DETAIL: Process 3144 waits for ShareLock on transaction 2517124; blocked by > process 3143. > Process 3143 waits for ShareLock on transaction 2517123; blocked by process > 3144. > HINT: See server log for query details. > 'UPDATE workflow_invocation SET update_time=%(update_time)s WHERE > workflow_invocation.id = %(workflow_invocation_id)s' {'update_time': > datetime.datetime(2015, 1, 26, 14, 20, 4, 155440), 'workflow_invocation_id': > 5454} > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
