Hi Dave, Thanks for the clarification. Our testing shows that the local data manager works in the vanilla Galaxy (simply hg clone, then run.sh). It failed on our clustered instance that it has been configured to use multiple web threads, one job manager and two job handlers.
Local data manager is a great idea, it would be even better if it can fetch the indexes from the local file system. Cheers, Derrick On Tue, Sep 11, 2012 at 10:31 PM, Dave Bouvier <d...@bx.psu.edu> wrote: > Derrick, > > I have not actually tested the local data manager on CloudBioLinux, but > that sounds like something worth looking into. > > As for the rsync server mentioned in another email, the data manager > checks our rsync server first, then tries generating indexes if the rsync > server doesn't have them. However, it will always produce indexes, as I > decided to allow for the option to regenerate indexes, if the local ones > had somehow become corrupted. > > --Dave B. > > > On 9/11/12 02:03:45.000, Derrick Lin wrote: > >> Dave, >> >> I upgraded the galaxy to Sep 7 release. The KeyError error has gone as >> expected. >> >> But the deferred_job_queue error persists. We are running galaxy in the >> cluster (SGE) here. Is it possible that Manage local data need to be >> configured in the cluster env? >> >> Regards, >> Derrick >> >> On Mon, Sep 10, 2012 at 10:59 PM, Dave Bouvier <d...@bx.psu.edu >> <mailto:d...@bx.psu.edu>> wrote: >> >> Derrick, >> >> The deferred_job_queue instance in the job manager object is only >> instantiated when the enable_beta_job_managers variable is set to >> True. Is it possible that your previous setting of that variable was >> somehow reverted? >> >> The KeyError error you mentioned in your previous should be fixed in >> the latest -dist and -central revisions. >> >> --Dave B. >> >> >> On 9/9/12 21:20:44.000, Derrick Lin wrote: >> >> Hi guys, >> >> I found that the problematic loc files were created by >> CloudBioLinux and >> point to the reference indexes downloaded by CloudBioLinux >> script. I >> suspect that Manage Local Data cannot recognize the file structure >> created by CloudBioLinux? >> >> Anyhow, I removed all the loc files from CloudBioLinux, then >> Manage >> Local Data can run now. Soon after I tried to download the >> bowtie hg18 >> from ucsc, I was given another error caused by "deferred job >> queue": >> >> URL: >> http://129.94.136.11:8080/__**galaxy_dev/data_admin/__** >> download_build<http://129.94.136.11:8080/__galaxy_dev/data_admin/__download_build> >> <http://129.94.136.11:8080/**galaxy_dev/data_admin/** >> download_build<http://129.94.136.11:8080/galaxy_dev/data_admin/download_build> >> > >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> WebError-0.8a-py2.6.egg/__**weberror/evalexception/__**middleware.py', >> >> line 364 in respond >> app_iter = self.application(environ, detect_start_response) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**debug/prints.py', >> >> line 98 in __call__ >> environ, self.app) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**wsgilib.py', >> >> line 539 in intercept_output >> app_iter = application(environ, replacement_start_response) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**recursive.py', >> >> line 80 in __call__ >> return self.application(environ, start_response) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**httpexceptions.py', >> >> line 632 in __call__ >> return self.application(environ, start_response) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/lib/__** >> galaxy/web/framework/base.py', >> >> line 160 in __call__ >> body = method( trans, **kwargs ) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/lib/__** >> galaxy/web/framework/__init__.**__py', >> >> line 184 in decorator >> return func( self, trans, *args, **kwargs ) >> File >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/lib/__** >> galaxy/web/controllers/data___**admin.py', >> >> line 199 in download_build >> jobid = >> trans.app.job_manager.__**deferred_job_queue.plugins['__** >> GenomeTransferPlugin'].create_**__job( >> >> trans, url, dbkey, longname, indexers ) >> AttributeError: 'JobManager' object has no attribute >> 'deferred_job_queue' >> >> Anyone can give some hints about what was going on? >> >> Regards, >> Derrick >> >> On Fri, Sep 7, 2012 at 4:26 PM, Derrick Lin <klin...@gmail.com >> <mailto:klin...@gmail.com> >> <mailto:klin...@gmail.com <mailto:klin...@gmail.com>>> wrote: >> >> Hi guys, >> >> I enabled the tool with enable_beta_job_managers = True, >> server runs >> fine. But after I clicked on Manage local data, it gave the >> following error: >> >> URL: >> http://pwbc.garvan.unsw.edu.__**au/galaxy_dev/data_admin/__** >> manage_data >> <http://pwbc.garvan.unsw.edu.**au/galaxy_dev/data_admin/** >> manage_data<http://pwbc.garvan.unsw.edu.au/galaxy_dev/data_admin/manage_data> >> > >> [^ >> >> <http://pwbc.garvan.unsw.edu._**_au/galaxy_dev/data_admin/__** >> manage_data >> <http://pwbc.garvan.unsw.edu.**au/galaxy_dev/data_admin/** >> manage_data<http://pwbc.garvan.unsw.edu.au/galaxy_dev/data_admin/manage_data> >> >>] >> >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> WebError-0.8a-py2.6.egg/__**weberror/evalexception/__**middleware.py', >> >> line 364 in respond >> app_iter = self.application(environ, >> detect_start_response) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**debug/prints.py', >> >> line 98 in __call__ >> environ, self.app) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**wsgilib.py', >> >> line 539 in intercept_output >> app_iter = application(environ, >> replacement_start_response) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**recursive.py', >> >> line 80 in __call__ >> return self.application(environ, start_response) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/eggs/__** >> Paste-1.6-py2.6.egg/paste/__**httpexceptions.py', >> >> line 632 in __call__ >> return self.application(environ, start_response) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/lib/__** >> galaxy/web/framework/base.py', >> >> line 160 in __call__ >> body = method( trans, **kwargs ) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/lib/__** >> galaxy/web/framework/__init__.**__py', >> >> line 184 in decorator >> return func( self, trans, *args, **kwargs ) >> File >> >> '/misc/PWBCAdminStore/app/__**galaxy/galaxy_dist/dev/lib/__** >> galaxy/web/controllers/data___**admin.py', >> >> line 47 in manage_data >> genomes[ dbkey ][ table ] = dict( state='Generated', >> style=self.jobstyles[ 'done' ] ) >> KeyError: 'phix' >> >> I tried to fix it by commenting out all the phix entries in >> all .loc >> files. Then it gave the same error but this time caused by >> "hg18". I >> think it must have something to do with builts.txt, >> manual_builts.txt and the loc files? >> >> Thanks >> Derrick >> >> >> >> >> ______________________________**______________________________**_ >> >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> >> ______________________________**______________________________**_ >> >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> >> >>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
