Re: [galaxy-dev] Galaxy startup takes very long. Normal?
I'm using galxy-dist. That's why I don't have the setting. Thanks for the information, I'll wait for the next stable release and update the settings then. Geert On 03/26/2014 01:59 PM, Greg Von Kuster wrote: The changeset that made this a configurable setting was committed to the stable branch ind270d3c, which was committed on 2014-02-19 https://bitbucket.org/galaxy/galaxy-central/commits/d270d3cf7627a42b6fac2aa69701deadb89c8ffc If you are tracking the stable branch in the galaxy central repo (which it looks like you are doing), then you should be able to pull it from there. Greg Von Kuster On Mar 26, 2014, at 8:04 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be mailto:geert.vandewey...@uantwerpen.be wrote: Hi Greg, The setting was not in my universe_sample file. I added it, set log_info to DEBUG and restarted galaxy. The startup time remained the same, and there were many entries in the log file indicating that the lag is indeed the creation of the dependency system: tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:01,176 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:02,645 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:03,874 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:05,383 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:06,533 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:08,054 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:09,126 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:10,596 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:11,803 Adding an entry for version 1.02.00 of package lastz to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:13,332 Adding an entry for version 1.02.00 of package lastz to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:14,552 Adding an entry for version 3.0.1 of package R_3_0_1 to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:15,943 Adding an entry for version 3.0.1 of package R_3_0_1 to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:17,162 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:18,594 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. etc... I also note many double entries in the log files. Do I need a specific changeset for the parameter setting to take effect? Universe_wsgi.ini setting: # Enable use of an in-memory registry with bi-directional relationships between repositories manage_dependency_relationships = False Best, Geert On 03/26/2014 11:53 AM, Greg Von Kuster wrote: Hi Geert, The setting should be in your universe_wsgi.ini.sample, and you woud have to manually edit your universe_wsgi.ini to add it. I would say that 125 installed packages is probably what is causing the slow starts. This feature is not currently useful, so it can be set to not function with no problems. In the future I'll introduce additional benefits for this feature, and I'll look at ways to improve the startup speed. Greg Von Kuster On Mar 26, 2014, at 3:43 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be mailto:geert.vandewey...@uantwerpen.be wrote: hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summary is (hg incoming doesn't show any available updates): parent: 12276:dc067a95261d tip Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable commit: 22 modified, 1 deleted, 127 unknown update: (current) I currently have 125 packages from toolsheds. Is that considered to be many? Most of them are the migrated tools and the devteam mappers/gatk/tophat stuff. Geert On 03/25/2014 05:16 PM, Greg Von Kuster wrote: Hello Geert, Do you have a lot of repositories installed from the Tool Shed into you Galaxy instance? if so, the time you're experience may be due to loading the in-memory installed repository registry. Using this registry is optional, but the default configuration setting is to use it. You can set the following to entry to False in your universe_wsgi.ini file and restart your Galaxy server to see if that is the problem. This registry is not currently used by anythin except to display the set of dependent repositories that will be affected if a repository is uninstalled. If this is ot what is causing the slow statup, then I'm not sure where else to look. # Enable use of an in-memory registry with bi-directional relationships # between repositories (i.e., in addition to lists of dependencies for a # repository, keep an in-memory registry of dependent items for each repository. #manage_dependency_relationships = True Greg Von Kuster On Mar 25, 2014, at 11:08 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be mailto:geert.vandewey...@uantwerpen.be wrote: Dear all, I'm wondering if the following behaviour is normal. Since I reinstalled the latest galaxy distribution, every restart hangs/loads for up to 10 minutes at the following lines: galaxy.model.migrate.check INFO 2014-03-25 15:56:57,965 At database version 118 tool_shed.galaxy_install.migrate.check INFO 2014-03-25 15:56:58,477 At migrate_tools version 9 galaxy.config INFO 2014-03-25 15:56:58,694 Install database targetting Galaxy's database configuration. After several minutes of no heavy processing (cpu/database is almost idle), the startup continues. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be mailto:geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
Hi Geert, The setting should be in your universe_wsgi.ini.sample, and you woud have to manually edit your universe_wsgi.ini to add it. I would say that 125 installed packages is probably what is causing the slow starts. This feature is not currently useful, so it can be set to not function with no problems. In the future I'll introduce additional benefits for this feature, and I'll look at ways to improve the startup speed. Greg Von Kuster On Mar 26, 2014, at 3:43 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be wrote: hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summary is (hg incoming doesn't show any available updates): parent: 12276:dc067a95261d tip Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable commit: 22 modified, 1 deleted, 127 unknown update: (current) I currently have 125 packages from toolsheds. Is that considered to be many? Most of them are the migrated tools and the devteam mappers/gatk/tophat stuff. Geert On 03/25/2014 05:16 PM, Greg Von Kuster wrote: Hello Geert, Do you have a lot of repositories installed from the Tool Shed into you Galaxy instance? if so, the time you're experience may be due to loading the in-memory installed repository registry. Using this registry is optional, but the default configuration setting is to use it. You can set the following to entry to False in your universe_wsgi.ini file and restart your Galaxy server to see if that is the problem. This registry is not currently used by anythin except to display the set of dependent repositories that will be affected if a repository is uninstalled. If this is ot what is causing the slow statup, then I'm not sure where else to look. # Enable use of an in-memory registry with bi-directional relationships # between repositories (i.e., in addition to lists of dependencies for a # repository, keep an in-memory registry of dependent items for each repository. #manage_dependency_relationships = True Greg Von Kuster On Mar 25, 2014, at 11:08 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be wrote: Dear all, I'm wondering if the following behaviour is normal. Since I reinstalled the latest galaxy distribution, every restart hangs/loads for up to 10 minutes at the following lines: galaxy.model.migrate.check INFO 2014-03-25 15:56:57,965 At database version 118 tool_shed.galaxy_install.migrate.check INFO 2014-03-25 15:56:58,477 At migrate_tools version 9 galaxy.config INFO 2014-03-25 15:56:58,694 Install database targetting Galaxy's database configuration. After several minutes of no heavy processing (cpu/database is almost idle), the startup continues. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
Hi Greg, The setting was not in my universe_sample file. I added it, set log_info to DEBUG and restarted galaxy. The startup time remained the same, and there were many entries in the log file indicating that the lag is indeed the creation of the dependency system: tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:01,176 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:02,645 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:03,874 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:05,383 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:06,533 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:08,054 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:09,126 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:10,596 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:11,803 Adding an entry for version 1.02.00 of package lastz to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:13,332 Adding an entry for version 1.02.00 of package lastz to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:14,552 Adding an entry for version 3.0.1 of package R_3_0_1 to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:15,943 Adding an entry for version 3.0.1 of package R_3_0_1 to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:17,162 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:18,594 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. etc... I also note many double entries in the log files. Do I need a specific changeset for the parameter setting to take effect? Universe_wsgi.ini setting: # Enable use of an in-memory registry with bi-directional relationships between repositories manage_dependency_relationships = False Best, Geert On 03/26/2014 11:53 AM, Greg Von Kuster wrote: Hi Geert, The setting should be in your universe_wsgi.ini.sample, and you woud have to manually edit your universe_wsgi.ini to add it. I would say that 125 installed packages is probably what is causing the slow starts. This feature is not currently useful, so it can be set to not function with no problems. In the future I'll introduce additional benefits for this feature, and I'll look at ways to improve the startup speed. Greg Von Kuster On Mar 26, 2014, at 3:43 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be mailto:geert.vandewey...@uantwerpen.be wrote: hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summary is (hg incoming doesn't show any available updates): parent: 12276:dc067a95261d tip Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable commit: 22 modified, 1 deleted, 127 unknown update: (current) I currently have 125 packages from toolsheds. Is that considered to be many? Most of them are the migrated tools and the devteam mappers/gatk/tophat stuff. Geert On 03/25/2014 05:16 PM, Greg Von Kuster wrote: Hello Geert, Do you have a lot of repositories installed from the Tool Shed into you Galaxy instance? if so, the time you're experience may be due to loading the in-memory installed repository registry. Using this registry is optional, but the default configuration setting is to use it. You can
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
The changeset that made this a configurable setting was committed to the stable branch in d270d3c, which was committed on 2014-02-19 https://bitbucket.org/galaxy/galaxy-central/commits/d270d3cf7627a42b6fac2aa69701deadb89c8ffc If you are tracking the stable branch in the galaxy central repo (which it looks like you are doing), then you should be able to pull it from there. Greg Von Kuster On Mar 26, 2014, at 8:04 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be wrote: Hi Greg, The setting was not in my universe_sample file. I added it, set log_info to DEBUG and restarted galaxy. The startup time remained the same, and there were many entries in the log file indicating that the lag is indeed the creation of the dependency system: tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:01,176 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:02,645 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:03,874 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:05,383 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:06,533 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:08,054 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:09,126 Adding an entry for version 2.11.0 of package R to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:10,596 Adding an entry for version 2.11.0 of package R to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:11,803 Adding an entry for version 1.02.00 of package lastz to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:13,332 Adding an entry for version 1.02.00 of package lastz to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:14,552 Adding an entry for version 3.0.1 of package R_3_0_1 to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:15,943 Adding an entry for version 3.0.1 of package R_3_0_1 to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:17,162 Adding an entry for version 0.1.18 of package samtools to runtime_tool_dependencies_of_installed_tool_dependencies. tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-03-26 13:01:18,594 Adding an entry for version 0.1.18 of package samtools to installed_runtime_dependent_tool_dependencies_of_installed_tool_dependencies. etc... I also note many double entries in the log files. Do I need a specific changeset for the parameter setting to take effect? Universe_wsgi.ini setting: # Enable use of an in-memory registry with bi-directional relationships between repositories manage_dependency_relationships = False Best, Geert On 03/26/2014 11:53 AM, Greg Von Kuster wrote: Hi Geert, The setting should be in your universe_wsgi.ini.sample, and you woud have to manually edit your universe_wsgi.ini to add it. I would say that 125 installed packages is probably what is causing the slow starts. This feature is not currently useful, so it can be set to not function with no problems. In the future I'll introduce additional benefits for this feature, and I'll look at ways to improve the startup speed. Greg Von Kuster On Mar 26, 2014, at 3:43 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be wrote: hi Greg, I don't have that setting in my universe. Should I just add it? The output of hg summary is (hg incoming doesn't show any available updates): parent: 12276:dc067a95261d tip Added tag release_2014.02.10 for changeset 5e605ed6069f branch: stable commit: 22 modified, 1 deleted, 127 unknown
Re: [galaxy-dev] Galaxy startup takes very long. Normal?
Hello Geert, Do you have a lot of repositories installed from the Tool Shed into you Galaxy instance? if so, the time you're experience may be due to loading the in-memory installed repository registry. Using this registry is optional, but the default configuration setting is to use it. You can set the following to entry to False in your universe_wsgi.ini file and restart your Galaxy server to see if that is the problem. This registry is not currently used by anythin except to display the set of dependent repositories that will be affected if a repository is uninstalled. If this is ot what is causing the slow statup, then I'm not sure where else to look. # Enable use of an in-memory registry with bi-directional relationships # between repositories (i.e., in addition to lists of dependencies for a # repository, keep an in-memory registry of dependent items for each repository. #manage_dependency_relationships = True Greg Von Kuster On Mar 25, 2014, at 11:08 AM, Geert Vandeweyer geert.vandewey...@uantwerpen.be wrote: Dear all, I'm wondering if the following behaviour is normal. Since I reinstalled the latest galaxy distribution, every restart hangs/loads for up to 10 minutes at the following lines: galaxy.model.migrate.check INFO 2014-03-25 15:56:57,965 At database version 118 tool_shed.galaxy_install.migrate.check INFO 2014-03-25 15:56:58,477 At migrate_tools version 9 galaxy.config INFO 2014-03-25 15:56:58,694 Install database targetting Galaxy's database configuration. After several minutes of no heavy processing (cpu/database is almost idle), the startup continues. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandewe...@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
