Re,
This is the log of pulsar.
As pointed by Keith Superman, the pulsar don't restart since the step is
skipped.
How can I force pulsar to restart ?
Thanks,
Luc
------------
Luc Cornet, PhD
Bio-informatician
Mycology and Aerobiology
Sciensano
----- Mail original -----
De: "Marius van den Beek" <m.vandenb...@gmail.com>
À: "Luc Cornet" <luc.cor...@uliege.be>
Cc: "HelpGalaxy" <galaxy-dev@lists.galaxyproject.org>, "Baurain Denis"
<denis.baur...@uliege.be>, "Pierre Becker" <pierre.bec...@sciensano.be>,
"Colignon David" <david.colig...@uliege.be>
Envoyé: Mercredi 30 Juin 2021 16:44:50
Objet: Re: [galaxy-dev] Re: Galaxy install problems
If you've set up pulsar to start as a systemd service you should get the
logs with journalctl.
I assume that's `journalctl -u pulsar.service`.
For a quickstart in journalctl check out
https://www.linode.com/docs/guides/how-to-use-journalctl/
On Wed, 30 Jun 2021 at 16:38, Luc Cornet <luc.cor...@uliege.be> wrote:
> OK,
>
> Sure but can you please tell me how to get these pulsar application logs ?
>
> Thanks
> Luc
>
> ------------
> Luc Cornet, PhD
> Bio-informatician
> Mycology and Aerobiology
> Sciensano
>
> ----- Mail original -----
> De: "Marius van den Beek" <m.vandenb...@gmail.com>
> À: "Luc Cornet" <luc.cor...@uliege.be>
> Cc: "HelpGalaxy" <galaxy-dev@lists.galaxyproject.org>, "Baurain Denis" <
> denis.baur...@uliege.be>, "Pierre Becker" <pierre.bec...@sciensano.be>,
> "Colignon David" <david.colig...@uliege.be>
> Envoyé: Mercredi 30 Juin 2021 16:32:16
> Objet: [galaxy-dev] Re: Galaxy install problems
>
> Thanks, but these are not the pulsar application logs. Can you provide
> these please ?
>
> On Wed, 30 Jun 2021 at 16:30, Luc Cornet < [ mailto:luc.cor...@uliege.be
> | luc.cor...@uliege.be ] > wrote:
>
>
> Dear Marius,
>
> Many thank for your feedback.
>
> I join to this email: the playbook, the pulsarservers.yml file and the log
> of pulsar playbook.
>
> CLI plugin is for us the best solution since we have nothing to maintain.
> DRMAA is not actively developed for slurm, correct ?
>
>
> In the playbook, we use systemd which I think should restart pulsar but It
> might not be the case:
>
> TASK [galaxyproject.pulsar : systemd daemon-reload and enable/start
> service]
> ****************************************************************************
>
> ok: [HPC]
>
> RUNNING HANDLER [galaxyproject.pulsar : default restart pulsar handler]
> *********************************************************************************
>
> skipping: [HPC]
>
> Currently, we never used DRMAA. The job were executed immediately on the
> cluster with CLI or DRMAA. We had this part in pulsarservers.yml, to
> activate CLI:
> managers:
> _default_:
> type: queued_cli
> job_plugin: slurm
> native_specification: "-p batch --tasks=1 --cpus-per-task=2
> --mem-per-cpu=1000 -t 10:00"
> min_polling_interval: 0.5
> amqp_publish_retry: True
> amqp_publish_retry_max_retries: 5
> amqp_publish_retry_interval_start: 10
> amqp_publish_retry_interval_step: 10
> amqp_publish_retry_interval_max: 60
>
>
> Thanks for your help,
> Luc
>
>
> ------------
> Luc Cornet, PhD
> Bio-informatician
> Mycology and Aerobiology
> Sciensano
>
> ----- Mail original -----
> De: "Marius van den Beek" < [ mailto:m.vandenb...@gmail.com |
> m.vandenb...@gmail.com ] >
> À: "Luc Cornet" < [ mailto:luc.cor...@uliege.be | luc.cor...@uliege.be ]
> >
> Cc: "HelpGalaxy" < [ mailto:galaxy-dev@lists.galaxyproject.org |
> galaxy-dev@lists.galaxyproject.org ] >, "Baurain Denis" < [ mailto:
> denis.baur...@uliege.be | denis.baur...@uliege.be ] >, "Pierre Becker" <
> [ mailto:pierre.bec...@sciensano.be | pierre.bec...@sciensano.be ] >,
> "Colignon David" < [ mailto:david.colig...@uliege.be |
> david.colig...@uliege.be ] >
> Envoyé: Mercredi 30 Juin 2021 16:02:04
> Objet: [galaxy-dev] Re: Galaxy install problems
>
> Hi Luc,
>
> I'm sorry to hear that you're struggling to set up Galaxy to your liking.
> Let me start by pointing out that [ [ http://usegalaxy.org/ |
> http://usegalaxy.org/ ] | [ http://usegalaxy.org/ | usegalaxy.org ] ]
> uses slurm with DRMAA, this is certainly going to be more performant and
> reliable than the CLI plugin.
> There is little maintenance necessary, so maybe that is why activity on
> slurm-drmaa is low (See also [ [ https://github.com/natefoo/slurm-drmaa |
> https://github.com/natefoo/slurm-drmaa ] | [
> https://github.com/natefoo/slurm-drmaa |
> https://github.com/natefoo/slurm-drmaa ] ] ).
> I would be curious to know how you came to the conclusion that there is
> some incompatibility between DRMAA and slurm
> Note that one of the setups we teach during the training submits via DRMAA
> to slurm.
>
> Then I'd like to point out that there are a huge variety of different ways
> in which you can configure Galaxy and the job submission.
> We teach the most common ones during the training week, with the aim that
> you understand how these things work together,
> as well as giving you a handle on how you can manage these different
> settings and services using a configuration management system.
> We cannot tailor a solution to your infrastructure during this week.
>
> About your problem specifically, I had asked this on gitter before:
>
> > Did you restart pulsar after rolling out the new config ?
>
> to which you've answered that you re-ran the playbook, but that's not a
> sufficient answer.
>
> Every playbook is different, and we cannot know if this includes a
> restarter service for pulsar.
> Also please don't assume that everyone that could potentially help you
> knows ansible and the playbooks that are being taught intimately,
> and in what ways you have customized your playbook.
> It is much more helpful to write up the relevant settings you've changed
> and the logs that go with it.
>
> You've also been asked to provide logs of the restart, which as far as I
> can tell you haven't provided.
> You had mentioned on gitter that pulsar continues to use DRMAA to submit
> jobs, so you'll
> want to double check whether you've really restarted pulsar after the
> config changes,
> and look at the startup logs for pulsar, and find out how it is possible
> for pulsar to submit jobs
> via drmaa if it is not set up to do so.
>
> Best,
> Marius
>
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
> [ http://galaxyproject.org/search/ | http://galaxyproject.org/search/ ]
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/
>
$ systemctl status pulsar
â pulsar.service - Pulsar
Loaded: loaded (/etc/systemd/system/pulsar.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2021-06-07 15:49:57 CEST; 3 weeks 2 days ago
Main PID: 2214545 (pulsar)
Tasks: 10 (limit: 1649650)
Memory: 142.2M (limit: 6.0G)
CPU: 51min 58.415s
CGroup: /system.slice/pulsar.service
ââ2214545 /bin/bash /opt/pulsar/venv/bin/pulsar -c /opt/pulsar/config -m webless
ââ2214551 /opt/pulsar/venv/bin/python3 /opt/pulsar/venv/bin/pulsar-main
Jun 18 17:05:12 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:12,100 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:94671c20-d046-11eb-87d4-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 18 17:05:12 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:12,101 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:94671c20-d046-11eb-87d4-b42e99da3a3b] Published to key pulsar__status_update
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,602 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=9]]
collecting output None with action FileAction[path=/data/000/dataset_9.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/0a248a1f62a0>
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,768 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=postprocess]-[job=9]]
Publishing Pulsar state change with status complete for job_id 9
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,769 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=9]]
[publish:9565b74e-d046-11eb-9811-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,769 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=9]]
[publish:9565b74e-d046-11eb-9811-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,769 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=9]]
[publish:9565b74e-d046-11eb-9811-b42e99da3a3b] Published to key pulsar__status_update
Jun 20 21:22:58 nic5-login1 pulsar[2214545]: 2021-06-20 21:22:58,153 WARNI
[pulsar.client.amqp_exchange][consume-setup-pyamqp://galaxy_au:********@galaxy.inbios.uliege.be:5671//pulsar/galaxy_au?ssl=1] Got TimeoutError, will retry:
[Errno 110] Connection timed out
Jun 20 21:22:58 nic5-login1 pulsar[2214545]: 2021-06-20 21:22:58,680 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__setup] AMQP heartbeat thread
exiting
Jun 20 21:22:59 nic5-login1 pulsar[2214545]: 2021-06-20 21:22:59,714 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__setup] AMQP heartbeat thread
alive
lines 1-21/21 (END)
$ journalctl -qfeu pulsar.service
Jun 07 15:49:57 nic5-login1 systemd[1]: Started Pulsar.
Jun 07 15:49:57 nic5-login1 pulsar[2214545]: Sourcing file /opt/pulsar/config/local_env.sh
Jun 07 15:49:57 nic5-login1 pulsar[2214545]: Activating virtual environment /opt/pulsar/venv/bin/activate
Jun 07 15:49:57 nic5-login1 pulsar[2214545]: Starting pulsar with command [pulsar-main]
Jun 07 15:49:58 nic5-login1 pulsar[2214545]: /opt/pulsar/venv/lib/python3.6/site-packages/pulsar/util/pastescript/loadwsgi.py:125:
PkgResourcesDeprecationWarning: Parameters to load are deprecated. Call .resolve and .require separately.
Jun 07 15:49:58 nic5-login1 pulsar[2214545]: return pkg_resources.EntryPoint.parse("x=" + s).load(False)
Jun 07 15:49:58 nic5-login1 pulsar[2214545]: 2021-06-07 15:49:58,037 INFO [pulsar.core][MainThread] Starting the Pulsar without a toolbox to white-list.Ensure
this application is protected by firewall or a configured private token.
Jun 07 15:49:58 nic5-login1 pulsar[2214545]: 2021-06-07 15:49:58,040 INFO [galaxy.tool_util.deps.conda_util][MainThread] Installing conda, this may take
several minutes.
Jun 07 15:49:59 nic5-login1 pulsar[2214545]: PREFIX=/opt/pulsar/deps/_conda
Jun 07 15:49:59 nic5-login1 pulsar[2214545]: installing: python-3.7.3-h0371630_0 ...
Jun 07 15:50:02 nic5-login1 pulsar[2214545]: Python 3.7.3
Jun 07 15:50:02 nic5-login1 pulsar[2214545]: installing: ca-certificates-2019.1.23-0 ...
Jun 07 15:50:02 nic5-login1 pulsar[2214545]: installing: libgcc-ng-8.2.0-hdf63c60_1 ...
Jun 07 15:50:03 nic5-login1 pulsar[2214545]: installing: libstdcxx-ng-8.2.0-hdf63c60_1 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: libffi-3.2.1-hd88cf55_4 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: ncurses-6.1-he6710b0_1 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: openssl-1.1.1b-h7b6447c_1 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: xz-5.2.4-h14c3975_4 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: yaml-0.1.7-had09818_2 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: zlib-1.2.11-h7b6447c_3 ...
Jun 07 15:50:04 nic5-login1 pulsar[2214545]: installing: libedit-3.1.20181209-hc058e9b_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: readline-7.0-h7b6447c_5 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: tk-8.6.8-hbc83047_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: sqlite-3.27.2-h7b6447c_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: asn1crypto-0.24.0-py37_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: certifi-2019.3.9-py37_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: chardet-3.0.4-py37_1 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: idna-2.8-py37_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: pycosat-0.6.3-py37h14c3975_0 ...
Jun 07 15:50:05 nic5-login1 pulsar[2214545]: installing: pycparser-2.19-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: pysocks-1.6.8-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: ruamel_yaml-0.15.46-py37h14c3975_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: six-1.12.0-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: cffi-1.12.2-py37h2e261b9_1 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: setuptools-41.0.0-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: cryptography-2.6.1-py37h1ba5d50_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: wheel-0.33.1-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: pip-19.0.3-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: pyopenssl-19.0.0-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: urllib3-1.24.1-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: requests-2.21.0-py37_0 ...
Jun 07 15:50:06 nic5-login1 pulsar[2214545]: installing: conda-4.6.14-py37_0 ...
Jun 07 15:50:07 nic5-login1 pulsar[2214545]: installation finished.
Jun 07 15:50:07 nic5-login1 pulsar[2214545]: 2021-06-07 15:50:07,184 DEBUG [galaxy.tool_util.deps.conda_util][MainThread] Executing command:
/opt/pulsar/deps/_conda/bin/conda install -y --override-channels --channel conda-forge --channel bioconda --channel defaults conda=4.6.14
Jun 07 15:50:40 nic5-login1 pulsar[2214545]: Collecting package metadata: ...working... done
Jun 07 15:51:09 nic5-login1 pulsar[2214545]: Solving environment: ...working... done
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: ## Package Plan ##
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: environment location: /opt/pulsar/deps/_conda
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: added / updated specs:
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: - conda=4.6.14
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: The following packages will be downloaded:
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: package | build
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: ---------------------------|-----------------
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: _libgcc_mutex-0.1 | conda_forge 3 KB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: _openmp_mutex-4.5 | 1_gnu 22 KB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: ca-certificates-2021.5.30 | ha878542_0 136 KB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: certifi-2021.5.30 | py37h89c1867_0 141 KB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: conda-4.6.14 | py37_0 2.1 MB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: libgcc-ng-9.3.0 | h2828fa1_19 7.8 MB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: libgomp-9.3.0 | h2828fa1_19 376 KB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: openssl-1.1.1k | h7f98852_0 2.1 MB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: python_abi-3.7 | 1_cp37m 4 KB conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: ------------------------------------------------------------
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: Total: 12.7 MB
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: The following NEW packages will be INSTALLED:
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: _libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: libgomp conda-forge/linux-64::libgomp-9.3.0-h2828fa1_19
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: python_abi conda-forge/linux-64::python_abi-3.7-1_cp37m
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: The following packages will be UPDATED:
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: ca-certificates pkgs/main::ca-certificates-2019.1.23-0 --> conda-forge::ca-certificates-2021.5.30-ha878542_0
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: certifi pkgs/main::certifi-2019.3.9-py37_0 --> conda-forge::certifi-2021.5.30-py37h89c1867_0
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: libgcc-ng pkgs/main::libgcc-ng-8.2.0-hdf63c60_1 --> conda-forge::libgcc-ng-9.3.0-h2828fa1_19
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: openssl pkgs/main::openssl-1.1.1b-h7b6447c_1 --> conda-forge::openssl-1.1.1k-h7f98852_0
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: The following packages will be SUPERSEDED by a higher-priority channel:
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: conda pkgs/main --> conda-forge
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: Downloading and Extracting Packages
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: [107B blob data]
Jun 07 15:51:10 nic5-login1 pulsar[2214545]: [107B blob data]
Jun 07 15:51:11 nic5-login1 pulsar[2214545]: [269B blob data]
Jun 07 15:51:12 nic5-login1 pulsar[2214545]: [215B blob data]
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: [161B blob data]
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: [107B blob data]
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: [107B blob data]
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: [107B blob data]
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: [107B blob data]
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: Preparing transaction: ...working... done
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: Verifying transaction: ...working... done
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: Executing transaction: ...working... done
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: ==> WARNING: A newer version of conda exists. <==
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: current version: 4.6.14
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: latest version: 4.10.1
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: Please update conda by running
Jun 07 15:51:13 nic5-login1 pulsar[2214545]: $ conda update -n base -c defaults conda
Jun 07 15:51:15 nic5-login1 pulsar[2214545]: 2021-06-07 15:51:15,322 INFO [pulsar.locks][MainThread] pylockfile module not found, skipping experimental
lockfile handling.
Jun 07 15:51:15 nic5-login1 pulsar[2214545]: 2021-06-07 15:51:15,323 INFO [pulsar.messaging.bind_amqp][MainThread] bind_manager_to_queue called for
[pyamqp://galaxy_au:********@galaxy.inbios.uliege.be:5671//pulsar/galaxy_au?ssl=1] and manager [_default_]
Jun 07 15:51:15 nic5-login1 pulsar[2214545]: 2021-06-07 15:51:15,323 DEBUG
[pulsar.client.amqp_exchange][consume-setup-pyamqp://galaxy_au:********@galaxy.inbios.uliege.be:5671//pulsar/galaxy_au?ssl=1] Consuming queue '<unbound Queue
pulsar__setup -> <unbound Exchange pulsar(direct)> -> pulsar__setup>'
Jun 07 15:51:15 nic5-login1 pulsar[2214545]: 2021-06-07 15:51:15,324 DEBUG
[pulsar.client.amqp_exchange][consume-kill-pyamqp://galaxy_au:********@galaxy.inbios.uliege.be:5671//pulsar/galaxy_au?ssl=1] Consuming queue '<unbound Queue
pulsar__kill -> <unbound Exchange pulsar(direct)> -> pulsar__kill>'
Jun 07 15:51:15 nic5-login1 pulsar[2214545]: 2021-06-07 15:51:15,324 DEBUG
[pulsar.client.amqp_exchange][consume-status-pyamqp://galaxy_au:********@galaxy.inbios.uliege.be:5671//pulsar/galaxy_au?ssl=1] Consuming queue '<unbound Queue
pulsar__status -> <unbound Exchange pulsar(direct)> -> pulsar__status>'
Jun 10 16:43:50 nic5-login1 pulsar[2214545]: 2021-06-10 16:43:50,178 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__setup] AMQP heartbeat thread
alive
Jun 10 16:44:09 nic5-login1 pulsar[2214545]: 2021-06-10 16:44:09,193 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__status] AMQP heartbeat thread
alive
Jun 10 16:44:09 nic5-login1 pulsar[2214545]: 2021-06-10 16:44:09,419 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__kill] AMQP heartbeat thread alive
Jun 10 16:56:11 nic5-login1 pulsar[2214545]: 2021-06-10 16:56:11,595 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=6]]
Staging input 'dataset_1.dat' via
FileAction[path=/data/000/dataset_1.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/a799d38679e985db/files?job_key=39d1186210d29421&path=/data/000/dataset_1.dat&file_type=input]
to /opt/pulsar/files/staging/6/inputs/dataset_1.dat
Jun 10 16:56:11 nic5-login1 pulsar[2214545]: 2021-06-10 16:56:11,661 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=6]]
Staging input 'dataset_2.dat' via
FileAction[path=/data/000/dataset_2.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/a799d38679e985db/files?job_key=39d1186210d29421&path=/data/000/dataset_2.dat&file_type=input]
to /opt/pulsar/files/staging/6/inputs/dataset_2.dat
Jun 10 16:56:11 nic5-login1 pulsar[2214545]: 2021-06-10 16:56:11,726 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=6]] job_id: 6 -
Checking authorization of command_line [set -o | grep -q pipefail && set -o pipefail; bwa mem -t "${GALAXY_SLOTS:-1}" -v 1
'/cvmfs/data.galaxyproject.org/managed/bwa_mem_index/eschColi_K12/eschColi_K12.fa' '/opt/pulsar/files/staging/6/inputs/dataset_1.dat'
'/opt/pulsar/files/staging/6/inputs/dataset_2.dat' | samtools sort -@${GALAXY_SLOTS:-2} -T "${TMPDIR:-.}" -O bam -o
'/opt/pulsar/files/staging/6/outputs/dataset_6.dat'; return_code=$?; sh -c "exit $return_code"]
Jun 10 16:56:11 nic5-login1 pulsar[2214545]: 2021-06-10 16:56:11,731 DEBUG [galaxy.tool_util.deps.conda_util][[manager=_default_]-[action=preprocess]-[job=6]]
Executing command: /opt/pulsar/deps/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults --name
mulled-v1-a1698655ddc683fd2767d127c7c583056e87322876f94e167ba3900da02c1fc4 bwa=0.7.17 samtools=1.6
Jun 10 16:56:48 nic5-login1 pulsar[2214545]: Collecting package metadata: ...working... done
Jun 10 16:56:49 nic5-login1 pulsar[2214545]: Solving environment: ...working... done
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: ## Package Plan ##
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: environment location:
/opt/pulsar/deps/_conda/envs/mulled-v1-a1698655ddc683fd2767d127c7c583056e87322876f94e167ba3900da02c1fc4
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: added / updated specs:
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: - bwa=0.7.17
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: - samtools=1.6
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: The following packages will be downloaded:
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: package | build
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: ---------------------------|-----------------
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: bwa-0.7.17 | h5bf99c6_8 608 KB bioconda
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: bzip2-1.0.8 | h7f98852_4 484 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: c-ares-1.17.1 | h7f98852_1 109 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: curl-7.77.0 | hea6ffbf_0 149 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: krb5-1.19.1 | hcc1bbae_0 1.4 MB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libcurl-7.77.0 | h2574ce0_0 334 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libedit-3.1.20191231 | he28a2e2_2 121 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libev-4.33 | h516909a_1 104 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libnghttp2-1.43.0 | h812cca2_0 808 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libssh2-1.9.0 | ha56f1ee_6 226 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libstdcxx-ng-9.3.0 | h6de172a_19 4.0 MB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: ncurses-6.2 | h58526e2_4 985 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: perl-5.32.1 | 0_h7f98852_perl5 14.5 MB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: samtools-1.6 | hb116620_7 514 KB bioconda
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: tk-8.6.10 | h21135ba_1 3.2 MB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: xz-5.2.5 | h516909a_1 343 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: zlib-1.2.11 | h516909a_1010 106 KB conda-forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: ------------------------------------------------------------
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: Total: 28.0 MB
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: The following NEW packages will be INSTALLED:
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: _libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-1_gnu
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: bwa bioconda/linux-64::bwa-0.7.17-h5bf99c6_8
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: bzip2 conda-forge/linux-64::bzip2-1.0.8-h7f98852_4
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: c-ares conda-forge/linux-64::c-ares-1.17.1-h7f98852_1
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: ca-certificates conda-forge/linux-64::ca-certificates-2021.5.30-ha878542_0
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: curl conda-forge/linux-64::curl-7.77.0-hea6ffbf_0
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: krb5 conda-forge/linux-64::krb5-1.19.1-hcc1bbae_0
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libcurl conda-forge/linux-64::libcurl-7.77.0-h2574ce0_0
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libedit conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libev conda-forge/linux-64::libev-4.33-h516909a_1
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libgcc-ng conda-forge/linux-64::libgcc-ng-9.3.0-h2828fa1_19
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libgomp conda-forge/linux-64::libgomp-9.3.0-h2828fa1_19
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libnghttp2 conda-forge/linux-64::libnghttp2-1.43.0-h812cca2_0
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libssh2 conda-forge/linux-64::libssh2-1.9.0-ha56f1ee_6
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-9.3.0-h6de172a_19
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: ncurses conda-forge/linux-64::ncurses-6.2-h58526e2_4
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: openssl conda-forge/linux-64::openssl-1.1.1k-h7f98852_0
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: perl conda-forge/linux-64::perl-5.32.1-0_h7f98852_perl5
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: samtools bioconda/linux-64::samtools-1.6-hb116620_7
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: tk conda-forge/linux-64::tk-8.6.10-h21135ba_1
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: xz conda-forge/linux-64::xz-5.2.5-h516909a_1
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: zlib conda-forge/linux-64::zlib-1.2.11-h516909a_1010
Jun 10 16:56:58 nic5-login1 pulsar[2214545]: Preparing transaction: ...working... done
Jun 10 16:56:59 nic5-login1 pulsar[2214545]: Verifying transaction: ...working... done
Jun 10 16:56:59 nic5-login1 pulsar[2214545]: Executing transaction: ...working... done
Jun 10 16:57:01 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:01,416 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=6]] Using
dependency bwa version 0.7.17 of type conda
Jun 10 16:57:01 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:01,417 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=6]] Using
dependency samtools version 1.6 of type conda
Jun 10 16:57:01 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:01,793 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=monitor]] Publishing Pulsar
state change with status running for job_id 6
Jun 10 16:57:01 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:01,794 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:1cda4f6a-c9fc-11eb-87d4-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 10 16:57:01 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:01,798 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:1cda4f6a-c9fc-11eb-87d4-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 10 16:57:01 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:01,817 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:1cda4f6a-c9fc-11eb-87d4-b42e99da3a3b] Published to key pulsar__status_update
Jun 10 16:57:18 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:18,814 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=6]]
collecting output None with action
FileAction[path=/data/000/dataset_6.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/a799d38679e985db/files?job_key=39d1186210d29421&path=/data/000/dataset_6.dat&file_type=output]
Jun 10 16:57:18 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:18,982 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=postprocess]-[job=6]]
Publishing Pulsar state change with status complete for job_id 6
Jun 10 16:57:18 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:18,983 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=6]]
[publish:271922f8-c9fc-11eb-8ca0-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 10 16:57:18 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:18,983 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=6]]
[publish:271922f8-c9fc-11eb-8ca0-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 10 16:57:18 nic5-login1 pulsar[2214545]: 2021-06-10 16:57:18,984 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=6]]
[publish:271922f8-c9fc-11eb-8ca0-b42e99da3a3b] Published to key pulsar__status_update
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,354 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=7]]
Staging input 'dataset_1.dat' via
FileAction[path=/data/000/dataset_1.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/5969b1f7201f12ae/files?job_key=ea6643a3afeae9ed&path=/data/000/dataset_1.dat&file_type=input]
to /opt/pulsar/files/staging/7/inputs/dataset_1.dat
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,416 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=7]]
Staging input 'dataset_2.dat' via
FileAction[path=/data/000/dataset_2.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/5969b1f7201f12ae/files?job_key=ea6643a3afeae9ed&path=/data/000/dataset_2.dat&file_type=input]
to /opt/pulsar/files/staging/7/inputs/dataset_2.dat
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,472 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=7]] job_id: 7 -
Checking authorization of command_line [set -o | grep -q pipefail && set -o pipefail; bwa mem -t "${GALAXY_SLOTS:-1}" -v 1
'/cvmfs/data.galaxyproject.org/managed/bwa_mem_index/eschColi_K12/eschColi_K12.fa' '/opt/pulsar/files/staging/7/inputs/dataset_1.dat'
'/opt/pulsar/files/staging/7/inputs/dataset_2.dat' | samtools sort -@${GALAXY_SLOTS:-2} -T "${TMPDIR:-.}" -O bam -o
'/opt/pulsar/files/staging/7/outputs/dataset_7.dat'; return_code=$?; sh -c "exit $return_code"]
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,473 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=7]] Using
dependency bwa version 0.7.17 of type conda
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,473 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=7]] Using
dependency samtools version 1.6 of type conda
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,974 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=monitor]] Publishing Pulsar
state change with status running for job_id 7
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,974 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:32fe09bc-cf51-11eb-87d4-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,974 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:32fe09bc-cf51-11eb-87d4-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,974 ERROR [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:32fe09bc-cf51-11eb-87d4-b42e99da3a3b] Connection error while publishing: TimeoutError(110, 'Connection timed out')
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: Traceback (most recent call last):
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/kombu/connection.py", line 525, in _ensured
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: return fun(*args, **kwargs)
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/kombu/messaging.py", line 200, in _publish
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: mandatory=mandatory, immediate=immediate,
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/amqp/channel.py", line 1775, in _basic_publish
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: self.connection.drain_events(timeout=0)
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/amqp/connection.py", line 522, in drain_events
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: while not self.blocking_read(timeout):
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/amqp/connection.py", line 527, in blocking_read
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: frame = self.transport.read_frame()
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/amqp/transport.py", line 268, in read_frame
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: frame_header = read(7, True)
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/opt/pulsar/venv/lib/python3.6/site-packages/amqp/transport.py", line 405, in _read
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: s = recv(n - len(rbuf)) # see note above
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/usr/lib64/python3.6/ssl.py", line 833, in read
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: return self._sslobj.read(len, buffer)
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: File "/usr/lib64/python3.6/ssl.py", line 592, in read
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: v = self._sslobj.read(len)
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: TimeoutError: [Errno 110] Connection timed out
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,979 INFO [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:32fe09bc-cf51-11eb-87d4-b42e99da3a3b] Retrying in 0 seconds
Jun 17 11:48:41 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:41,999 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:32fe09bc-cf51-11eb-87d4-b42e99da3a3b] Published to key pulsar__status_update
Jun 17 11:48:43 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:43,977 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=7]]
collecting output None with action
FileAction[path=/data/000/dataset_7.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/5969b1f7201f12ae/files?job_key=ea6643a3afeae9ed&path=/data/000/dataset_7.dat&file_type=output]
Jun 17 11:48:44 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:44,176 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=postprocess]-[job=7]]
Publishing Pulsar state change with status complete for job_id 7
Jun 17 11:48:44 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:44,176 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=7]]
[publish:344e16b8-cf51-11eb-a9a4-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 17 11:48:44 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:44,176 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=7]]
[publish:344e16b8-cf51-11eb-a9a4-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 17 11:48:44 nic5-login1 pulsar[2214545]: 2021-06-17 11:48:44,177 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=7]]
[publish:344e16b8-cf51-11eb-a9a4-b42e99da3a3b] Published to key pulsar__status_update
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,300 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=8]]
Staging input 'dataset_1.dat' via
FileAction[path=/data/000/dataset_1.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/df7a1f0c02a5b08e/files?job_key=767b0b0a60efe655&path=/data/000/dataset_1.dat&file_type=input]
to /opt/pulsar/files/staging/8/inputs/dataset_1.dat
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,360 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=8]]
Staging input 'dataset_2.dat' via
FileAction[path=/data/000/dataset_2.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/df7a1f0c02a5b08e/files?job_key=767b0b0a60efe655&path=/data/000/dataset_2.dat&file_type=input]
to /opt/pulsar/files/staging/8/inputs/dataset_2.dat
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,418 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=8]] job_id: 8 -
Checking authorization of command_line [set -o | grep -q pipefail && set -o pipefail; bwa mem -t "${GALAXY_SLOTS:-1}" -v 1
'/cvmfs/data.galaxyproject.org/managed/bwa_mem_index/eschColi_K12/eschColi_K12.fa' '/opt/pulsar/files/staging/8/inputs/dataset_1.dat'
'/opt/pulsar/files/staging/8/inputs/dataset_2.dat' | samtools sort -@${GALAXY_SLOTS:-2} -T "${TMPDIR:-.}" -O bam -o
'/opt/pulsar/files/staging/8/outputs/dataset_8.dat'; return_code=$?; sh -c "exit $return_code"]
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,418 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=8]] Using
dependency bwa version 0.7.17 of type conda
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,418 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=8]] Using
dependency samtools version 1.6 of type conda
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,843 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=monitor]] Publishing Pulsar
state change with status running for job_id 8
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,843 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:af86fa20-cf65-11eb-87d4-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,844 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:af86fa20-cf65-11eb-87d4-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 17 14:15:20 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:20,844 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:af86fa20-cf65-11eb-87d4-b42e99da3a3b] Published to key pulsar__status_update
Jun 17 14:15:22 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:22,345 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=8]]
collecting output None with action
FileAction[path=/data/000/dataset_8.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/df7a1f0c02a5b08e/files?job_key=767b0b0a60efe655&path=/data/000/dataset_8.dat&file_type=output]
Jun 17 14:15:22 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:22,445 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=postprocess]-[job=8]]
Publishing Pulsar state change with status complete for job_id 8
Jun 17 14:15:22 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:22,445 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=8]]
[publish:b07b72ee-cf65-11eb-b53e-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 17 14:15:22 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:22,446 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=8]]
[publish:b07b72ee-cf65-11eb-b53e-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 17 14:15:22 nic5-login1 pulsar[2214545]: 2021-06-17 14:15:22,446 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=8]]
[publish:b07b72ee-cf65-11eb-b53e-b42e99da3a3b] Published to key pulsar__status_update
Jun 18 17:05:11 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:11,565 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=9]]
Staging input 'dataset_1.dat' via
FileAction[path=/data/000/dataset_1.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/0a248a1f62a0cc04/files?job_key=99525bca5c821ddb&path=/data/000/dataset_1.dat&file_type=input]
to /opt/pulsar/files/staging/9/inputs/dataset_1.dat
Jun 18 17:05:11 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:11,623 DEBUG [pulsar.managers.staging.pre][[manager=_default_]-[action=preprocess]-[job=9]]
Staging input 'dataset_2.dat' via
FileAction[path=/data/000/dataset_2.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/0a248a1f62a0cc04/files?job_key=99525bca5c821ddb&path=/data/000/dataset_2.dat&file_type=input]
to /opt/pulsar/files/staging/9/inputs/dataset_2.dat
Jun 18 17:05:11 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:11,681 DEBUG [pulsar.managers.base][[manager=_default_]-[action=preprocess]-[job=9]] job_id: 9 -
Checking authorization of command_line [set -o | grep -q pipefail && set -o pipefail; bwa mem -t "${GALAXY_SLOTS:-1}" -v 1
'/cvmfs/data.galaxyproject.org/managed/bwa_mem_index/eschColi_K12/eschColi_K12.fa' '/opt/pulsar/files/staging/9/inputs/dataset_1.dat'
'/opt/pulsar/files/staging/9/inputs/dataset_2.dat' | samtools sort -@${GALAXY_SLOTS:-2} -T "${TMPDIR:-.}" -O bam -o
'/opt/pulsar/files/staging/9/outputs/dataset_9.dat'; return_code=$?; sh -c "exit $return_code"]
Jun 18 17:05:11 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:11,681 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=9]] Using
dependency bwa version 0.7.17 of type conda
Jun 18 17:05:11 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:11,681 DEBUG [galaxy.tool_util.deps][[manager=_default_]-[action=preprocess]-[job=9]] Using
dependency samtools version 1.6 of type conda
Jun 18 17:05:12 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:12,100 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=monitor]] Publishing Pulsar
state change with status running for job_id 9
Jun 18 17:05:12 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:12,100 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:94671c20-d046-11eb-87d4-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 18 17:05:12 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:12,100 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:94671c20-d046-11eb-87d4-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 18 17:05:12 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:12,101 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=monitor]]
[publish:94671c20-d046-11eb-87d4-b42e99da3a3b] Published to key pulsar__status_update
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,602 INFO [pulsar.client.staging.down][[manager=_default_]-[action=postprocess]-[job=9]]
collecting output None with action
FileAction[path=/data/000/dataset_9.dat,action_type=remote_transfer,url=https://galaxy.inbios.uliege.be/api/jobs/0a248a1f62a0cc04/files?job_key=99525bca5c821ddb&path=/data/000/dataset_9.dat&file_type=output]
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,768 DEBUG [pulsar.messaging.bind_amqp][[manager=_default_]-[action=postprocess]-[job=9]]
Publishing Pulsar state change with status complete for job_id 9
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,769 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=9]]
[publish:9565b74e-d046-11eb-9811-b42e99da3a3b] Begin publishing to key pulsar__status_update
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,769 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=9]]
[publish:9565b74e-d046-11eb-9811-b42e99da3a3b] Have producer for publishing to key pulsar__status_update
Jun 18 17:05:13 nic5-login1 pulsar[2214545]: 2021-06-18 17:05:13,769 DEBUG [pulsar.client.amqp_exchange][[manager=_default_]-[action=postprocess]-[job=9]]
[publish:9565b74e-d046-11eb-9811-b42e99da3a3b] Published to key pulsar__status_update
Jun 20 21:22:58 nic5-login1 pulsar[2214545]: 2021-06-20 21:22:58,153 WARNI
[pulsar.client.amqp_exchange][consume-setup-pyamqp://galaxy_au:********@galaxy.inbios.uliege.be:5671//pulsar/galaxy_au?ssl=1] Got TimeoutError, will retry:
[Errno 110] Connection timed out
Jun 20 21:22:58 nic5-login1 pulsar[2214545]: 2021-06-20 21:22:58,680 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__setup] AMQP heartbeat thread
exiting
Jun 20 21:22:59 nic5-login1 pulsar[2214545]: 2021-06-20 21:22:59,714 DEBUG [pulsar.client.amqp_exchange][consume-heartbeat-pulsar__setup] AMQP heartbeat thread
alive
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
%(web_page_url)s
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/
