Hey Brad,
So I've dug around and replicated all the requests in the web log you sent
(central#default tip -- what revision are you running?), but I'm not seeing
any unencoded id queries my sql logs (expected this more or less, w/ all of
the requests in your web log being 200 OK's). The next guess is that it
has to do with job metrics or metadata. The job itself failed, do you know
why that is? What job metrics do you have enabled? And has this been a
consistent issue, or you just happened to notice the error this one time?
-Dannon
On Tue, Aug 5, 2014 at 5:53 PM, Langhorst, Brad langho...@neb.com wrote:
I tried to do some time correlation between galaxy logs and postgres
logsā¦
postgres
2014-08-01 17:15:15.643 EDT
handler1.log
galaxy.jobs.output_checker INFO 2014-08-01 17:15:08,875 Job 123681:
Fatal error: Exit code 1 (Interval subtraction error)
galaxy.jobs DEBUG 2014-08-01 17:15:08,948 setting dataset state to ERROR
galaxy.jobs INFO 2014-08-01 17:15:09,196 Collecting job metrics for
galaxy.model.Job object at 0x7fcbacccbb50
galaxy.jobs DEBUG 2014-08-01 17:15:09,208 job 123681 ended
galaxy.datatypes.metadata DEBUG 2014-08-01 17:15:09,208 Cleaning up
external metadata files
galaxy.jobs.runners.drmaa DEBUG 2014-08-01 17:17:32,338 (123678/1521440)
state change: job finished normally
handler0.log
laxy.datatypes.metadata DEBUG 2014-08-01 17:14:40,572 loading metadata
from file for: HistoryDatasetAssociation 192134
galaxy.jobs INFO 2014-08-01 17:14:40,804 Collecting job metrics for
galaxy.model.Job object at 0x7f15fc281850
galaxy.jobs DEBUG 2014-08-01 17:14:40,815 job 123680 ended
galaxy.datatypes.metadata DEBUG 2014-08-01 17:14:40,815 Cleaning up
external metadata files
python path is:
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/kombu-3.0.12-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/anyjson-0.3.3-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/numpy-1.6.0-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/mercurial-2.2.3-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/pycrypto-2.5-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/python_lzo-1.08_2.03_static-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/bx_python-0.7.2-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/amqp-1.4.3-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/pexpect-2.4-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Babel-1.3-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/pytz-2013.9-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Mako-0.4.1-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/WebHelpers-1.3-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/MarkupSafe-0.12-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/WebOb-0.8.5-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Routes-1.12.3-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/wchartype-0.1-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/docutils-0.7-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/PasteDeploy-1.5.0-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg,
/mnt/galaxy/data/galaxy/galaxy-dist/lib,
/mnt/galaxy/data/galaxy/sw/lib/python2.7/site-packages/cutadapt-1.3-py2.7-linux-x86_64.egg,
/mnt/galaxy/data/galaxy/sw/lib/python2.7/site-packages/distribute-0.6.10-py2.7.egg,
/mnt/galaxy/data/galaxy/sw/lib/python27.zip,
/mnt/galaxy/data/galaxy/sw/lib/python2.7,
/mnt/galaxy/data/galaxy/sw/lib/python2.7/plat-linux2,
/mnt/galaxy/data/galaxy/sw/lib/python2.7/lib-tk, /mnt/galaxy/da
ta/galaxy/sw/lib/python2.7/lib-old,
/mnt/galaxy/data/galaxy/sw/lib/python2.7/lib-dynload,
/mnt/galaxy/data/galaxy/sw/lib/python2.7/site-packages
web0.log
172.17.130.113 - - [01/Aug/2014:17:14:58 -0400] GET
/api/histories/39a249c523d1b262/contents HTTP/1.0 200 -
http://galaxy.neb.com/root/index; Mozilla/5.0 (Macintosh; Intel Mac OS X
10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chr
ome/36.0.1985.125 Safari/537.36
172.17.130.113 - - [01/Aug/2014:17:15:03 -0400] GET
/datasets/49b19822485c7897/display/?preview=True HTTP/1.0 200 -
http://galaxy.neb.com/root/index; Mozilla/5.0 (Macintosh; Intel Mac OS X
10_9_4) AppleWebKit/537.36 (KHTML, like Ge
cko) Chrome/36.0.1985.125 Safari/537.36
172.17.131.223 - - [01/Aug/2014:17:15:03 -0400] GET
/api/histories/11ff4619ea0fcba9/contents HTTP/1.0 200 -
http://galaxy.neb.com/; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3)
AppleWebKit/537.76.4 (KHTML, like Gecko) Version/7.0
.4 Safari/537.76.4
172.17.130.113 - - [01/Aug/2014:17:15:04