Hi all,

after an update to the following changeset(14859:7ba05957588a, stable,
05.12.14), our bam files that are uploaded(linked) to a data library, are
no longer indexed. The metadata_xxx.dat is created, but it stays empty.
The following error message appears in the log, although the state of the
dataset is 'ok':

galaxy.jobs WARNING 2014-12-05 12:47:02,218 Error accessing /g/K/K27.bam,
will retry: [Errno 1] Operation not permitted: '/g/K/K27.bam'
galaxy.jobs WARNING 2014-12-08 13:38:57,045 Error accessing /g/K/K2.bam,
will retry: [Errno 1] Operation not permitted: '/g/K/K2.bam'

All file permissions are correct (i.e. galaxy owns them). Furthermore,
executing samtools index, just works on those files:
samtools index /g/K/K2.bam
/g/galaxy/galaxy_data/files/_metadata_files/006/metadata_6598.dat

When uploading the file - "copy files into galaxy" - the samtools index
just works.

==============

Now, on a clean local install(14874:885f940bff64, stable, 05.12.14) and
samtools installed globally and with the bam file sorted, I get the
following situation:
When I try to upload this bam to a data library by linking the following
error is shown on the dataset (note: here the dataset is set in error
state, which does not happen on our server)

Uploaded by: schol...@embl.de
Date uploaded: Mon Dec 8 17:42:39 2014 (UTC)
File size: 2.6 GB
UUID: d23cf11a-0372-41cb-939a-7c8761d78b73
Data type: auto
Build: ?
Miscellaneous information: Traceback (most recent call last): File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in __main__() File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 396, in _
Job Standard Error

Traceback (most recent call last):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in
    __main__()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 396, in __main__
    add_file( dataset, registry, json_file, output_path )
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 294, in add_file
    if datatype.dataset_content_needs_grooming( dataset.path ):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 147, in dataset_content_needs_grooming
    version = self._get_samtools_version()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 129, in _get_samtools_version
    output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE,
stdout=subprocess.PIPE ).communicate()[1]
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 711, in __init__
    errread, errwrite)
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

error
Database/Build: ?
Number of data lines: None
Disk file: /Users/scholtalbers/workspace/idr_data/WT1.sort.bam

===============================
When uploading the bam without linking, I see the following processes:
Upload->set meta->samtools index->'error state'

 Miscellaneous information: uploaded bam file Traceback (most recent call
last): File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in __main__() File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.p
Job Standard Error

Traceback (most recent call last):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 407, in
    __main__()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 396, in __main__
    add_file( dataset, registry, json_file, output_path )
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py",
line 324, in add_file
    if link_data_only == 'copy_files' and
datatype.dataset_content_needs_grooming( output_path ):
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 147, in dataset_content_needs_grooming
    version = self._get_samtools_version()
  File
"/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py",
line 129, in _get_samtools_version
    output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE,
stdout=subprocess.PIPE ).communicate()[1]
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 711, in __init__
    errread, errwrite)
  File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

error
Database/Build: ?
Number of data lines: None
Disk file:
/Users/scholtalbers/workspace/galaxy-dist-new/database/files/000/dataset_6.dat


=================================

Although the error messages are different, it might be related?

Cheers,

Jelle
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to