So the permissions on these files seem fine right. That is odd. Next things I would check are that each directory leading up to /panfs/storage.local/scratch/galaxy-data/ is world executable and that the whole thing (/panfs/storage.local/scratch/galaxy-data/) is mounted and with the same permissions on the cluster worker nodes.
If users can indeed read that file - the error message must be misleading some how (that would not be entirely surprising). -John On Tue, Jun 10, 2014 at 4:48 PM, Shrum, Donald C <dcsh...@admin.fsu.edu> wrote: > Hi Jon and thanks for the reply. I am attempting to run jobs as the real > user as jobs will go to our HPC cluster. This will be an enterprise server. > > /panfs/storage.local/scratch/galaxy-data/ is world writable- > drwxrwxrwx 4 galaxy galaxy 4096 May 7 09:08 galaxy-data > > as is tmp > -bash-4.1$ ls -l /panfs/storage.local/scratch/galaxy-data/ > total 160 > drwxrwxrwx 2 galaxy galaxy 20480 Jun 10 17:00 tmp > > I got a little lost on the integrated datatypes configuration file... is that > an XML file? I'm not sure which file I'm looking for and I'm new to galaxy. > > --Donny > > -----Original Message----- > From: John Chilton [mailto:jmchil...@gmail.com] > Sent: Tuesday, June 10, 2014 5:33 PM > To: Shrum, Donald C > Cc: galaxy-dev@lists.bx.psu.edu > Subject: Re: [galaxy-dev] upload problems > > You didn't include this context, but I am guessing you are attempting to run > jobs as the "real" user? If not, ignore the rest of the e-mail. > > I would generally not recommend running the uploads as "real" user - it is a > complex process but should go relatively quick. > > Understand that may not be possible though. So that file is the integrated > datatypes configuration file I believe. There is just one global copy that > gets created with Galaxy boots up - so it cannot be chown-ed on a per job > basis. The thing is that Galaxy should be modifying it to be world readable > (https://bitbucket.org/galaxy/galaxy-central/annotate/e2b761a9b1d6d41db71b28df8b62862c7c300eba/lib/galaxy/datatypes/registry.py?at=default#cl-811) > - something is going wrong if it is not. Can you verify the file is 644? > > That leads me to believe that users don't have read access to the global temp > directory. Can you check if users can read > /panfs/storage.local/scratch/galaxy-data/tmp/? I think they will need to to > use some tools including uploads? > > If you cannot make this directory accessible to users - can you change > Galaxy's new_file_path so that it is some directory globally readable? > > -John > > On Tue, Jun 10, 2014 at 4:10 PM, Shrum, Donald C <dcsh...@admin.fsu.edu> > wrote: >> Hi all, >> >> I'm working with a problem with user uploaded files. After digging >> through the logs a bit and running the commands on at a time manually >> I think I've narrowed it to a permissions problem. This was confirmed >> by just running galaxy as root and the problem went away ;) >> >> -bash-4.1$ PYTHONPATH=/panfs/storage.local/software/galaxy-dist/lib/ >> -bash-4.1$ python >> /panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py >> /panfs/storage.local/software/galaxy-dist//panfs/storage.local/scratch/galaxy-data/tmp/tmpSuHquR >> /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf >> 6:/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/6/dataset_6_files:/panfs/storage.local/software/galaxy-dist/database/job_working_directory/000/6/galaxy_dataset_6.dat >> Traceback (most recent call last): >> File >> "/panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py", >> line 394, in <module> >> __main__() >> File >> "/panfs/storage.local/software/galaxy-dist/tools/data_source/upload.py", >> line 369, in __main__ >> registry.load_datatypes( root_dir=sys.argv[1], config=sys.argv[2] ) >> File >> "/panfs/storage.local/software/galaxy-dist/lib/galaxy/datatypes/registry.py", >> line 97, in load_datatypes >> tree = galaxy.util.parse_xml( config ) >> File >> "/panfs/storage.local/software/galaxy-dist/lib/galaxy/util/__init__.py", >> line 154, in parse_xml >> tree = ElementTree.parse(fname) >> File "build/bdist.linux-x86_64-ucs4/egg/elementtree/ElementTree.py", line >> 859, in parse >> File "build/bdist.linux-x86_64-ucs4/egg/elementtree/ElementTree.py", >> line 576, in parse >> IOError: [Errno 13] Permission denied: >> '/panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf' >> >> >> -bash-4.1$ ls -l >> /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf >> -rw------- 1 dcshrum dcshrum 317 Jun 10 16:30 >> /panfs/storage.local/scratch/galaxy-data/tmp/tmpYGRnAf >> >> >> It does not look like galaxy is using sudo to run the script. Suggestions >> to work this out? >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this and other >> Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
