On Apr 25, 2012, at 8:25 AM, Louise-Amélie Schmitt wrote: > Hi, > > Thanks a lot it actually helped. It is not exactly as straightforward in > drmaa.py but somehow I could manage. > > However, it was not the problem. For some reason, the user needs to write > files from the node to job_working_directory/00X/XXXX/ and the latter is not > world-writable. I had to make everyone chmod everything to 777 to make it > work. Did I miss something?
That doesn't seem right. The subdirectory for your job underneath job_working_directory/00X/XXX/ should be chowned to the real user before the job is submitted and then chowned back to the galaxy user once the job is complete. --nate > > Best, > L-A > > > > Le 24/04/2012 15:17, Alban Lermine a écrit : >> Hi L-A, >> >> I run Galaxy as real user on our cluster with pbs (free version). >> >> We first configure LDAP authentification for having email account >> related to unix account (just cut the @curie.fr) >> Then I have modify pbs.py (in >> <GALAXY_DIR>galaxy-dist/lib/galaxy/jobs/runners) >> >> I have just disconnected the pbs submission through python library and >> replace it by a system call (just like I send jobs to the cluster with >> command line), here is the code used: >> >> galaxy_job_id = job_wrapper.job_id >> log.debug("(%s) submitting file %s" % ( galaxy_job_id, job_file ) ) >> log.debug("(%s) command is: %s" % ( galaxy_job_id, command_line ) ) >> >> # Submit job with system call instead of using python PBS library - >> Permit to run jobs as .. with sudo -u cmd prefix >> >> galaxy_job_idSTR = str(job_wrapper.job_id) >> galaxy_tool_idSTR = str(job_wrapper.tool.id) >> galaxy_job_name = >> galaxy_job_idSTR+"_"+galaxy_tool_idSTR+"_"+job_wrapper.user >> torque_options = runner_url.split("/") >> queue = torque_options >> ressources = torque_options >> user_mail = job_wrapper.user.split("@") >> username = user_mail >> >> torque_cmd = "sudo -u username echo "+"\""+command_line+"\" | qsub >> -o "+ofile+" -e "+efile+" -M "+job_wrapper.user+" -N "+galaxy_job_name+" >> -q "+queue+" "+ressources >> >> submit_pbs_job = os.popen(torque_cmd) >> >> job_id = submit_pbs_job.read().rstrip("\n") >> >> #Original job launcher >> #job_id = pbs.pbs_submit(c, job_attrs, job_file, pbs_queue_name, None) >> >> pbs.pbs_disconnect(c) >> >> Second thing I have done is to wait error and output file from torque in >> the finish_job function (if not, I never receive the output, seems to be >> your problem..), here is the code used: >> >> def finish_job( self, pbs_job_state ): >> """ >> Get the output/error for a finished job, pass to >> `job_wrapper.finish` >> and cleanup all the PBS temporary files. >> """ >> ofile = pbs_job_state.ofile >> efile = pbs_job_state.efile >> job_file = pbs_job_state.job_file >> >> # collect the output >> try: >> >> # With qsub system call, need to wait efile and ofile creation >> at the end of the job execution before reading them >> >> efileExists = os.path.isfile(efile) >> ofileExists = os.path.isfile(ofile) >> efileExistsSTR = str(efileExists) >> ofileExistsSTR = str(ofileExists) >> >> while efileExistsSTR == "False": >> time.sleep( 1 ) >> efileExists = os.path.isfile(efile) >> efileExistsSTR = str(efileExists) >> >> while ofileExistsSTR == "False": >> time.sleep( 1 ) >> ofileExists = os.path.isfile(ofile) >> ofileExistsSTR = str(ofileExists) >> >> # Back to original code >> >> ofh = file(ofile, "r") >> efh = file(efile, "r") >> stdout = ofh.read( 32768 ) >> stderr = efh.read( 32768 ) >> except: >> stdout = '' >> stderr = 'Job output not returned by PBS: the output >> datasets were deleted while the job was running, the job was manually >> dequeued or there was a cluster error.' >> log.debug(stderr) >> >> * Last step is to allow galaxy user to run sudo >> >> >> Hope it can help you finding your problem.. >> >> See you, >> >> Alban >> > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/