Re: [galaxy-user] Flagstat crashes in non-linear workflows with TORQUE

[Louise-Amélie Schmitt]

Le 06/04/2011 13:57, Nate Coraor a écrit :
Louise-Amélie Schmitt wrote:
Hello everyone
I observed an issue when flagstat is incorporated in a workflow in which
the BAM file it works on is also used by another program (generate
pileup for instance) and is NOT the input dataset (generated by sam to
bam within the workflow).

I tested it with the local job runner and with TORQUE (with the pbs
scheduler and Maui).

- With the local job runner, it works just fine

- With TORQUE I get the following error message:
pbs_submit failed, PBS error 15033: No free connections
Hi,

This can most likely be fixed by increasing the value of NCONNECTS in
the TORQUE source, in src/include/libpbs.h, and recompiling on your
TORQUE server.  I haven't seen a problem after increasing the value to
20.

--nate

Hi,

Sorry this is old but I tried recompiling torque after setting the 
NCONNECTS to 20 and the issue's still there.

But there's more: It doesn't affect only flagstat but other non-linear 
workflows. One of the two jobs that are submitted when their "father" 
stopped running triggers the same error:
PBS error 15033: No free connections

And the best part is that it works fine sometimes. But when it crashes, 
it's always the same job that crashes.

Does anyone have a clue?

Cheers,
L-A
Surprisingly, other non-linear workflows work fine. I only observed this
error with flagstat. Moreover, when flagstat is in a linear workflow, it
works fine too. Ad if it is non-linear but the input dataset is the bam
file flagstat works on, it works fine too.

Please find attached one of the test workflow where I found the error.
The input dataset is a sam file.

Any clue?

Cheers,
LA
{
     "a_galaxy_workflow": "true",
     "annotation": "to see if it fails if not forked",
     "format-version": "0.1",
     "name": "test flagstat",
     "steps": {
         "0": {
             "annotation": "",
             "id": 0,
             "input_connections": {},
             "inputs": [
                 {
                     "description": "",
                     "name": "Input Dataset"
                 }
             ],
             "name": "Input dataset",
             "outputs": [],
             "position": {
                 "left": 200,
                 "top": 200
             },
             "tool_errors": null,
             "tool_id": null,
             "tool_state": "{\"name\": \"Input Dataset\"}",
             "tool_version": null,
             "type": "data_input",
             "user_outputs": []
         },
         "1": {
             "annotation": "",
             "id": 1,
             "input_connections": {
                 "source|input1": {
                     "id": 0,
                     "output_name": "output"
                 }
             },
             "inputs": [],
             "name": "SAM-to-BAM",
             "outputs": [
                 {
                     "name": "output1",
                     "type": "bam"
                 }
             ],
             "position": {
                 "left": 274.5,
                 "top": 307
             },
             "tool_errors": null,
             "tool_id": "sam_to_bam",
             "tool_state": "{\"source\": \"{\\\"index_source\\\": \\\"cached\\\", \\\"input1\\\": null, 
\\\"__current_case__\\\": 0}\", \"__page__\": 0}",
             "tool_version": "1.1.1",
             "type": "tool",
             "user_outputs": []
         },
         "2": {
             "annotation": "",
             "id": 2,
             "input_connections": {
                 "input1": {
                     "id": 1,
                     "output_name": "output1"
                 }
             },
             "inputs": [],
             "name": "flagstat",
             "outputs": [
                 {
                     "name": "output1",
                     "type": "txt"
                 }
             ],
             "position": {
                 "left": 396.5,
                 "top": 445
             },
             "tool_errors": null,
             "tool_id": "samtools_flagstat",
             "tool_state": "{\"__page__\": 0, \"input1\": \"null\"}",
             "tool_version": "1.0.0",
             "type": "tool",
             "user_outputs": []
         },
         "3": {
             "annotation": "",
             "id": 3,
             "input_connections": {
                 "refOrHistory|input1": {
                     "id": 1,
                     "output_name": "output1"
                 }
             },
             "inputs": [],
             "name": "Generate pileup",
             "outputs": [
                 {
                     "name": "output1",
                     "type": "tabular"
                 }
             ],
             "position": {
                 "left": 519,
                 "top": 340
             },
             "tool_errors": null,
             "tool_id": "sam_pileup",
             "tool_state": "{\"__page__\": 0, \"c\": \"{\\\"consensus\\\": \\\"no\\\", \\\"__current_case__\\\": 0}\", \"indels\": \"\\\"no\\\"\", \"refOrHistory\": 
\"{\\\"input1\\\": null, \\\"reference\\\": \\\"indexed\\\", \\\"__current_case__\\\": 0}\", \"lastCol\": \"\\\"no\\\"\", \"mapCap\": \"\\\"60\\\"\"}",
             "tool_version": "1.1.1",
             "type": "tool",
             "user_outputs": []
         }
     }
}
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

   http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

   http://lists.bx.psu.edu/

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

 http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

 http://lists.bx.psu.edu/