Re: [galaxy-dev] Non-admin tool install/import
Sorry, I meant Tool Factory when I wrote Tool Shed in my previous email. On Tue, Jun 10, 2014 at 5:25 PM, Ross wrote: > > > On Tue, Jun 10, 2014 at 7:37 AM, Karthik Gururaj < > gururaj.kart...@gmail.com> wrote: > >> Thanks - will take a look at the Tool Shed and see if there is anything >> there we can use in safe manner. Else, the old fashioned way of asking >> users to test out on their tools on a standalone Galaxy system and then >> requesting the administrator to pull in the relevant XML files. >> > > Sorry this is a bit confusing... > Just to be quite clear: The Tool Factory is available from the main Tool > Shed, but Tool Factory =/= Tool Shed !! > > The Tool Factory is just another Galaxy tool administrators can use to run > scripts interactively in Galaxy. > It installs automatically from the main Tool Shed and optionally generates > new Galaxy tools from working scripts. > New generated tools are in a tgz archive ready to be uploaded into Tool > Shed repositories. > > The Tool Shed is a specialised web server that supports version control > and management of Galaxy tool source code and automated installation into > Galaxy instances. > You could run a local Tool Shed and yes, your admins could use it to > install properly configured user provided tools. > > The Tool Factory allows your scripting-capable users of creating new > Galaxy tools from scripts if they run it as administrators of their own > laptop/development instances, or your administrators could use it to run > scripts directly on your private instance and optionally generate new safe > tools (by uploading the archives to your local tool shed then installing > those new tools into your local Galaxy) for ordinary users to use in their > workflows. > > > > >> Thanks, >> Karthik >> >> >> On Sun, Jun 8, 2014 at 6:25 PM, Ross wrote: >> >>> Hi, Karthik and John. >>> >>> Some details on the tool factory for anyone interested. >>> >>> Executive summary: it may be helpful in this context but only for >>> trusted administrators. >>> >>> TL;DR: >>> >>> Firstly, it will refuse to run for anyone other than a local Galaxy >>> administrator. This is because it exposes unrestricted scripting so should >>> only be installed if you can trust your administrative users not to run "cd >>> /; rm -rf *". I'd advise installing ONLY on your own private instance and >>> NEVER on a public Galaxy. >>> >>> Secondly, it has two modes of operation - script running and tool >>> generation. >>> >>> When executed without the option to generate a tool archive, it will run >>> a pasted (perl, python, R, bash) script creating an output in the history. >>> This history output is re-doable in the usual Galaxy way including allowing >>> the script to be edited and rerun, so it's possible to (eg) get a script >>> working interactively - galaxy as an IDE anyone ? :) >>> >>> Once a script runs on some test data, the tool factory will optionally >>> generate a complete tool shed compatible gzip which can be uploaded to any >>> tool shed as a new or updated repository. The generated tool includes the >>> supplied test data as a proper Galaxy functional test. Once a tool is in a >>> toolshed, it is just another Galaxy tool, ready to be installed to any >>> Galaxy like any other tool - but will require restarting of multiple web >>> processes as John mentions. >>> >>> If the script is safe, the tool is safe - there are no specific security >>> risks for tool factory generated tools other than the script itself. >>> >>> Finally, currently it takes only one input and generates one output >>> which is a substantial restriction - but of course the generated tool >>> source code is easy to edit if you need more complex I/O. It has a really >>> neat option to create a simple but useful HTML display with links and >>> thumbnails to arbitrary (eg) pdf or other output files from a script - the >>> tool form includes examples in all 4 scripting languages ready to cut and >>> paste, including one which generates 50 random images and presents them in >>> a grid as an HTML page for the user. >>> >>> On Mon, Jun 9, 2014 at 10:32 AM, John Chilton >>> wrote: >>> Galaxy doesn't really support this use case and it will be major effort to get it work this way I suspect. Pieces to look at include: The Galaxy Tool Factory (it has the ability to create reusable tools from scripts): http://www.ncbi.nlm.nih.gov/pubmed/23024011 You may be able to modify it in such a way that each tool is tagged with who created and then use ToolBox filters to limit added tools to a given user: https://wiki.galaxyproject.org/UserDefinedToolboxFilters I think the latest version of Galaxy has improved support for adding tools without requiring restarts (using message queues). I don't know if this will automatically work with the tool factory or not. I suspect fighting Galaxy at every step on this will frustrate you and th
Re: [galaxy-dev] Non-admin tool install/import
On Tue, Jun 10, 2014 at 7:37 AM, Karthik Gururaj wrote: > Thanks - will take a look at the Tool Shed and see if there is anything > there we can use in safe manner. Else, the old fashioned way of asking > users to test out on their tools on a standalone Galaxy system and then > requesting the administrator to pull in the relevant XML files. > Sorry this is a bit confusing... Just to be quite clear: The Tool Factory is available from the main Tool Shed, but Tool Factory =/= Tool Shed !! The Tool Factory is just another Galaxy tool administrators can use to run scripts interactively in Galaxy. It installs automatically from the main Tool Shed and optionally generates new Galaxy tools from working scripts. New generated tools are in a tgz archive ready to be uploaded into Tool Shed repositories. The Tool Shed is a specialised web server that supports version control and management of Galaxy tool source code and automated installation into Galaxy instances. You could run a local Tool Shed and yes, your admins could use it to install properly configured user provided tools. The Tool Factory allows your scripting-capable users of creating new Galaxy tools from scripts if they run it as administrators of their own laptop/development instances, or your administrators could use it to run scripts directly on your private instance and optionally generate new safe tools (by uploading the archives to your local tool shed then installing those new tools into your local Galaxy) for ordinary users to use in their workflows. > Thanks, > Karthik > > > On Sun, Jun 8, 2014 at 6:25 PM, Ross wrote: > >> Hi, Karthik and John. >> >> Some details on the tool factory for anyone interested. >> >> Executive summary: it may be helpful in this context but only for trusted >> administrators. >> >> TL;DR: >> >> Firstly, it will refuse to run for anyone other than a local Galaxy >> administrator. This is because it exposes unrestricted scripting so should >> only be installed if you can trust your administrative users not to run "cd >> /; rm -rf *". I'd advise installing ONLY on your own private instance and >> NEVER on a public Galaxy. >> >> Secondly, it has two modes of operation - script running and tool >> generation. >> >> When executed without the option to generate a tool archive, it will run >> a pasted (perl, python, R, bash) script creating an output in the history. >> This history output is re-doable in the usual Galaxy way including allowing >> the script to be edited and rerun, so it's possible to (eg) get a script >> working interactively - galaxy as an IDE anyone ? :) >> >> Once a script runs on some test data, the tool factory will optionally >> generate a complete tool shed compatible gzip which can be uploaded to any >> tool shed as a new or updated repository. The generated tool includes the >> supplied test data as a proper Galaxy functional test. Once a tool is in a >> toolshed, it is just another Galaxy tool, ready to be installed to any >> Galaxy like any other tool - but will require restarting of multiple web >> processes as John mentions. >> >> If the script is safe, the tool is safe - there are no specific security >> risks for tool factory generated tools other than the script itself. >> >> Finally, currently it takes only one input and generates one output which >> is a substantial restriction - but of course the generated tool source code >> is easy to edit if you need more complex I/O. It has a really neat option >> to create a simple but useful HTML display with links and thumbnails to >> arbitrary (eg) pdf or other output files from a script - the tool form >> includes examples in all 4 scripting languages ready to cut and paste, >> including one which generates 50 random images and presents them in a grid >> as an HTML page for the user. >> >> On Mon, Jun 9, 2014 at 10:32 AM, John Chilton >> wrote: >> >>> Galaxy doesn't really support this use case and it will be major >>> effort to get it work this way I suspect. Pieces to look at include: >>> >>> The Galaxy Tool Factory (it has the ability to create reusable tools >>> from scripts): >>> >>> http://www.ncbi.nlm.nih.gov/pubmed/23024011 >>> >>> You may be able to modify it in such a way that each tool is tagged >>> with who created and then use ToolBox filters to limit added tools to >>> a given user: >>> >>> https://wiki.galaxyproject.org/UserDefinedToolboxFilters >>> >>> I think the latest version of Galaxy has improved support for adding >>> tools without requiring restarts (using message queues). I don't know >>> if this will automatically work with the tool factory or not. >>> >>> I suspect fighting Galaxy at every step on this will frustrate you and >>> the users - and you are exposing all of your users data to every user >>> you give this privilege to. Is this a shared cluster or is dedicated >>> to Galaxy? If it is shared - it might be better for advanced users to >>> just get importing and exporting data to user directories really well
Re: [galaxy-dev] Non-admin tool install/import
Thanks - will take a look at the Tool Shed and see if there is anything there we can use in safe manner. Else, the old fashioned way of asking users to test out on their tools on a standalone Galaxy system and then requesting the administrator to pull in the relevant XML files. Thanks, Karthik On Sun, Jun 8, 2014 at 6:25 PM, Ross wrote: > Hi, Karthik and John. > > Some details on the tool factory for anyone interested. > > Executive summary: it may be helpful in this context but only for trusted > administrators. > > TL;DR: > > Firstly, it will refuse to run for anyone other than a local Galaxy > administrator. This is because it exposes unrestricted scripting so should > only be installed if you can trust your administrative users not to run "cd > /; rm -rf *". I'd advise installing ONLY on your own private instance and > NEVER on a public Galaxy. > > Secondly, it has two modes of operation - script running and tool > generation. > > When executed without the option to generate a tool archive, it will run a > pasted (perl, python, R, bash) script creating an output in the history. > This history output is re-doable in the usual Galaxy way including allowing > the script to be edited and rerun, so it's possible to (eg) get a script > working interactively - galaxy as an IDE anyone ? :) > > Once a script runs on some test data, the tool factory will optionally > generate a complete tool shed compatible gzip which can be uploaded to any > tool shed as a new or updated repository. The generated tool includes the > supplied test data as a proper Galaxy functional test. Once a tool is in a > toolshed, it is just another Galaxy tool, ready to be installed to any > Galaxy like any other tool - but will require restarting of multiple web > processes as John mentions. > > If the script is safe, the tool is safe - there are no specific security > risks for tool factory generated tools other than the script itself. > > Finally, currently it takes only one input and generates one output which > is a substantial restriction - but of course the generated tool source code > is easy to edit if you need more complex I/O. It has a really neat option > to create a simple but useful HTML display with links and thumbnails to > arbitrary (eg) pdf or other output files from a script - the tool form > includes examples in all 4 scripting languages ready to cut and paste, > including one which generates 50 random images and presents them in a grid > as an HTML page for the user. > > On Mon, Jun 9, 2014 at 10:32 AM, John Chilton wrote: > >> Galaxy doesn't really support this use case and it will be major >> effort to get it work this way I suspect. Pieces to look at include: >> >> The Galaxy Tool Factory (it has the ability to create reusable tools >> from scripts): >> >> http://www.ncbi.nlm.nih.gov/pubmed/23024011 >> >> You may be able to modify it in such a way that each tool is tagged >> with who created and then use ToolBox filters to limit added tools to >> a given user: >> >> https://wiki.galaxyproject.org/UserDefinedToolboxFilters >> >> I think the latest version of Galaxy has improved support for adding >> tools without requiring restarts (using message queues). I don't know >> if this will automatically work with the tool factory or not. >> >> I suspect fighting Galaxy at every step on this will frustrate you and >> the users - and you are exposing all of your users data to every user >> you give this privilege to. Is this a shared cluster or is dedicated >> to Galaxy? If it is shared - it might be better for advanced users to >> just get importing and exporting data to user directories really well. >> In my previous position at MSI we created a set of tools that allowed >> Galaxy to SCP files as the user to our login nodes (using a SSH key >> scheme) - likewise creating directories user's can upload files can >> enabled power users. >> >> Hopefully this helps, >> >> -John >> >> On Sat, Jun 7, 2014 at 1:12 PM, Karthik Gururaj >> wrote: >> > Hello, >> > We have a centralized Galaxy installation for out cluster which can be >> > used by many users (backend interfaces with HTCondor to run jobs). Is >> there >> > any way for non-administrator users to import/install their own tools so >> > that they may use it in their workflows (under a subsection called >> > "unstable", for example? Ideally, without having to restart the Galaxy >> > daemons whenever a change is made. >> > It would be fine if these tools are not visible to other users till the >> > administrator says so. >> > Thanks, >> > Karthik >> > >> > ___ >> > Please keep all replies on the list by using "reply all" >> > in your mail client. To manage your subscriptions to this >> > and other Galaxy lists, please use the interface at: >> > http://lists.bx.psu.edu/ >> > >> > To search Galaxy mailing lists use the unified search at: >> > http://galaxyproject.org/search/mailinglists/ >> ___
Re: [galaxy-dev] Non-admin tool install/import
Hi, Karthik and John. Some details on the tool factory for anyone interested. Executive summary: it may be helpful in this context but only for trusted administrators. TL;DR: Firstly, it will refuse to run for anyone other than a local Galaxy administrator. This is because it exposes unrestricted scripting so should only be installed if you can trust your administrative users not to run "cd /; rm -rf *". I'd advise installing ONLY on your own private instance and NEVER on a public Galaxy. Secondly, it has two modes of operation - script running and tool generation. When executed without the option to generate a tool archive, it will run a pasted (perl, python, R, bash) script creating an output in the history. This history output is re-doable in the usual Galaxy way including allowing the script to be edited and rerun, so it's possible to (eg) get a script working interactively - galaxy as an IDE anyone ? :) Once a script runs on some test data, the tool factory will optionally generate a complete tool shed compatible gzip which can be uploaded to any tool shed as a new or updated repository. The generated tool includes the supplied test data as a proper Galaxy functional test. Once a tool is in a toolshed, it is just another Galaxy tool, ready to be installed to any Galaxy like any other tool - but will require restarting of multiple web processes as John mentions. If the script is safe, the tool is safe - there are no specific security risks for tool factory generated tools other than the script itself. Finally, currently it takes only one input and generates one output which is a substantial restriction - but of course the generated tool source code is easy to edit if you need more complex I/O. It has a really neat option to create a simple but useful HTML display with links and thumbnails to arbitrary (eg) pdf or other output files from a script - the tool form includes examples in all 4 scripting languages ready to cut and paste, including one which generates 50 random images and presents them in a grid as an HTML page for the user. On Mon, Jun 9, 2014 at 10:32 AM, John Chilton wrote: > Galaxy doesn't really support this use case and it will be major > effort to get it work this way I suspect. Pieces to look at include: > > The Galaxy Tool Factory (it has the ability to create reusable tools > from scripts): > > http://www.ncbi.nlm.nih.gov/pubmed/23024011 > > You may be able to modify it in such a way that each tool is tagged > with who created and then use ToolBox filters to limit added tools to > a given user: > > https://wiki.galaxyproject.org/UserDefinedToolboxFilters > > I think the latest version of Galaxy has improved support for adding > tools without requiring restarts (using message queues). I don't know > if this will automatically work with the tool factory or not. > > I suspect fighting Galaxy at every step on this will frustrate you and > the users - and you are exposing all of your users data to every user > you give this privilege to. Is this a shared cluster or is dedicated > to Galaxy? If it is shared - it might be better for advanced users to > just get importing and exporting data to user directories really well. > In my previous position at MSI we created a set of tools that allowed > Galaxy to SCP files as the user to our login nodes (using a SSH key > scheme) - likewise creating directories user's can upload files can > enabled power users. > > Hopefully this helps, > > -John > > On Sat, Jun 7, 2014 at 1:12 PM, Karthik Gururaj > wrote: > > Hello, > > We have a centralized Galaxy installation for out cluster which can be > > used by many users (backend interfaces with HTCondor to run jobs). Is > there > > any way for non-administrator users to import/install their own tools so > > that they may use it in their workflows (under a subsection called > > "unstable", for example? Ideally, without having to restart the Galaxy > > daemons whenever a change is made. > > It would be fine if these tools are not visible to other users till the > > administrator says so. > > Thanks, > > Karthik > > > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in y
Re: [galaxy-dev] Non-admin tool install/import
Galaxy doesn't really support this use case and it will be major effort to get it work this way I suspect. Pieces to look at include: The Galaxy Tool Factory (it has the ability to create reusable tools from scripts): http://www.ncbi.nlm.nih.gov/pubmed/23024011 You may be able to modify it in such a way that each tool is tagged with who created and then use ToolBox filters to limit added tools to a given user: https://wiki.galaxyproject.org/UserDefinedToolboxFilters I think the latest version of Galaxy has improved support for adding tools without requiring restarts (using message queues). I don't know if this will automatically work with the tool factory or not. I suspect fighting Galaxy at every step on this will frustrate you and the users - and you are exposing all of your users data to every user you give this privilege to. Is this a shared cluster or is dedicated to Galaxy? If it is shared - it might be better for advanced users to just get importing and exporting data to user directories really well. In my previous position at MSI we created a set of tools that allowed Galaxy to SCP files as the user to our login nodes (using a SSH key scheme) - likewise creating directories user's can upload files can enabled power users. Hopefully this helps, -John On Sat, Jun 7, 2014 at 1:12 PM, Karthik Gururaj wrote: > Hello, > We have a centralized Galaxy installation for out cluster which can be > used by many users (backend interfaces with HTCondor to run jobs). Is there > any way for non-administrator users to import/install their own tools so > that they may use it in their workflows (under a subsection called > "unstable", for example? Ideally, without having to restart the Galaxy > daemons whenever a change is made. > It would be fine if these tools are not visible to other users till the > administrator says so. > Thanks, > Karthik > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Non-admin tool install/import
Hello, We have a centralized Galaxy installation for out cluster which can be used by many users (backend interfaces with HTCondor to run jobs). Is there any way for non-administrator users to import/install their own tools so that they may use it in their workflows (under a subsection called "unstable", for example? Ideally, without having to restart the Galaxy daemons whenever a change is made. It would be fine if these tools are not visible to other users till the administrator says so. Thanks, Karthik ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
