Re: [galaxy-dev] Packaging tools with language specific dependencies
Hi Renato, Am 05.08.2014 um 17:34 schrieb Renato Alves: Hi Bjoern, Thanks for the info and the fix. So far we had no issues but it's not extensive testing. I'll let you know if it fails for some reason. In addition to the shebang change we also started using "perl script" instead of simply "script". Since "perl" is looked up in PATH, all else works as expected. Can you try to explain this a bit more? It does get a bit tricky if you need to guess where the script is located. "$(which script)" works but it's hacky. Still, it's the only way you can "fix" the shebang without changing it before. PS: There's shutil.which() but only python 3.3 onwards. Anything I can do to support you? I'm right that the shebang modifications work for you? Do you think that should go into Galaxy Core, so that every perl script under /bin/ will get a simple shebang? Thanks, Bjoern Cheers, Renato Quoting Björn Grüning on 02-08-2014 15:26: Hi Renato, cpanm is not the only problem it seems to be an general perl 'feature'. How annoying :( As I figured from: https://metacpan.org/source/RJBS/perl-5.20.0/Configure#L9162 we can use:-Dstartperl='#!/usr/bin/env perl' to change them via install. Now every perl script from the main perl has the correct shebang. Unfortunately, every other package installed with cpanm or Makefile.PL will have this annoying long shebang. I'm not sure there is a switch in cpan/cpanm to do that. Under cpan there are a few repositories, called change_shebang... so I assume we are not the only ones that have this problem. So I ended up with: sed -i 's|#!$INSTALL_DIR/bin/|#!/usr/bin/env |' $INSTALL_DIR/bin/* I have uploaded it to the testtoolshed and to galaxytools. If this is working for you, also with depending repositories, I will try to add this to into the Tool Shed core to make this sed command implicit for all perl tools. Cheers, Bjoern Am 01.08.2014 um 16:06 schrieb Renato Alves: Hi BJörn, We have run into some problems with the perl environment. This might also affect other tools that rely on a shebang with absolute paths to the interpreter. Details below. During the installation of the perl package (we tried package_perl_5_18 from main and testing toolshed), the cpanm script gets installed but when executed it is invalid/not found. The issue seems to be the length of the shebang in the script which is limited to 80 characters. In cpanm it's: !#/galaxy_dist/tool_dependencies/perl/5.18.1/bgruening/package_perl_5_18/e89824189ec6/bin/perl That is 92 characters long. When launching the script it gets truncated at "...perl_5_18/e898241", causing the command to fail with a "bad interpreter" error. I was able to workaround this problem by editing the script manually and replacing the long shebang by: #!/usr/bin/env perl As it stands it seems relying on the shebang to use the correct interpreter is a problem. Repositories with long (character wise) names, versions or owners will be more likely to suffer from this. So will installations that are not close to the system root. Renato Quoting Björn Grüning on 30-07-2014 12:04: Hi Renato, Am 30.07.2014 um 12:21 schrieb Renato Alves: Hi everyone, Is there any standard or commonly used way to package tools that have language specific dependencies. I know that with Python libraries one can use setup_virtualenv and with Java jars the JAVA_JAR_LIB strategy is used. Is there anything equivalent for R, Perl and Ruby libraries? please have a look at: https://github.com/bgruening/galaxytools/tree/master/test_repositories If you have any questions I'm happy to help you! Bjoern Thanks Renato ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Packaging tools with language specific dependencies
Hi Bjoern, Thanks for the info and the fix. So far we had no issues but it's not extensive testing. I'll let you know if it fails for some reason. In addition to the shebang change we also started using "perl script" instead of simply "script". Since "perl" is looked up in PATH, all else works as expected. It does get a bit tricky if you need to guess where the script is located. "$(which script)" works but it's hacky. Still, it's the only way you can "fix" the shebang without changing it before. PS: There's shutil.which() but only python 3.3 onwards. Cheers, Renato Quoting Björn Grüning on 02-08-2014 15:26: > Hi Renato, > > cpanm is not the only problem it seems to be an general perl 'feature'. > How annoying :( > > As I figured from: > https://metacpan.org/source/RJBS/perl-5.20.0/Configure#L9162 > > we can use:-Dstartperl='#!/usr/bin/env perl' to change them via install. > Now every perl script from the main perl has the correct shebang. > > Unfortunately, every other package installed with cpanm or Makefile.PL > will have this annoying long shebang. > > I'm not sure there is a switch in cpan/cpanm to do that. Under cpan > there are a few repositories, called change_shebang... so I assume we > are not the only ones that have this problem. > > So I ended up with: > > sed -i > 's|#!$INSTALL_DIR/bin/|#!/usr/bin/env |' $INSTALL_DIR/bin/* > > I have uploaded it to the testtoolshed and to galaxytools. > If this is working for you, also with depending repositories, I will try > to add this to into the Tool Shed core to make this sed command implicit > for all perl tools. > > Cheers, > Bjoern > > > > Am 01.08.2014 um 16:06 schrieb Renato Alves: >> Hi BJörn, >> >> We have run into some problems with the perl environment. >> This might also affect other tools that rely on a shebang with absolute >> paths to the interpreter. Details below. >> >> During the installation of the perl package (we tried package_perl_5_18 >> from main and testing toolshed), the cpanm script gets installed but >> when executed it is invalid/not found. >> >> The issue seems to be the length of the shebang in the script which is >> limited to 80 characters. In cpanm it's: >> >> !#/galaxy_dist/tool_dependencies/perl/5.18.1/bgruening/package_perl_5_18/e89824189ec6/bin/perl >> >> >> That is 92 characters long. When launching the script it gets truncated >> at "...perl_5_18/e898241", causing the command to fail with a "bad >> interpreter" error. >> >> I was able to workaround this problem by editing the script manually and >> replacing the long shebang by: #!/usr/bin/env perl >> >> >> As it stands it seems relying on the shebang to use the correct >> interpreter is a problem. Repositories with long (character wise) names, >> versions or owners will be more likely to suffer from this. So will >> installations that are not close to the system root. >> >> Renato >> >> Quoting Björn Grüning on 30-07-2014 12:04: >>> Hi Renato, >>> >>> Am 30.07.2014 um 12:21 schrieb Renato Alves: Hi everyone, Is there any standard or commonly used way to package tools that have language specific dependencies. I know that with Python libraries one can use setup_virtualenv and with Java jars the JAVA_JAR_LIB strategy is used. Is there anything equivalent for R, Perl and Ruby libraries? >>> >>> please have a look at: >>> https://github.com/bgruening/galaxytools/tree/master/test_repositories >>> >>> If you have any questions I'm happy to help you! >>> Bjoern >>> Thanks Renato ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ >> signature.asc Description: OpenPGP digital signature ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Packaging tools with language specific dependencies
Hi Renato, cpanm is not the only problem it seems to be an general perl 'feature'. How annoying :( As I figured from: https://metacpan.org/source/RJBS/perl-5.20.0/Configure#L9162 we can use:-Dstartperl='#!/usr/bin/env perl' to change them via install. Now every perl script from the main perl has the correct shebang. Unfortunately, every other package installed with cpanm or Makefile.PL will have this annoying long shebang. I'm not sure there is a switch in cpan/cpanm to do that. Under cpan there are a few repositories, called change_shebang... so I assume we are not the only ones that have this problem. So I ended up with: sed -i 's|#!$INSTALL_DIR/bin/|#!/usr/bin/env |' $INSTALL_DIR/bin/* I have uploaded it to the testtoolshed and to galaxytools. If this is working for you, also with depending repositories, I will try to add this to into the Tool Shed core to make this sed command implicit for all perl tools. Cheers, Bjoern Am 01.08.2014 um 16:06 schrieb Renato Alves: Hi BJörn, We have run into some problems with the perl environment. This might also affect other tools that rely on a shebang with absolute paths to the interpreter. Details below. During the installation of the perl package (we tried package_perl_5_18 from main and testing toolshed), the cpanm script gets installed but when executed it is invalid/not found. The issue seems to be the length of the shebang in the script which is limited to 80 characters. In cpanm it's: !#/galaxy_dist/tool_dependencies/perl/5.18.1/bgruening/package_perl_5_18/e89824189ec6/bin/perl That is 92 characters long. When launching the script it gets truncated at "...perl_5_18/e898241", causing the command to fail with a "bad interpreter" error. I was able to workaround this problem by editing the script manually and replacing the long shebang by: #!/usr/bin/env perl As it stands it seems relying on the shebang to use the correct interpreter is a problem. Repositories with long (character wise) names, versions or owners will be more likely to suffer from this. So will installations that are not close to the system root. Renato Quoting Björn Grüning on 30-07-2014 12:04: Hi Renato, Am 30.07.2014 um 12:21 schrieb Renato Alves: Hi everyone, Is there any standard or commonly used way to package tools that have language specific dependencies. I know that with Python libraries one can use setup_virtualenv and with Java jars the JAVA_JAR_LIB strategy is used. Is there anything equivalent for R, Perl and Ruby libraries? please have a look at: https://github.com/bgruening/galaxytools/tree/master/test_repositories If you have any questions I'm happy to help you! Bjoern Thanks Renato ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Packaging tools with language specific dependencies
Hi BJörn, We have run into some problems with the perl environment. This might also affect other tools that rely on a shebang with absolute paths to the interpreter. Details below. During the installation of the perl package (we tried package_perl_5_18 from main and testing toolshed), the cpanm script gets installed but when executed it is invalid/not found. The issue seems to be the length of the shebang in the script which is limited to 80 characters. In cpanm it's: !#/galaxy_dist/tool_dependencies/perl/5.18.1/bgruening/package_perl_5_18/e89824189ec6/bin/perl That is 92 characters long. When launching the script it gets truncated at "...perl_5_18/e898241", causing the command to fail with a "bad interpreter" error. I was able to workaround this problem by editing the script manually and replacing the long shebang by: #!/usr/bin/env perl As it stands it seems relying on the shebang to use the correct interpreter is a problem. Repositories with long (character wise) names, versions or owners will be more likely to suffer from this. So will installations that are not close to the system root. Renato Quoting Björn Grüning on 30-07-2014 12:04: > Hi Renato, > > Am 30.07.2014 um 12:21 schrieb Renato Alves: >> Hi everyone, >> >> Is there any standard or commonly used way to package tools that have >> language specific dependencies. >> >> I know that with Python libraries one can use setup_virtualenv and with >> Java jars the JAVA_JAR_LIB strategy is used. >> Is there anything equivalent for R, Perl and Ruby libraries? > > please have a look at: > https://github.com/bgruening/galaxytools/tree/master/test_repositories > > If you have any questions I'm happy to help you! > Bjoern > >> Thanks >> Renato >> >> >> >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >>http://galaxyproject.org/search/mailinglists/ >> signature.asc Description: OpenPGP digital signature ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Packaging tools with language specific dependencies
Hi Björn, Thanks, this is exactly what I was looking for. Cheers, Renato Quoting Björn Grüning on 30-07-2014 12:04: > Hi Renato, > > Am 30.07.2014 um 12:21 schrieb Renato Alves: >> Hi everyone, >> >> Is there any standard or commonly used way to package tools that have >> language specific dependencies. >> >> I know that with Python libraries one can use setup_virtualenv and with >> Java jars the JAVA_JAR_LIB strategy is used. >> Is there anything equivalent for R, Perl and Ruby libraries? > > please have a look at: > https://github.com/bgruening/galaxytools/tree/master/test_repositories > > If you have any questions I'm happy to help you! > Bjoern > >> Thanks >> Renato >> >> >> >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >>http://galaxyproject.org/search/mailinglists/ >> signature.asc Description: OpenPGP digital signature ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Packaging tools with language specific dependencies
Hi Renato, Am 30.07.2014 um 12:21 schrieb Renato Alves: Hi everyone, Is there any standard or commonly used way to package tools that have language specific dependencies. I know that with Python libraries one can use setup_virtualenv and with Java jars the JAVA_JAR_LIB strategy is used. Is there anything equivalent for R, Perl and Ruby libraries? please have a look at: https://github.com/bgruening/galaxytools/tree/master/test_repositories If you have any questions I'm happy to help you! Bjoern Thanks Renato ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Packaging tools with language specific dependencies
Hi everyone, Is there any standard or commonly used way to package tools that have language specific dependencies. I know that with Python libraries one can use setup_virtualenv and with Java jars the JAVA_JAR_LIB strategy is used. Is there anything equivalent for R, Perl and Ruby libraries? Thanks Renato signature.asc Description: OpenPGP digital signature ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
