Re: [easybuild] easy_update patch release

[Sam Moors]

Thank you John!

Sam

On Sat, Mar 9, 2019 at 1:42 AM Dey, John F  wrote:

> Patch release for easy_update has been pushed. Many small improvements
> have been made to Python package checking.  Two new command line options
> have been added:  --Meta and –tree.  The new options are documented in the
> repo,  along with examples on how to use command line options.
>
>
>
> https://github.com/fizwit/easy_update
>
>
>
>
>
> *John Dey*
>
> *HPC Operations*
>
> SciComp Computing
> *O* *206.667.4308*
> *M* *360.649.2731*
>
> jf...@fredhutch.org
>
>
>
> SciComp Office Hours
>
> Wed. 10-noon M4-B102
>
>
>
> [image:
> http://www.fredhutch.org/content/dam/public/email-signatures/3/fred_hutch_logo.png]
> 
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N., Mail Stop J4-402
> Seattle, WA 98109
> *fredhutch.org *
>

[easybuild] easy_update patch release

[Dey, John F]

Patch release for easy_update has been pushed. Many small improvements have 
been made to Python package checking.  Two new command line options have been 
added:  --Meta and –tree.  The new options are documented in the repo,  along 
with examples on how to use command line options.

https://github.com/fizwit/easy_update


John Dey
HPC Operations
SciComp Computing
O 206.667.4308
M 360.649.2731
jf...@fredhutch.org

SciComp Office Hours
Wed. 10-noon M4-B102

[http://www.fredhutch.org/content/dam/public/email-signatures/3/fred_hutch_logo.png]
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N., Mail Stop J4-402
Seattle, WA 98109
fredhutch.org

[easybuild] easy_update new release

[Dey, John F]

Easy_Update now supports PythonPackage and RPackage easyconfigs. Easy_Update 
will read exts_list from R and Python easyconfig listed in the depencies of the 
R or Python package. Only the package name will be verified from the dependent 
language. Version information will be updated for the primary EasyConfig R or 
Python easyconfig that is specified. Easy_Update does not search EB paths all 
easyconfig files specified must be local. The --add feature has been removed.

Available here:
https://github.com/fizwit/easy_update
and from the scripts directory:
https://github.com/FredHutch/easybuild-life-sciences

./easy_update.py --verbose statsmodels-0.6.1-foss-2018b-Python-2.7.15.eb
   patsy : 0.4.1 -> 0.5.1   (update) [ 2, 1]
 statsmodels : 0.6.1 -> 0.9.0   (update) [ 2, 2]
Updated Packages: 2
New Packages: 0
Dropped Packages: 0

Note:  statsmodels verion is displayed but not updated in the ‘.update’ file. 
The version will have to updated by hand. The version number is defined at the 
top of the file and is not in the exts_list.  This will be fixed in a future 
version :-)




John Dey
HPC Operations
SciComp Computing
O 206.667.4308
M 360.649.2731
jf...@fredhutch.org

SciComp Office Hours
Wed. 10-noon M4-B102

[http://www.fredhutch.org/content/dam/public/email-signatures/3/fred_hutch_logo.png]
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N., Mail Stop J4-402
Seattle, WA 98109
fredhutch.org

[easybuild] easy_update

[Dey, John F]

Easy_update has been updated for Python usage. Easy_update updates the 
exts_list for Python EasyBuild configs. The program is in the scripts directory 
of our easybuild repository:   
https://github.com/FredHutch/easybuild-life-sciences


-  Updated to use Pypi.org

-  Updates checksums

-  Easy_update has improved meta data debugging for Python modules.

-  search feature to locate single modules and output is in format for 
exts_list

-  R update features are broken for now, but Python was a necessity.


Search for current version of iPython:
easy_update.py --search iPython --pyver 3.7 --meta
('iPython', '6.4.0', {
'source_urls': ['PYPI_SOURCE'],
'checksums': 
['eca537aa61592aca2fef4adea12af8e42f5c335004dfa80c78caf80e8b525e5c'],
}),
'filename': 'ipython-6.4.0.tar.gz'
'packagetype': 'sdist'
'url': 
'https://files.pythonhosted.org/packages/ee/01/2a85cd07f5a43fa2e86d60001c213647252662d44a0c2e3d69471a058f1b/ipython-6.4.0.tar.gz'
'python_version': 'source'
'requires_dist': '[u'numpy; python_version >= "3.4" and extra == \'test\'', 
u"ipykernel; extra == 'test'", u"nbformat; extra == 'test'", u"pygments; extra 
== 'test'", u"testpath; extra == 'test'", u"requests; extra == 'test'", u"nose 
(>=0.10.1); extra== 'test'", u"qtconsole; extra == 'qtconsole'", u"ipyparallel; 
extra == 'parallel'", u"ipywidgets; extra == 'notebook'", u"notebook; extra == 
'notebook'", u"nbformat; extra == 'nbformat'", u"nbconvert; extra == 
'nbconvert'", u"ipykernel; extra == 'kernel'", u"Sphinx (>=1.3); extra == 
'doc'", u"pygments; extra == 'all'", u"nbconvert; extra == 'all'", 
u"ipywidgets; extra == 'all'", u"qtconsole; extra == 'all'", u"testpath; extra 
== 'all'", u"Sphinx (>=1.3); extra == 'all'", u"requests; extra == 'all'", 
u"ipyparallel; extra == 'all'", u"nose (>=0.10.1); extra == 'all'", u"notebook; 
extra == 'all'", u"ipykernel; extra == 'all'", u"nbformat; extra == 'all'", 
u'win-unicode-console (>=0.5); sys_platform == "win32" and python_version < 
"3.6"', u'colorama; sys_platform == "win32"', u'appnope; sys_platform == 
"darwin"', u'pexpect; sys_platform != "win32"', u'pathlib2; python_version == 
"3.3"', u'typing; python_version <= "3.4"', u'backcall', u'pygments', 
u'prompt-toolkit (<2.0.0,>=1.0.15)', u'traitlets (>=4.2)', u'simplegeneric 
(>0.8)', u'pickleshare', u'decorator', u'jedi (>=0.10)', u'setuptools 
(>=18.5)']'
'requires_python': '>=3.3'


Example run with the current Python 3.7.0 Easyconfig:
./easy_update.py --verbose Python-3.7.0-foss-2018b.eb
versionsuffix not defined
  setuptools : 39.2.0 -> 40.0.0 (update) [42, 1]
 pip : 10.0.1 (keep) [42, 2]
nose : 1.3.7  (keep) [42, 3]
   numpy : 1.14.5 (keep) [42, 4]
   scipy : 1.1.0  (keep) [42, 5]
   blist : 1.3.6  (keep) [42, 6]
  mpi4py : 3.0.0  (keep) [42, 7]
paycheck : 1.0.2  (keep) [42, 8]
 pbr : 4.0.4 -> 4.1.0   (update) [42, 9]
  Cython : 0.28.3 -> 0.28.4 (update) [42, 10]
 six : 1.11.0 (keep) [42, 11]
 python-dateutil : 2.7.3  (keep) [42, 12]
deap : 1.2.2  (keep) [42, 13]
   decorator : 4.3.0  (keep) [42, 14]
   liac-arff : 2.2.2  (keep) [42, 15]
pycrypto : 2.6.1  (keep) [42, 16]
   ecdsa : 0.13   (keep) [42, 17]
   pycparser : 2.18   (keep) [42, 18]
cffi : 1.11.5 (keep) [42, 19]
  asn1crypto : 0.24.0 (keep) [42, 20]
idna : 2.7(keep) [42, 21]
cryptography : 2.2.2  (keep) [42, 22]
  pyasn1 : 0.4.3  (keep) [42, 23]
  PyNaCl : 1.2.1  (keep) [42, 24]
  bcrypt : 3.1.4  (keep) [42, 25]
paramiko : 2.4.1  (keep) [42, 26]
   pyparsing : 2.2.0  (keep) [42, 27]
   netifaces : 0.10.7 (keep) [42, 28]
 netaddr : 0.7.19 (keep) [42, 29]
mock : 2.0.0  (keep) [42, 30]
pytz : 2018.4 -> 2018.5 (update) [42, 31]
  pandas : 0.23.3 (keep) [42, 32]
   bitstring : 3.1.5  (keep) [42, 33]
  virtualenv : 16.0.0 (keep) [42, 34]
  docopt : 0.6.2  (keep) [42, 35]
  joblib : 0.11 -> 0.12.0   (update) [42, 36]
 chardet : 

Re: [easybuild] easy_update Update

[Kenneth Hoste]

Dear John,

Thanks for the update.

I still feel we should look into integrating this into the EasyBuild 
framework, but I haven't been able to find any time whatsoever for this...



regards,

Kenneth

On 07/03/2018 00:26, Dey, John F wrote:


I have released an improved version of easy_update.  It is available 
via github: https://github.com/FredHutch/easybuild-life-sciences/ In 
the Scripts directory. The README.md file in the scripts directory is 
the man page for easy_update.  Please file and issue with our repo if 
you encounter problems.


Many bugs have been fixed and there are a few new features.

--search [package name] easy_update will search a repository for a 
single package on the command line.  The output is in a format that 
can be cut/pasted into and easyconfig file.


--meta Output all metadata about a package from a repository. This is 
handy for debugging issues with a package.


--pyver, --rver, --biocver; These three flag are to be used with the 
Search feature. Either use pyver for PyPi or rver and biocver for 
searching CRAN and BioConductor.


At the FredHutch we have combined R packages and BioCondutor packages 
into a single R package. To implement having packages from multiple 
repositories in one easyconfig the variable; biocver must be defined.


I hope you can find this tool useful,

*John Dey*

*HPC Operations*

SciComp Computing
*O* _206.667.4308_
*M* _360.649.2731_

jf...@fredhutch.org 

SciComp Office Hours

Wed. 10-noon M4-B102

ttp://www.fredhutch.org/content/dam/public/email-signatures/3/fred_hutch_logo.png 


Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N., Mail Stop J4-402
Seattle, WA 98109
*fredhutch.org *

[easybuild] easy_update Update

[Dey, John F]

I have released an improved version of easy_update.  It is available via 
github:  https://github.com/FredHutch/easybuild-life-sciences/   In the Scripts 
directory. The README.md file in the scripts directory is the man page for 
easy_update.  Please file and issue with our repo if you encounter problems.

Many bugs have been fixed and there are a few new features.

--search [package name] easy_update will search a repository for a single 
package on the command line.  The output is in a format that can be cut/pasted 
into and easyconfig file.

--meta Output all metadata about a package from a repository. This is handy for 
debugging issues with a package.

--pyver, --rver, --biocver; These three flag are to be used with the Search 
feature. Either use pyver for PyPi or rver and biocver for searching CRAN and 
BioConductor.

At the FredHutch we have combined R packages and BioCondutor packages into a 
single R package. To implement having packages from multiple repositories in 
one easyconfig the variable; biocver must be defined.

I hope you can find this tool useful,

John Dey
HPC Operations
SciComp Computing
O 206.667.4308
M 360.649.2731
jf...@fredhutch.org

SciComp Office Hours
Wed. 10-noon M4-B102

[ttp://www.fredhutch.org/content/dam/public/email-signatures/3/fred_hutch_logo.png]
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N., Mail Stop J4-402
Seattle, WA 98109
fredhutch.org