Re: [Tutor] Moving a conda environment to an off-line computer

[Matt Ruffalo]

Hi Henrique-

It is quite easy to transfer an Anaconda installation from one machine
to the other by copying all of the files -- I have done this repeatedly
with cluster compute environments. It is sometimes nicer to run `conda
upgrade --all` in a local VM and then `rsync` the updated Anaconda
installation between machines, since (as you mentioned) internet access
can sometimes be an issue.

It looks like you did everything correctly, and everything is "working"
as well as you would expect. As Alan mentioned, though, it looks like
the 'deepchem' package is trying to access the internet to load one of
its data sets, and this is what is failing. You could perhaps download
that data set and put it somewhere on the cluster where deepchem would
know where to look for it, to avoid having to download it, but I am
completely unfamiliar with deepchem so I can't offer any advice about
how to do that.

MMR...

On 30/11/18 08:47, Henrique Castro wrote:
> Dear colleagues,
> Soon I'll start to use one of the powerful computers on my university as a 
> tool in my Ph.D. The computer does not have an internet connection and I need 
> to find a way to install a conda environment on it.
> At first I tried to install and set the conda environment that I need in a 
> computer with internet connection and taking care to copy everything in a 
> path that is similar in the off-line computer (i.e I installed everything on 
> /home/henrique/bin/anaconda3 at home and tried to copy everything to 
> /home/henrique/bin/anaconda3 in the off-line computer - with the same 
> .bashrc) but when I run conda I get an error(it works on my home computer):
>
> (deepchem) [henrique@europio qm7] $ python qm7_ANI.py
> /home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29:
>  DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and 
> should not be imported. It will be removed in a future NumPy release.
>   from numpy.core.umath_tests import inner1d
> Traceback (most recent call last):
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py", 
> line 1318, in do_open
> encode_chunked=req.has_header('Transfer-encoding'))
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", 
> line 1239, in request
> self._send_request(method, url, body, headers, encode_chunked)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", 
> line 1285, in _send_request
> self.endheaders(body, encode_chunked=encode_chunked)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", 
> line 1234, in endheaders
> self._send_output(message_body, encode_chunked=encode_chunked)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", 
> line 1026, in _send_output
> self.send(msg)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", 
> line 964, in send
> self.connect()
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", 
> line 936, in connect
> (self.host,self.port), self.timeout, self.source_address)
>   File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py", 
> line 704, in create_connection
> for res in getaddrinfo(host, port, 0, SOCK_STREAM):
>   File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py", 
> line 745, in getaddrinfo
> for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
> socket.gaierror: [Errno -2] Name or service not known
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>   File "qm7_ANI.py", line 15, in 
> featurizer='BPSymmetryFunction', split='stratified', move_mean=False)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/deepchem/molnet/load_function/qm7_datasets.py",
>  line 50, in load_qm7_from_mat
> 'http://deepchem.io.s3-website-us-west-1.amazonaws.com/datasets/qm7.mat'
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/deepchem/utils/__init__.py",
>  line 85, in download_url
> urlretrieve(url, os.path.join(dest_dir, name))
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py", 
> line 248, in urlretrieve
> with contextlib.closing(urlopen(url, data)) as fp:
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py", 
> line 223, in urlopen
> return opener.open(url, data, timeout)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py", 
> line 526, in open
> response = self._open(req, data)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py", 
> line 544, in _open
> '_open', req)
>   File 
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py", 
> line 

[Tutor] Wrap a pipe to get text mode

[Avi Gross]

Brief(er) message on topic:

 

Someone was having a problem that was traced to their use of a pipe to
communicate between tasks that normally passes data in binary mode.

 

They were offered solutions on how to convert the data that should work
fine. I am offering a different solution from the Lutz Book I am currently
reading:

 

Programming Python: Powerful Object-Oriented Programming

By Mark Lutz

 

This is the snippet on-line:

 

https://books.google.com/books?id=q8W3WQbNWmkC

=PA226=PA226=python+wrap+a+pipe=bl=Y8ddhRkA2E=NW2Yy
gRxI9qUtJjMQx77Xhwfy88=en=X=2ahUKEwj-sZygzv_eAhUCnFkKHbbxBu8Q6AEwC
noECAQQAQ#v=onepage=python%20wrap%20a%20pipe=false

 

The key is to take one end of the pipe and wrap a file around it that then
provides the interface that processes and returns raw data as text:

 

In the parent, take the side of the pipe you read from, called pipein as in:

 

pipe, pipeout = os.pipe

 

and do this:

 

pipein = os.fdopen(pipein)

 

NOTE that their method does not use Popen() to sun a command. Your child
process would need to start that program on their own. But the text received
would not require further processing from binary.

 

Many other solutions exist, such as on some systems using named pipes which
can be opened in text mode. Even simpler is to run the external process with
an argument like ">file" and have python open that file in text mode after
it completes. These solutions do require having permission to create files
and perhaps require additional effort to create a unique temp file.

 

 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] subprocess.Popen()

[Dave Hill]

I am a volunteer at a Heritage Railway in N.Wales and, amongst other 
things, I provide electronics and software for various exhibits in the 
museum. I use the Raspberry Pi to provide various video presentations, 
employing the omxplayer.


I am in the process of updating an application known as the 'Runaway 
Train', originally written some 4ish years ago, as it has started to 
exhibit intermittent faults. This involves playing a video (at 8 times 
normal speed), via two TV screens, of a narrow gauge train ride from the 
view of the locomotive driver, on request, with the visitor located in a 
replica footplate, in which the floor vibrates and smoke emanates.


As I learn more Python I wanted to improve the interaction with 
omxplayer. In the original application I called omxplayer using Popen


__

    omxp = Popen(['omxplayer', MOVIE_PATH])

followed by

    # wait for video process to finish
    omxp.wait()

__

Having 'graduated' to Python 3.7, I thought I would explore 
subprocess.Popen, and put the code in a Class, see code below. The video 
runs, but an error occurs, which I do not understand, see further below


__

# Import subprocess
import subprocess

from io import StringIO

class Player:

    def __init__(self, path, timeout ):

    self.path = path
    self.timeout = timeout

    def playVideo(self, filename, audio):

    MOVIE_PATH = self.path + filename

    if ( audio == "HDMI" ):
    omxp = subprocess.Popen(['omxplayer', '-s', '-o', 'hdmi',\
MOVIE_PATH],stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
    else:
    # Call omxplayer - audio to Analog Port
    omxp = subprocess.Popen(['omxplayer', '-s', '-o', 'local',\
MOVIE_PATH],stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)

    try:
    out = omxp.communicate(timeout=self.timeout)
    print("Try outs =  ", StringIO(out))
    except:
    omxp.kill()
    out = omxp.communicate()
    print("Except outs =  ", StringIO(out))

__

Traceback (most recent call last):
  File "/home/pi/Code/TestVideo#4.py", line 31, in 
player.playVideo(FILE, 'HDMI')
  File "/home/pi/Code/VideoPlayer.py", line 51, in playVideo
out = omxp.communicate()
  File "/usr/lib/python3.5/subprocess.py", line 801, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.5/subprocess.py", line 1437, in _communicate
selector.register(self.stdout, selectors.EVENT_READ)
  File "/usr/lib/python3.5/selectors.py", line 351, in register
key = super().register(fileobj, events, data)
  File "/usr/lib/python3.5/selectors.py", line 237, in register
key = SelectorKey(fileobj, self._fileobj_lookup(fileobj), events, data)
  File "/usr/lib/python3.5/selectors.py", line 224, in _fileobj_lookup
return _fileobj_to_fd(fileobj)
  File "/usr/lib/python3.5/selectors.py", line 39, in _fileobj_to_fd
"{!r}".format(fileobj)) from None
ValueError: Invalid file object: <_io.BufferedReader name=8>

__

From previous discussions, where do I sit in the demographics! I 
graduated from a degree, that was 95% physics with a small amount of 
electronics, spent 30 years as essentially an electronics engineer, 
during which time I wrote software using Fortran, Basic, Pascal (HP) and 
C++, typically for embedded applications. I have been retired for 10 
years, and occasionally play trains.


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor