Re: automated comparison tool

[Andrew Clark]

On Wednesday, September 21, 2016 at 8:26:37 PM UTC-5, Steve D'Aprano wrote:
> On Thu, 22 Sep 2016 01:55 am, Andrew Clark wrote:
> 
> > I reinstalled paramiko and now i'm getting a slighty different error but
> > still says no cryptography.
> 
> [...]
> 
> > ImportError: No module named 'cryptography'
> 
> 
> You appear to be missing a dependency of paramiko. You need to identify
> which "cryptography" module it relies on, and install it. Start by reading
> the paramiko documentation and especially look for "requirements"
> or "dependencies".
> 
> 
> 
> 
> -- 
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.

I finally realized that pip was not working due to the proxy. Once i used pip 
--proxy command i was able to download the packages no issue. Everything works 
now.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Steve D'Aprano]

On Thu, 22 Sep 2016 01:55 am, Andrew Clark wrote:

> I reinstalled paramiko and now i'm getting a slighty different error but
> still says no cryptography.

[...]

> ImportError: No module named 'cryptography'


You appear to be missing a dependency of paramiko. You need to identify
which "cryptography" module it relies on, and install it. Start by reading
the paramiko documentation and especially look for "requirements"
or "dependencies".




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Kalos Kalyre]

On 2016-09-21 08:55, Andrew Clark wrote:
> I reinstalled paramiko and now i'm getting a slighty different error but 
> still says no cryptography.
> 
> Traceback (most recent call last):
>   File "C:\Users\ac40935\workspace\AutoCompare\filecmp.py", line 6, in 
> 
> from paramiko import SSHConfig, SSHClient
>   File 
> "C:\Python35-32\lib\site-packages\paramiko-2.0.2-py3.5.egg\paramiko\__init__.py",
>  line 30, in 
>   File 
> "C:\Python35-32\lib\site-packages\paramiko-2.0.2-py3.5.egg\paramiko\transport.py",
>  line 32, in 
> ImportError: No module named 'cryptography'

Have you tried installing the cryptography module? I just did a quick
web search for "python cryptography module" and found this site:

https://cryptography.io/en/latest/

which says the module can be installed with pip:

pip install cryptography

-- 
Microblog: https://identi.ca/jdea
Blog: https://theopensource.ninja/



  Cut away the five (lower fetters), abandon the five (remaining fetters),
  and then develop the five (faculties). The bhikkhu who has transcended
  the five fetters is said to be "crossed over the flood". 370

>From the Dhammapada

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Andrew Clark]

I reinstalled paramiko and now i'm getting a slighty different error but still 
says no cryptography.

Traceback (most recent call last):
  File "C:\Users\ac40935\workspace\AutoCompare\filecmp.py", line 6, in 
from paramiko import SSHConfig, SSHClient
  File 
"C:\Python35-32\lib\site-packages\paramiko-2.0.2-py3.5.egg\paramiko\__init__.py",
 line 30, in 
  File 
"C:\Python35-32\lib\site-packages\paramiko-2.0.2-py3.5.egg\paramiko\transport.py",
 line 32, in 
ImportError: No module named 'cryptography'
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Andrew Clark]

On Tuesday, September 20, 2016 at 7:48:20 PM UTC-5, Steve D'Aprano wrote:
> On Wed, 21 Sep 2016 07:20 am, Andrew Clark wrote:
> 
> > I've restarted my code so many times i no longer have a working version of
> > anything.
> 
> 
> *Restarting* your code doesn't change it and cannot possibly stop it from
> working. My guess is that you actually mean that you've made so many random
> edits to your code, without understanding or thinking about what you are
> doing, that you've broken it irreversibly.
> 
> I think you've learned a few things:
> 
> (1) Debugging by making random perturbations to code should be a last
> resort, and even then, only done with the greatest of care.
> 
> (2) Use version control so you can roll back changes.
> 
> (3) Or at least make a backup copy of a known working version of your code.
> 
> (4) Most importantly, never make so many changes to your ONLY copy of the
> code in one go that you can break it so badly that you can't go back.
> 
> 
> You've described your problem quite well, and nicely broken it up into
> pieces. I suggest you take each piece, and try to write the code for it
> independently of the others. Start with the first piece:
> 
> "access remote directories"
> 
> 
> Okay, how are they accessible? Are they just network drives? Then that's
> simple: you can access them as if they were local directories. What's the
> path name of the network drive(s)? Simply use that. Problem solved.
> 
> If not, then you need to decide how to access them: over SSH, FTP,
> sneaker-net, whatever. The choice you make here is going to impact the
> complexity of your code. Think about carefully.
> 
> Once you have working code that can list the remote directory, you can use
> it to list your three sets of files:
> 
> 
> startupfiles = listdir(...StartupConfig)
> runningfiles = listdir(...RunningConfig)
> archivefiles = listdir(...ArchiveConfig)
> 
> 
> now you can move onto step 2:
> 
> "run through startup, running and archive to find files 
>  with same hostname(do this for all files)"
> 
> 
> Now you can forget all about remote file systems (at least for now) and just
> work with the three lists of file names. How do you decide which files
> belong to which file name? I don't know, because you don't say. Is the
> hostname in the file name? Or do you have to open the file and read the
> information from the file contents?
> 
> However you do it, start by generating a mapping of hostname: list of file
> names.
> 
> 
> mapping = {}
> for filename in list_of_filenames:
> hostname = get_hostname(filename)
> if hostname in mapping:
> mapping[hostname].append(filename)
> else:
> mapping[hostname] = [filename]  # start a new list with one entry
> 
> 
> 
> Once you've done that for all three directories, then you can collect all
> the file names for each host from all three directories, and compare the
> files.

> -- 
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.

Thank you all for the comments. I've been working on the part to remote into 
the server. Unfortunately it appears that the drives are network accessible and 
i have to connect to them via SFTP to retrieve the files. I've been working on 
the below code to do this. However i keep getting an error.

import os
import sys
from contextlib import closing
from paramiko import SSHConfig, SSHClient

# Specify hostname to connect to and the path
hostname, remote_dirname, destdir = sys.argv[1:]

# Load parameters to setup ssh connection
config = SSHConfig()
with open(os.path.expanduser('~/.ssh/config')) as config_file:
config.parse(config_file)
d = config.lookup(hostname)

# Connect
with closing(SSHClient()) as ssh:
ssh.load_system_host_keys() #NOTE: no AutoAddPolicy()
ssh.connect(d['hostname'], username=d.get('user'))
with closing(ssh.open_sftp()) as sftp:
# CD into remote directory
sftp.chdir(remote_dirname)
# CD to local destination directory
os.chdir(destdir)
# Download all files in it to destdir directory
for filename in sftp.listdir():
sftp.get(filename, filename)

Traceback (most recent call last):
  File "C:\Users\ac40935\workspace\AutoCompare\filecmp.py", line 6, in 
from paramiko import SSHConfig, SSHClient
  File "C:\Python35-32\paramiko-master\paramiko\__init__.py", line 30, in 

from paramiko.transport import SecurityOptions, Transport
  File "C:\Python35-32\paramiko-master\paramiko\transport.py", line 32, in 

from cryptography.hazmat.backends import default_backend
ImportError: No module named 'cryptography'
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Lawrence D’Oliveiro]

On Wednesday, September 21, 2016 at 12:14:54 PM UTC+12, Bill Deegan wrote:
> Use Git... at least you get can back what used to work..

That has become such a matter of programming habit, taken so much for granted 
that, to me, it’s like saying “don’t forget to breathe”. :)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Steve D'Aprano]

On Wed, 21 Sep 2016 07:20 am, Andrew Clark wrote:

> I've restarted my code so many times i no longer have a working version of
> anything.


*Restarting* your code doesn't change it and cannot possibly stop it from
working. My guess is that you actually mean that you've made so many random
edits to your code, without understanding or thinking about what you are
doing, that you've broken it irreversibly.

I think you've learned a few things:

(1) Debugging by making random perturbations to code should be a last
resort, and even then, only done with the greatest of care.

(2) Use version control so you can roll back changes.

(3) Or at least make a backup copy of a known working version of your code.

(4) Most importantly, never make so many changes to your ONLY copy of the
code in one go that you can break it so badly that you can't go back.


You've described your problem quite well, and nicely broken it up into
pieces. I suggest you take each piece, and try to write the code for it
independently of the others. Start with the first piece:

"access remote directories"


Okay, how are they accessible? Are they just network drives? Then that's
simple: you can access them as if they were local directories. What's the
path name of the network drive(s)? Simply use that. Problem solved.

If not, then you need to decide how to access them: over SSH, FTP,
sneaker-net, whatever. The choice you make here is going to impact the
complexity of your code. Think about carefully.

Once you have working code that can list the remote directory, you can use
it to list your three sets of files:


startupfiles = listdir(...StartupConfig)
runningfiles = listdir(...RunningConfig)
archivefiles = listdir(...ArchiveConfig)


now you can move onto step 2:

"run through startup, running and archive to find files 
 with same hostname(do this for all files)"


Now you can forget all about remote file systems (at least for now) and just
work with the three lists of file names. How do you decide which files
belong to which file name? I don't know, because you don't say. Is the
hostname in the file name? Or do you have to open the file and read the
information from the file contents?

However you do it, start by generating a mapping of hostname: list of file
names.


mapping = {}
for filename in list_of_filenames:
hostname = get_hostname(filename)
if hostname in mapping:
mapping[hostname].append(filename)
else:
mapping[hostname] = [filename]  # start a new list with one entry



Once you've done that for all three directories, then you can collect all
the file names for each host from all three directories, and compare the
files.


I leave the rest of the exercise to you, but remember the principle of
Divide And Conquer. Divide the problem into smaller problems which are easy
to solve, solve each small problem, and the big problem solves itself.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Bill Deegan]

Use Git... at least you get can back what used to work..

On Tue, Sep 20, 2016 at 2:20 PM, Andrew Clark  wrote:

> On Tuesday, September 20, 2016 at 3:25:11 PM UTC-5, Lawrence D’Oliveiro
> wrote:
> > On Wednesday, September 21, 2016 at 7:44:22 AM UTC+12, Andrew Clark
> wrote:
> > > If anyone can help me out with sudo code or set me in the right
> direction
> > > would be nice.
> >
> > You know What Aesop said: “the gods* help those who help themselves”.
> Start by posting an initial stab at your code for solving, if not the whole
> problem, at least part of it.
> >
> > *for “gods”, read “peanut gallery”. They’ll be falling over themselves
> to critique every little shortcoming, real or otherwise, in the code you
> post.
>
> Thanks. I've restarted my code so many times i no longer have a working
> version of anything. I'm trying to get the "go to remote directory" part to
> get files done first.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Andrew Clark]

On Tuesday, September 20, 2016 at 3:25:11 PM UTC-5, Lawrence D’Oliveiro wrote:
> On Wednesday, September 21, 2016 at 7:44:22 AM UTC+12, Andrew Clark wrote:
> > If anyone can help me out with sudo code or set me in the right direction
> > would be nice.
> 
> You know What Aesop said: “the gods* help those who help themselves”. Start 
> by posting an initial stab at your code for solving, if not the whole 
> problem, at least part of it.
> 
> *for “gods”, read “peanut gallery”. They’ll be falling over themselves to 
> critique every little shortcoming, real or otherwise, in the code you post.

Thanks. I've restarted my code so many times i no longer have a working version 
of anything. I'm trying to get the "go to remote directory" part to get files 
done first.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: automated comparison tool

[Lawrence D’Oliveiro]

On Wednesday, September 21, 2016 at 7:44:22 AM UTC+12, Andrew Clark wrote:
> If anyone can help me out with sudo code or set me in the right direction
> would be nice.

You know What Aesop said: “the gods* help those who help themselves”. Start by 
posting an initial stab at your code for solving, if not the whole problem, at 
least part of it.

*for “gods”, read “peanut gallery”. They’ll be falling over themselves to 
critique every little shortcoming, real or otherwise, in the code you post.

-- 
https://mail.python.org/mailman/listinfo/python-list