Re: Refactor/Rewrite Perl code in Python
Thanks everyone for the insight. I got the idea as to how and where to start. Guess I need to work in Perl for now, so as to start the conversion process. Regarding Tests, I had already started writing tests before posting. Writing tests for every module will be a pain as well as a nice experience. Thanks. ~Shashwat -- http://mail.python.org/mailman/listinfo/python-list
Re: Refactor/Rewrite Perl code in Python
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
On Sun, Jul 24, 2011 at 7:29 PM, Shashwat Anand anand.shash...@gmail.com
wrote:
How do I start ?
The idea is to rewrite module by module.
But how to make sure code doesn't break ?
By testing it.
Read up on test driven development.
At this point, you have this:
Perl modules: A, B, C, D
Python modules: none
Python tests: none
Now, before you rewrite each Perl module in Python, first write a good,
comprehension test suite in Python for that module. You need to have tests
for each function and method. Test that they do the right thing for both
good data and bad data. If you have functional requirements for the Perl
modules, use that as your reference, otherwise use the Perl code as the
reference.
For example, this might be a basic test suite for the len() built-in
function:
for empty in ([], {}, (), set([]), ):
if len(empty) != 0:
raise AssertionError('empty object gives non-zero len')
for n in range(1, 5):
if len(x*n) != n:
raise AssertionError('failure for string')
for kind in (list, tuple, set):
obj = kind([None]*n)
if len(obj) != n:
raise AssertionError('failure for %s' % obj)
if len({'a': 1, 'b': None, 42: 'spam'}) != 3:
raise AssertionError('failure for dict')
for bad_obj in (23, None, object(), 165.0, True):
try:
len(bad_obj)
except TypeError:
# Test passes!
pass
else:
# No exception means len() fails!
raise AssertionError('failed test')
Multiply that by *every* function and method in the module, and you have a
moderately good test suite for module A.
(You may want to learn about the unittest module, which will help.)
Now you have:
Perl modules: A, B, C, D
Python modules: none
Python tests: test_A
Now re-write the Python module, and test it against the test suite. If it
fails, fix the failures. Repeat until it passes, and you have:
Perl modules: A, B, C, D
Python modules: A
Python tests: test_A
Now you can be confident that Python A does everything that Perl A does.
Possibly *better* than the Perl module, since if you don't have a test
suite for it, it probably has many hidden bugs.
Continue in this way with the rest of the modules. At the end, you will have
a full test suite against the entire collection of modules.
A very sane approach.
I currently support a large legacy application written in C++ by
extending it with Python. We managed to do this by wrapping the legacy
application with a Python interface using the Boost::Python libraries.
It has allowed us to embrace and extend the legacy code and replace bits
of it one piece at a time while keeping the whole application running.
Now I am not aware of any Python bindings to the Perl interpreter, but
if there is some FFI library that makes it possible this approach might
be viable for your project.
The trade-off of this approach is the extra complexity of maintaining
another interface in your code. The beneift is that you can avoid
re-writing a large amount of code up front. This works particularly
well when the legacy system has a rather large user base and you don't
have a robust test suite for the legacy code.
One book I found particularly useful:
http://www.amazon.ca/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052
hth
--
http://mail.python.org/mailman/listinfo/python-list
RE: Refactor/Rewrite Perl code in Python
Sometimes it's worth asking Why? I assume there would be no need to rewrite if the existing code did most of what was needed. It may be easier to ask the customer what he really wants rather than to re-engineer a crappy solution to an obsolete problem. -- http://mail.python.org/mailman/listinfo/python-list
Refactor/Rewrite Perl code in Python
I am working with a huge codebase of Perl. The code have zero documentation and zero unit-tests. It seems like a huge hack. The underlying database schema is horrid. So I want to rewrite the whole of it in Python. How do I start ? The idea is to rewrite module by module. But how to make sure code doesn't break ? How can I import perl and python codes in each other ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Refactor/Rewrite Perl code in Python
On Sun, Jul 24, 2011 at 7:29 PM, Shashwat Anand anand.shash...@gmail.com wrote: How do I start ? The idea is to rewrite module by module. But how to make sure code doesn't break ? How can I import perl and python codes in each other ? Can you separate the project into separate executables that call on each other? You can pipe text from stdout of perl to stdin of python, for instance. Otherwise, it's not going to be easy. But if you're going to change the underlying database AND the code at the same time, it may be best to simply set aside the old code completely and code a brand new system in Python, capitalizing on the Second Mouse Effect (the early bird gets the worm, but the second mouse gets the cheese). You can learn from all the mistakes made in the first version, allowing you to make an entirely new set of mistakes. :) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Refactor/Rewrite Perl code in Python
On Sun, Jul 24, 2011 at 7:29 PM, Shashwat Anand anand.shash...@gmail.com
wrote:
How do I start ?
The idea is to rewrite module by module.
But how to make sure code doesn't break ?
By testing it.
Read up on test driven development.
At this point, you have this:
Perl modules: A, B, C, D
Python modules: none
Python tests: none
Now, before you rewrite each Perl module in Python, first write a good,
comprehension test suite in Python for that module. You need to have tests
for each function and method. Test that they do the right thing for both
good data and bad data. If you have functional requirements for the Perl
modules, use that as your reference, otherwise use the Perl code as the
reference.
For example, this might be a basic test suite for the len() built-in
function:
for empty in ([], {}, (), set([]), ):
if len(empty) != 0:
raise AssertionError('empty object gives non-zero len')
for n in range(1, 5):
if len(x*n) != n:
raise AssertionError('failure for string')
for kind in (list, tuple, set):
obj = kind([None]*n)
if len(obj) != n:
raise AssertionError('failure for %s' % obj)
if len({'a': 1, 'b': None, 42: 'spam'}) != 3:
raise AssertionError('failure for dict')
for bad_obj in (23, None, object(), 165.0, True):
try:
len(bad_obj)
except TypeError:
# Test passes!
pass
else:
# No exception means len() fails!
raise AssertionError('failed test')
Multiply that by *every* function and method in the module, and you have a
moderately good test suite for module A.
(You may want to learn about the unittest module, which will help.)
Now you have:
Perl modules: A, B, C, D
Python modules: none
Python tests: test_A
Now re-write the Python module, and test it against the test suite. If it
fails, fix the failures. Repeat until it passes, and you have:
Perl modules: A, B, C, D
Python modules: A
Python tests: test_A
Now you can be confident that Python A does everything that Perl A does.
Possibly *better* than the Perl module, since if you don't have a test
suite for it, it probably has many hidden bugs.
Continue in this way with the rest of the modules. At the end, you will have
a full test suite against the entire collection of modules.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Refactor/Rewrite Perl code in Python
On Sun, Jul 24, 2011 at 2:29 AM, Shashwat Anand anand.shash...@gmail.comwrote: I am working with a huge codebase of Perl. The code have zero documentation and zero unit-tests. It seems like a huge hack. My condolences. Er, actually, it sounds kind of fun. The underlying database schema is horrid. So I want to rewrite the whole of it in Python. Good choice. How do I start ? You're probably going to need to learn a lot about the perl code - it's probably a good idea to come up with unit, integration and system tests for it - in perl; document it; and diagram it using something like autodia. Then after you know what's going on inside and outside the perl, you're better equipped to rewrite into python. If you write your automated perl tests in a straightforward way, they can be mostly straightforwardly translated to python - to get an apples to apples comparison of how each is functioning. If you do this module by module, you're less likely to run into problems, but you're also more likely to preserve some of the original code's oddities. The idea is to rewrite module by module. But how to make sure code doesn't break ? See above. How can I import perl and python codes in each other ? I'm not confident you can, unless perhaps you find that you can use an alternative VM like Parrot. However, many languages on Parrot are not complete. -- http://mail.python.org/mailman/listinfo/python-list
