execnet-1.0.7: new env-setting and keyboardinterrupt fixes
execnet-1.0.7 is a backward compatible release avoiding a gateway termination race condition and adding the possibility to add env:NAME=value settings to gateway specifications, thanks to thanks Jakub Gustak. execnet is a small and stable pure-python library for automatically deploying and interacting with clusters of Python interpreters. It seamlessly instantiates remote interpreters through the 'ssh' command line tool or socket connections. It supports interactions between Python 2.4 through to 3.1, Jython-2.5.1 and pypy-c, therefore enabling generic Python-to-Python bridging. execnet-1.0.7 was successfully tested against Python2.7 as well as the usual armada of interpreters ranging from Python2.4 to Python3.1, Jython and PyPy. See http://codespeak.net/execnet for extensive documentation. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ANN: ActivePython 2.7.0.1 is now available
We are pleased to announce the availability of ActivePython 2.7.0.1. http://www.activestate.com/activepython This release corresponds to the recently released Python 2.7, and, like ActivePython 2.6, includes the Python Package Manager (PyPM) with essential packages such as Distribute (a compatible fork of setuptools), virtualenv, pip and SQLAlchemy. See the release notes for full details: http://docs.activestate.com/activepython/2.7/relnotes.html#changes For a high-level overview of this release, please see our blog post: http://www.activestate.com/blog/2010/07/activepython-27-released This is also the first ActivePython release with 64-bit support on MacOSX. What is ActivePython? - ActivePython is ActiveState's binary distribution of Python. Builds for Windows, Mac OS X, Linux are made freely available. Solaris, HP-UX and AIX builds, and access to older versions are available in ActivePython Business, Enterprise and OEM editions: http://www.activestate.com/python ActivePython includes the Python core and the many core extensions: zlib and bzip2 for data compression, the Berkeley DB (bsddb) and SQLite (sqlite3) database libraries, OpenSSL bindings for HTTPS support, the Tix GUI widgets for Tkinter, ElementTree for XML processing, ctypes (on supported platforms) for low-level library access, and others. The Windows distribution ships with PyWin32 -- a suite of Windows tools developed by Mark Hammond, including bindings to the Win32 API and Windows COM. ActivePython 2.6 and 2.7 also include a binary package manager for Python (PyPM) that can be used to install packages much easily. For example: C:\pypm install mysql-python [...] C:\python import MySQLdb See this page for full details: http://docs.activestate.com/activepython/2.7/whatsincluded.html As well, ActivePython ships with a wealth of documentation for both new and experienced Python programmers. In addition to the core Python docs, ActivePython includes the What's New in Python series, Dive into Python, the Python FAQs HOWTOs, and the Python Enhancement Proposals (PEPs). An online version of the docs can be found here: http://docs.activestate.com/activepython/2.7/ We would welcome any and all feedback to: activepython-feedb...@activestate.com Please file bugs against ActivePython at: http://bugs.activestate.com/enter_bug.cgi?product=ActivePython On what platforms does ActivePython run? ActivePython includes installers for the following platforms: - Windows/x86 - Windows/x64 (aka AMD64) - Mac OS X - Linux/x86 - Linux/x86_64 (aka AMD64) - Solaris/SPARC (Business, Enterprise or OEM edition only) - Solaris/x86 (Business, Enterprise or OEM edition only) - HP-UX/PA-RISC (Business, Enterprise or OEM edition only) - HP-UX/IA-64 (Enterprise or OEM edition only) - AIX/PowerPC (Business, Enterprise or OEM edition only) - AIX/PowerPC 64-bit (Business, Enterprise or OEM edition only) Custom builds are available in Enterprise Edition: http://www.activestate.com/enterprise-edition Thanks, and enjoy! The Python Team -- Sridhar Ratnakumar sridharr at activestate.com -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: The real problem with Python 3 - no business case for conversion (was I strongly dislike Python 3)
I just couldn't get through on the python-dev list that I couldn't just upgrade my code to 2.6 and then use 2to3 to keep in step across the 2-3 chasm, as this would leave behind my faithful pre-2.6 users. Not sure whom you had been talking to. But I would have tried to explain that you don't *have* to port to 2.6 to use 2to3 - it will work just as fine with 2.3 code, and earlier. - No use of sets. Instead I defined a very simple set simulation using dict keys, which could be interchanged with set for later versions. This I don't understand. IIUC, you want to support 2.3 and later. Now, 2.3 already has a set module, even though set is not a builtin. So you could use sets just as well. - No generator expressions, only list comprehensions. Ok. Not sure how this causes problems, though - just don't use them, then. - No use of decorators. BUT, pyparsing includes a decorator method, traceParseAction, which can be used by users with later Pythons as @traceParseAction in their own code. Of course, users of older Python versions could explicitly wrap the functions with the decorator if they wanted to. - No print statements. As pyparsing is intended to be an internal module, it does no I/O as part of its function - it only processes a given string, and returns a data structure. If you don't need them, fine. If you do, I'd just let 2to3 transform them. - Python 2-3 compatible exception syntax. This may have been my trickiest step. The change of syntax for except from except ExceptionType, ex: to: except ExceptionType as ex: is completely forward and backward incompatible. The workaround is to rewrite as: except ExceptionType: ex = sys.exc_info()[0] Likewise, and more importantly so: use 2to3. It can be done this way, but I find the 2to3 solution much cleaner. But in the meantime, I am still able to support all versions of Python NOW, and I plan to continue doing so (albeit support for 2.x versions will eventually mean continue to offer a frozen feature set, with minimal bug-fixing if any). The same would have been possible if you had chosen to use 2to3. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I add method dynamically to module using C API?
* Martin v. Loewis, on 08.07.2010 07:23:
And since things work for a single method when I declare 'def' as
'static', I suspect that means that the function object created by
PyCFunction_NewEx holds on to a pointer to the PyMethodDef structure?
Correct; it doesn't make a copy of the struct. So when you want the
function object to outlive the setRoutine call, you need to allocate
the PyMethodDef on the heap.
Thanks!
That's the direction I tentatively had started to investigate.
But second problem now is cleanup: I'd like to deallocate when the module is
freed. I tried (1) adding a __del__, but no dice, I guess because it wasn't
really an object method but just a free function in a module; and (2) the m_free
callback in the module definition structure, but it was not called.
Perhaps I don't need to to clean up?
Anyway, current looks like this:
code
// progrock.cppy -- C++ plus Python
// A simple C++ framework for writing Python 3.x extensions.
//
// Copyright (C) Alf P. Steinbach, 2010.
#ifndef CPPY_MODULE_H
#define CPPY_MODULE_H
#include progrock/cppx/devsupport/better_experience.h
//- Dependencies:
#include Ptr.h
#include progrock/cppx/exception/throwing.h
#include list
//- Interface:
namespace progrock{ namespace cppy {
namespace detail {
inline PyModuleDef* moduleDefPtr()
{
static PyMethodDef methodDefs[] = {
//...
//{ system, pyni_system, METH_VARARGS, Execute a shell
command. },
//{ __del__, onModuleDestroy, METH_NOARGS, Destructor },
//...
{ NULL, NULL, 0, NULL } // Sentinel
};
static PyModuleDef moduleDef = {
PyModuleDef_HEAD_INIT,
cppy, // name of module
NULL, // m_doc, // module documentation in UTF-8
sizeof(void*), // size of per-interpreter state of the
module,
//or -1 if the module keeps state in global
variables.
methodDefs,
NULL, // m_reload
NULL, // m_traverse
NULL, // m_clear
NULL// m_free
};
return moduleDef;
}
}
class Module
{
private:
struct InstanceData
{
std::list PyMethodDef methodDefs;
};
void* instanceMemory() const
{
void* const result = PyModule_GetState( p_.get() );
assert( result != 0 );
return result;
}
InstanceData* instanceDataPtr() const
{
return *reinterpret_cast InstanceData** ( instanceMemory() );
}
Ptr p_;
InstanceData* data_;
public:
Module()
: p_( ::PyModule_Create( detail::moduleDefPtr() ) )
{
assert( def.m_size == sizeof( void* ) );
(p_.get() != 0) || cppx::throwX( Module::init: failed );
instanceDataPtr() = data_ = new InstanceData();
}
PyObject* rawPtr() const{ return p_.get(); }
PyObject* release() { return p_.release(); }
void setDocString( wchar_t const s[] )
{
Ptr const v = ::PyUnicode_FromWideChar( s, -1 );
(v.get() != 0)
|| cppx::throwX( Module::setDocString: PyUnicode_FromWideChar
failed );
::PyObject_SetAttrString( p_.get(), __doc__, v.get() )
cppx::is( cppx::notMinusOne )
|| cppx::throwX( Module::setDocString: PyObject_SetAttrString
failed );
}
void addRoutine( char const name[], PyCFunction f, char const doc[] =
)
{
PyMethodDef const defData = { name, f, METH_VARARGS, doc };
data_-methodDefs.push_back( defData );
try
{
PyMethodDef*pDef= data_-methodDefs.back();
Ptr const pyName = ::PyUnicode_FromString( name );
Ptr r = ::PyCFunction_NewEx( pDef, p_.get(),
pyName.get());
::PyModule_AddObject( p_.get(), name, r.release() )
cppx::is( cppx::notMinusOne )
|| cppx::throwX( Module::addRoutine: PyModule_AddObject
failed );
}
catch( ... )
{
data_-methodDefs.pop_back();
throw;
}
}
};
} } // namespace progrock::cppy
#endif
/code
I changed the module name from pyni* to cppy... ;-)
Cheers thanks!, but how to clean up, or must I?
- Alf
--
blog at url: http://alfps.wordpress.com
--
http://mail.python.org/mailman/listinfo/python-list
Appending to a file using Python API
Hello all, How do you append to a file using Python os::file APIs. So that it appends to the content of the file. Not adding the content to the new line. But just appends next to the exiting content of the file. Example : current content of file A B C if we append D to it, it should be A B C D Not like: A B C D regards, lavanya -- http://mail.python.org/mailman/listinfo/python-list
Re: C interpreter in Lisp/scheme/python
On 7 July, 17:38, Rivka Miller rivkaumil...@gmail.com wrote: Although C comes with a regex library, C does not come with a regexp library Anyone know what the first initial of L. Peter Deutsch stand for ? Laurence according to wikipedia (search time 2s) -- http://mail.python.org/mailman/listinfo/python-list
Re: C interpreter in Lisp/scheme/python
On 8 July, 08:08, Nick Keighley nick_keighley_nos...@hotmail.com wrote: On 7 July, 17:38, Rivka Miller rivkaumil...@gmail.com wrote: Anyone know what the first initial of L. Peter Deutsch stand for ? Laurence according to wikipedia (search time 2s) oops! He was born Laurence but changed it legally to L. including the dot -- http://mail.python.org/mailman/listinfo/python-list
Re: Appending to a file using Python API
On Wed, 07 Jul 2010 23:55:46 -0700, lavanya wrote:
Hello all,
How do you append to a file using Python os::file APIs. So that it
appends to the content of the file. Not adding the content to the new
line. But just appends next to the exiting content of the file.
Example : current content of file
A B C
if we append D to it, it should be
A B C D
Not like:
A B C
D
f = open(myfile.txt, a)
f.write(D)
will append to the end of myfile.txt, regardless of what is already in
myfile.txt. If it ends with a newline:
A B C\n
then you will end up with:
A B C\nD
If there is no newline, you will end up with:
A B CD
If you need anything more complicated than that, the easiest solution is
something like this:
# read the entire file
# modify the last line in memory
# write the lines back to the file
f = open(myfile.txt, r)
lines = f.readlines()
f.close()
if not lines:
# file exists but is empty
lines.append('')
last_line = lines[-1]
last_line = last_line.rstrip() # remove all trailing whitespace
if last_line:
last_line += D\n
else:
last_line = D\n
lines[-1] = last_line
f = open(myfile.txt, w)
f.writelines(lines)
f.close()
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: How do I add method dynamically to module using C API?
I tried (1) adding a __del__, but no dice, I guess because it wasn't really an object method but just a free function in a module; and (2) the m_free callback in the module definition structure, but it was not called. m_free will be called if the module object gets deallocated. So if m_free really isn't called, the module got never deallocated. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Thu, 08 Jul 2010 06:04:33 +0200, David Cournapeau wrote: On Thu, Jul 8, 2010 at 5:41 AM, Zooko O'Whielacronx zo...@zooko.com wrote: I'm starting to think that one should use Decimals by default and reserve floats for special cases. This is somewhat analogous to the way that Python provides arbitrarily-big integers by default and Python programmers only use old-fashioned fixed-size integers for special cases, such as interoperation with external systems or highly optimized pieces (in numpy or in native extension modules, for example). I don't think it is analogous at all. Arbitrary-bit integers have a simple tradeoff: you are willing to lose performance and memory for bigger integer. If you leave performance aside, there is no downside that I know of for using big int instead of machine int. Well, sure, but that's like saying that if you leave performance aside, there's no downside to Bubblesort instead of Quicksort. However, I believe that in Python at least, the performance cost of arbitrary-sized longs is quite minimal compared to the benefit, at least for reasonable sized ints, and so the actual real-world cost of unifying the int and long types is minimal. On the other hand, until Decimal is re-written in C, it will always be *significantly* slower than float. $ python -m timeit 2.0/3.0 100 loops, best of 3: 0.139 usec per loop $ python -m timeit -s from decimal import Decimal as D D(2)/D(3) 1000 loops, best of 3: 549 usec per loop That's three orders of magnitude difference in speed. That's HUGE, and *alone* is enough to disqualify changing to Decimal as the default floating point data type. Perhaps in the future, if and when Decimal has a fast C implementation, this can be re-thought. Since you are using python, you already bought this kind of tradeoff anyway. Decimal vs float is a different matter altogether: decimal has downsides compared to float. First, there is this irreconcilable fact that no matter how small your range is, it is impossible to represent exactly all (even most) numbers exactly with finite memory - float and decimal are two different solutions to this issue, with different tradeoffs. Yes, but how is this a downside *compared* to float? In what way does Decimal have downsides that float doesn't? Neither can represent arbitrary real numbers exactly, but if anything float is *worse* compared to Decimal for two reasons: * Python floats are fixed to a single number of bits, while the size of Decimals can be configured by the user; * floats can represent sums of powers of two exactly, while Decimals can represent sums of powers of ten exactly. Not only does that mean that any number exactly representable as a float can also be exactly represented as a Decimal, but Decimals can *additionally* represent exactly many numbers of human interest that floats cannot. Decimal are more intuitive than float for numbers that can be represented as decimal - but most numbers cannot be represented as (finite) decimal. True, but any number that can't be, also can't be exactly represented as a float either, so how does using float help? [...] And most of the time (in my experience) the inputs and outputs to your system and the literals in your code are actually decimal, so converting them to float immediately introduces a lossy data conversion before you've even done any computation. Decimal doesn't have that problem. That's not true anymore once you start doing any computation, if by decimal you mean finite decimal. And that will be never true once you start using non trivial computation (i.e. transcendental functions like log, exp, etc...). But none of those complications are *inputs*, and, again, floats suffer from exactly the same problem. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: The real problem with Python 3 - no business case for conversion
Martin v. Loewis mar...@v.loewis.de writes: The point, one more time with feeling, is that the incompatibilities between 2.x and 3.x will *increase* over time. I think this is unfounded, and actually false. Since many other people have responded with similar sentiments, I can only think I must have been expressing myself very poorly today. I agree with everything you say here, but it doesn't go against what I've been saying. If I cared about the issue more, I might have another crack at putting my meaning into words; but those who want to support a broad stride of Python versions are more motivated than I to figure it out, so instead I depart the field. -- \ “People's Front To Reunite Gondwanaland: Stop the Laurasian | `\ Separatist Movement!” —wiredog, http://kuro5hin.org/ | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: The real problem with Python 3 - no business case for conversion (was I strongly dislike Python 3)
On Wed, 07 Jul 2010 14:10:57 -0700, Brendan Abel wrote: The entire fact that 3.x was *designed* to be incompatible should tell you that supporting 2.x and 3.x with a single code base is a bad idea, except for the very smallest of projects. I don't see that follows at all. If the incompatibilities are designed to be mechanically translatable (e.g. x.keys() - list(x.keys()) then you can start with a single code-base and run the translator. If only there were some sort of Python program that could be used to translate code from 2 to 3... *wink* -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 released
On 08/07/2010 03:17, imageguy wrote: I, too, have multiple versions installed -- newer ones for running code I haven't upgraded; older ones for compatibility testing where needed. I just install to the default c:\pythonxy directories (although I like the idea of a common root) and I put NTFS hardlinks into my general c:\tools directory which is on the path. The out-of-context hardlinks work because of the registry settings which pick up the correct context for each version. Sorry to be daft here, but what do you mean by a hardlink ? A windows Shortcut ? I have just installed 2.7 and want to start upgrading some code, but alas still want to maintain some 2.5 code too. Hardlinks have always been present on NTFS, just not very widely advertised. They are a way of saying that *this* file and *that* file are actually the *same* file. (They must be on the same volume as they underlying implementation relies on pointing to the volume's master index -- the MFT). They're not copies: if one changes, the other changes. They're not shortcuts, which are a Shell (ie Desktop) mechanism, not a filesystem one I have hardlinks called python26.exe, python31.exe, etc. which point to c:\python26\python.exe, c:\python31\python.exe etc. and also a python3.exe which is another link to c:\python31\python.exe but which will move when python 3.2 is released. However, this is simply a convenience I use. It's perfectly possible to have and to use several versions of Python concurrently without this. How you do it depends on your working practice: whether you use an IDE or double-click on .py files or run from a cmd window, etc. TJG -- http://mail.python.org/mailman/listinfo/python-list
simple python deployment tool
Hi, I am writing a python package deployment tool for linux based platforms. I have tried various existing tool sets but none of them is up to the mark and they have their own issues. Initially I'll start with simple approach. 1. Find all the modules/packages and copy to lib directory. 2. Find python's *.so dependencies(system libs) and copy them to syslib 3. Copy python(executable) and libpython2.6.so.1.0 into dist directory. 4. Set up temp environment 5. Run main script using python main script.py The idea is to produce a cleaner directory structure. Neither I am creating a boot loader nor I am converting main script file into executable. It's plain vanilla stuff. Using above steps I have pulled down a package using wxPython's demo example pySketch.py. You can download the archive from here: http://rapidshare.com/files/405255400/dist.zip (Note : It's for linux users.) After extracting the contents, run the executable file using ./run. This file sets temporary environment variables and execute pySketch.py. I apologize for the archive size as all the systems libs are also included. This is eventually not working. I think I am not able to set up temporary environment properly or may be missing some other stuff. I would appreciate if some one can point out mistakes. Here is the practical approach: /lib (all python libs, files and third party modules.) /syslib (all the system libs. libX*.*, cairo, etc.) python (executable) pySketch.py run.sh Contents of run.sh. -- set LD_LIBRARY_PATH=syslib/ set PYTHONPATH=/lib python pySketch.py Cheers Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: The real problem with Python 3 - no business case for conversion (was I strongly dislike Python 3)
2010/7/8 Michele Simionato michele.simion...@gmail.com: On Jul 7, 10:55 pm, Carl Banks pavlovevide...@gmail.com wrote: On Jul 7, 1:31 am, Paul McGuire pt...@austin.rr.com wrote: I just couldn't get through on the python-dev list that I couldn't just upgrade my code to 2.6 and then use 2to3 to keep in step across the 2-3 chasm, as this would leave behind my faithful pre-2.6 users. This is a point I do not understand. My recent module plac is meant to work from Python 2.3 to Python 3.1 and to this goal I make use of 2to3 at the *client* side. Users of Python 2.X get the original code with no magic whatsoever; users of Python 3.X get the same code, but at installation time 2to3 is run by the setup.py script. The mechanism requires distribute to be installed, but I would say that having distribute is a must for Python 3.X users; Python 2.X users do not need anything, so the approach is backward compatible. I thought this was the recommended way of using 2to3 and so far is working for me. M. Simionato -- http://mail.python.org/mailman/listinfo/python-list I used the same approach (2.x default code base which gets translated by 2to3 at installation time) and I managed to do this with distutils alone by doing a little hack in setup.py. Take a look at: http://code.google.com/p/psutil/source/browse/tags/release-0.1.3/setup.py#11 Regards, --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil -- http://mail.python.org/mailman/listinfo/python-list
Re: C interpreter in Lisp/scheme/python
Nick Keighley nick_keighley_nos...@hotmail.com writes: On 8 July, 08:08, Nick Keighley nick_keighley_nos...@hotmail.com wrote: On 7 July, 17:38, Rivka Miller rivkaumil...@gmail.com wrote: Anyone know what the first initial of L. Peter Deutsch stand for ? Laurence according to wikipedia (search time 2s) oops! He was born Laurence but changed it legally to L. including the dot Too bad, Laurence is a nice name. -- __Pascal Bourguignon__ http://www.informatimago.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I add method dynamically to module using C API?
* Martin v. Loewis, on 08.07.2010 09:13:
I tried (1) adding a __del__, but no dice, I guess
because it wasn't really an object method but just a free function in a
module; and (2) the m_free callback in the module definition structure,
but it was not called.
m_free will be called if the module object gets deallocated. So if
m_free really isn't called, the module got never deallocated.
Thanks again. I don't know what I did wrong. Now it's called. :-)
But I wasted much time googling to try to find out the /responsibilities/ of the
m_free callback, and what its void* argument was. E.g., should it deallocate the
module object, and if so, via what deallocation routine? I found some info, but
even your PEP, otherwise clear, was silent about this fine point.
Finally I looked at the source code that invokes it and found that it has no
responsibilities whatsoever, just a use-as-you-wish finalization callback. Nice!
But I think that could be more clear in the docs...
Code, for those who might be interested:
code
// progrock.cppy -- C++ plus Python
// A simple C++ framework for writing Python 3.x extensions.
//
// Copyright (C) Alf P. Steinbach, 2010.
#ifndef CPPY_MODULE_H
#define CPPY_MODULE_H
#include progrock/cppx/devsupport/better_experience.h
//- Dependencies:
#include Ptr.h
#include progrock/cppx/exception/throwing.h
#include list
//- Interface:
namespace progrock{ namespace cppy {
namespace detail {
struct InstanceData
{
std::list PyMethodDef methodDefs;
};
inline void* instanceMemoryOf( PyObject* pObject )
{
void* const result = PyModule_GetState( pObject );
assert( result != 0 );
return result;
}
inline InstanceData* instanceDataPtrOf( PyObject* pObject )
{
return *reinterpret_cast InstanceData** (
instanceMemoryOf( pObject )
);
}
inline void on_freeModule( void* p )
{
PyObject* const pModule = reinterpret_cast PyObject* ( p );
InstanceData* const pData = instanceDataPtrOf( pModule );
delete pData;
printf( Deallocated!\n ); // TODO: Remove.
}
inline PyModuleDef* moduleDefPtr()
{
static PyMethodDef methodDefs[] = {
//...
//{ system, pyni_system, METH_VARARGS, Execute a shell
command. },
//{ __del__, onModuleDestroy, METH_NOARGS, Destructor },
//...
{ NULL, NULL, 0, NULL } // Sentinel
};
static PyModuleDef moduleDef = {
PyModuleDef_HEAD_INIT,
cppy, // name of module
NULL, // m_doc, // module documentation in UTF-8
sizeof(void*), // size of per-interpreter state of the
module,
//or -1 if the module keeps state in global
variables.
methodDefs,
NULL, // m_reload
NULL, // m_traverse
NULL, // m_clear
on_freeModule // m_free
};
return moduleDef;
}
}
class Module
{
private:
Ptr p_;
detail::InstanceData* data_;
detail::InstanceData* instanceDataPtr() const
{
return detail::instanceDataPtrOf( p_.get() );
}
public:
Module()
: p_( ::PyModule_Create( detail::moduleDefPtr() ) )
, data_( 0 )
{
assert( detail::moduleDefPtr()-m_size == sizeof( void* ) );
(p_.get() != 0) || cppx::throwX( Module::init: failed );
instanceDataPtr() = data_ = new detail::InstanceData();
}
PyObject* rawPtr() const{ return p_.get(); }
PyObject* release() { return p_.release(); }
void setDocString( wchar_t const s[] )
{
Ptr const v = ::PyUnicode_FromWideChar( s, -1 );
(v.get() != 0)
|| cppx::throwX( Module::setDocString: PyUnicode_FromWideChar
failed );
::PyObject_SetAttrString( p_.get(), __doc__, v.get() )
cppx::is( cppx::notMinusOne )
|| cppx::throwX( Module::setDocString: PyObject_SetAttrString
failed );
}
void addRoutine( char const name[], PyCFunction f, char const doc[] =
)
{
PyMethodDef const defData = { name, f, METH_VARARGS, doc };
data_-methodDefs.push_back( defData );
try
{
PyMethodDef* const pDef= data_-methodDefs.back();
Ptr const pyName = ::PyUnicode_FromString( name );
(pyName.get() != 0)
|| cppx::throwX(
Re: delegation pattern via descriptor
kedra marbun a écrit :
On Jul 7, 2:46 am, Bruno Desthuilliers
bdesth.quelquech...@free.quelquepart.fr wrote:
Gregory Ewing a écrit :
Bruno Desthuilliers wrote:
kedra marbun a écrit :
if we limit our discussion to py:
why __{get|set|delete}__ don't receive the 'name' 'class' from
__{getattribute|{set|del}attr}__
'name' is the name that is searched
While it would have been technically possible, I fail to imagine any use
case for this.
I think he wants to have generic descriptors that are
shared between multiple attributes, but have them do
different things based on the attribute name.
I already understood this, but thanks !-)
What I dont understand is what problem it could solve that couldn't be
solved more simply using the either _getattr__ hook or hand-coded
delegation, since such a descriptor would be so tightly coupled to the
host class that it just doesn't make sense writing a descriptor for this.
yeah, i finally can agree descriptor isn't supposed to be used as
delegation in general, it should be the job of __getattr__
however i still think passing name would open up some other
possibilities of use
Nothing prevents you to pass a name to the descriptor instance when
instanciating it, ie:
class Desc(object):
def __init__(self, name):
self.name = name
def __get__(self, inst, cls):
# ...
def __set__(self, inst, value):
# ...
class Foo(object):
bar = Desc(bar)
baaz = Desc(baaz)
Ok, this is not necessarily what you were looking for, but it's IMHO
less brittle than relying on which attribute name was looked up (which
is something the descriptor shouldn't have to care about).
btw, is there a common approach to let the interface of a class that
uses __getattr__, to include names that are delegated?
In Python 2.x, not that I know (but it may have passed under my radar).
If what you want it to automate delegation of a set of methods without
too much manual coding, you can use a custom metaclass that will add the
relevant methods to the class, based on (ie) a list (or mapping) of
methods names. But that might be a bit overkill.
class A:
def do_this(self): ...
class B:
a = A()
I don't see the point of using delegation on a class attribute. That's
typically what inheritance is for.
def do_that(self): ...
def __getattr__(self, name):
try:
return types.MethodType(getattr(self.a, name), self)
Err... Did you try the simple way ?
return getattr(self.a, name)
--
http://mail.python.org/mailman/listinfo/python-list
Howto write XML file with comments?
Hello, - I've to write a XML document including comments - the document should be formatted that it could be viewed with a text editor What is the fastest (time for realization) approach doing it in python 2.5? Any help or hints are very welcome Thanks Alexander -- http://mail.python.org/mailman/listinfo/python-list
Re: simple python deployment tool
King wrote: Hi, I am writing a python package deployment tool for linux based platforms. I have tried various existing tool sets but none of them is up to the mark and they have their own issues. Initially I'll start with simple approach. I'm sorry, but your approach is not going to work. The Right Way(tm) is to not ship any external libraries, but let the user install the dependencies with the distro's package manager. And to not try to build custom packages, but to let the package maintainers of the different distros package your application for you. 1. Find all the modules/packages and copy to lib directory. 2. Find python's *.so dependencies(system libs) and copy them to syslib That would include all the way down to libc, your archive is going to be huge, but the actual problem is, what if the libc isn't compatible with the kernel, what if the WX, GTK, X11, etc libraries aren't compatible with the running Xserver? End of the story is, you would need to package a minimal (but almost complete) Linux system into your package, which of course is not want you want. 3. Copy python(executable) and libpython2.6.so.1.0 into dist directory. 4. Set up temp environment 5. Run main script using python main script.py The idea is to produce a cleaner directory structure. Neither I am creating a boot loader nor I am converting main script file into executable. It's plain vanilla stuff. It's not vanilla stuff, but a very hard problem. In fact you are fighting against the whole system structure. -- http://mail.python.org/mailman/listinfo/python-list
Re: Howto write XML file with comments?
Alexander Eisenhuth, 08.07.2010 11:08: - I've to write a XML document including comments write in the sense of typing in a text editor? Or constructing one programmatically in memory? Or ... ? And what kind of data from what kind of source do you want to put into the document? All of that has an impact on the 'right' answer. - the document should be formatted that it could be viewed with a text editor What is the fastest (time for realization) approach doing it in python 2.5? If you mean to build an XML document programmatically, potentially using data that you get from somewhere, take a look at cElementTree. There's also a short recipe for pretty printing the tree before writing it out. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: simple python deployment tool
On Jul 8, 2:21 pm, Alexander Kapps alex.ka...@web.de wrote: King wrote: Hi, I am writing a python package deployment tool for linux based platforms. I have tried various existing tool sets but none of them is up to the mark and they have their own issues. Initially I'll start with simple approach. I'm sorry, but your approach is not going to work. The Right Way(tm) is to not ship any external libraries, but let the user install the dependencies with the distro's package manager. And to not try to build custom packages, but to let the package maintainers of the different distros package your application for you. Yes, you an do this by creating a .deb package that will take care of installing the required libraries. It may possible that either required version is not available in the distro's repository or the one available is older one then required. So best deal would be to ship at least all the python's libs along with your package. 1. Find all the modules/packages and copy to lib directory. 2. Find python's *.so dependencies(system libs) and copy them to syslib That would include all the way down to libc, your archive is going to be huge, but the actual problem is, what if the libc isn't compatible with the kernel, what if the WX, GTK, X11, etc libraries aren't compatible with the running Xserver? You are right here. It seems there is no standard definition of system libraries on linux. To over come the problem of binary compatibility, I am using ubuntu (hardy) which is on glibc 2.7. Every other external python library including python is compiled on hardy. This is would give you maximum chances that libs would be compatible in case if you run on any other distro which is using glibc =2.7 End of the story is, you would need to package a minimal (but almost complete) Linux system into your package, which of course is not want you want. 3. Copy python(executable) and libpython2.6.so.1.0 into dist directory. 4. Set up temp environment 5. Run main script using python main script.py The idea is to produce a cleaner directory structure. Neither I am creating a boot loader nor I am converting main script file into executable. It's plain vanilla stuff. It's not vanilla stuff, but a very hard problem. In fact you are fighting against the whole system structure. Ok, forget about system libs(libX*.* etc.), I don't know why it sounds too complicated. I am hoping it should work and eventually it's easier then tools that create an executable using bootloader. The problem is in setting the correct python environment. Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: Download Microsoft C/C++ compiler for use with Python 2.6/2.7 ASAP
Really? I wasn't entirely aware of this effect of the io module. Somehow, without at all paying attention (because certain third party modules blocking me for awhile, I never looked close enough), I just sort of thought the io module was mostly thin wrappers around stdio primitives, into a more Pythonic API. Yes, really. :) The new io modules doesn't use stdio. It operates solely on file descriptors and low level functions like open(2) instead of fopen(3). All high level functions like buffering is implemented in Python (and some C for speed). Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: delegation pattern via descriptor
kedra marbun wrote: i wonder what are the reasons for not passing the class on which the descriptor is attached to, what pattern is encouraged by this? The same answer applies. It's assumed that you will be writing a custom piece of code for each attribute of each class, and giving each one its own descriptor. By the time you get to the get or set method of a descriptor, you've already dispatched on both the class and attribute name. There is not usually any point in funnelling things back into a single function and then dispatching on the class or name again. Passing the class or name would just add overhead that was unnecessary in the majority of use cases. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Howto write XML file with comments?
Stefan Behnel schrieb: Alexander Eisenhuth, 08.07.2010 11:08: - I've to write a XML document including comments write in the sense of typing in a text editor? Or constructing one programmatically in memory? Or ... ? write means write to a file And what kind of data from what kind of source do you want to put into the document? Data is present as tags and attributes All of that has an impact on the 'right' answer. - the document should be formatted that it could be viewed with a text editor What is the fastest (time for realization) approach doing it in python 2.5? If you mean to build an XML document programmatically, potentially using data that you get from somewhere, take a look at cElementTree. There's also a short recipe for pretty printing the tree before writing it out. Is the API of cElementTree different from ElementTree in the python standard library? Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Howto write XML file with comments?
Alexander Eisenhuth, 08.07.2010 12:07: Stefan Behnel schrieb: Alexander Eisenhuth, 08.07.2010 11:08: - I've to write a XML document including comments write in the sense of typing in a text editor? Or constructing one programmatically in memory? Or ... ? write means write to a file You seam to imply that it's obvious what you want to do. From the little information that you give us, it's not. And what kind of data from what kind of source do you want to put into the document? Data is present as tags and attributes Whatever that is supposed to mean in this context. All of that has an impact on the 'right' answer. ... and it still does. - the document should be formatted that it could be viewed with a text editor What is the fastest (time for realization) approach doing it in python 2.5? If you mean to build an XML document programmatically, potentially using data that you get from somewhere, take a look at cElementTree. There's also a short recipe for pretty printing the tree before writing it out. Is the API of cElementTree different from ElementTree in the python standard library? Same thing, different import. Note that both were updated in Py2.7, BTW. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: C interpreter in Lisp/scheme/python
On 14 June, 00:07, bolega gnuist...@gmail.com wrote: I am trying to compare LISP/Scheme/Python for their expressiveness. For this, I propose a vanilla C interpreter. I have seen a book which writes C interpreter in C. The criteria would be the small size and high readability of the code. Are there already answers anywhere ? How would a gury approach such a project ? Bolega Probably you want to look at this thread http://groups.google.co.uk/group/comp.lang.lisp/browse_frm/thread/7b1ab36f5d5cce0a/54afe11153025e27?hl=enlnk=gstq=minim#54afe11153025e27 where I specified a toy language Minim (much simpler than C) and the goal was to construct an interpreter for it. Similar problem. Many solutions were given in different languages. The thread is very long. One thing you might look at is whether some sort of lexer/parser is supported in any of your targets. Qi supports a compiler-compiler Qi- YACC that allows you to write in BNF which makes this kind of project much easier. See http://www.lambdassociates.org/Book/page404.htm for an overview Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Ireland's pre-PyCon drinks - Wed, 14th July @ Trinity Capital Hotel
Vicky Twomey-Lee wrote: Hi All, Join us for drinks and a chat (a warm-up session to PyCon Ireland ;-) ). When: Wed 14th July, from 7pm Where: Trinity Capital Hotel More details at: http://www.python.ie/meetup/2010/python_ireland_meetup_-_july_2010/ Hope you all had a good piss-up! See you a week on Saturday. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Howto write XML file with comments?
Sorry for my little riddle, but you solved it quite good with: - http://effbot.org/zone/element-lib.htm#prettyprint and comments are also in ElementTree (xml.etree.ElementTree.Comment) Thanks Stefan Behnel schrieb: Alexander Eisenhuth, 08.07.2010 12:07: Stefan Behnel schrieb: Alexander Eisenhuth, 08.07.2010 11:08: - I've to write a XML document including comments write in the sense of typing in a text editor? Or constructing one programmatically in memory? Or ... ? write means write to a file You seam to imply that it's obvious what you want to do. From the little information that you give us, it's not. And what kind of data from what kind of source do you want to put into the document? Data is present as tags and attributes Whatever that is supposed to mean in this context. All of that has an impact on the 'right' answer. ... and it still does. - the document should be formatted that it could be viewed with a text editor What is the fastest (time for realization) approach doing it in python 2.5? If you mean to build an XML document programmatically, potentially using data that you get from somewhere, take a look at cElementTree. There's also a short recipe for pretty printing the tree before writing it out. Is the API of cElementTree different from ElementTree in the python standard library? Same thing, different import. Note that both were updated in Py2.7, BTW. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 12:53 am, Zooko O'Whielacronx zo...@zooko.com wrote: I don't understand. I described two different problems: problem one is that the inputs, outputs and literals of your program might be in a different encoding (in my experience they have most commonly been in decimal). Problem two is that your computations may be lossy. If the inputs, outputs and literals of your program are decimal (as they have been for most of my programs) then using decimal is better than using float because of problem one. Neither has a strict advantage over the other in terms of problem two. I can't think of any program I've ever written where the inputs are actually intended to be decimal. Consider a simple video editing program, and the user specifies a frame rate 23.976 fps. Is that what they really wanted? No, they wanted 24000/1001 but didn't feel like typing that. It should be instructive and telling that mplayer now strongly prefers the rational expressions over the decimal ones. Just because you haven't rounded your input in parsing it doesn't change the fact your input may have be rounded before it was presented to the program, which is the reality of the matter more often than not. Even in fields where the numbers are decimal, like finance, the rules are considerably more complicated than what you were taught in the schoolbook. This is one of the reasons IBM can still make a killing selling mainframes, they got all of those rules right decades ago. And that will be never true once you start using non trivial computation (i.e. transcendental functions like log, exp, etc...). I'm sorry, what will never be true? Are you saying that decimals have a disadvantage compared to floats? If so, what is their disadvantage? He's saying that once you get past elementary operations, you quickly run into irrational numbers, which you will not be representing accurately. Moreover, in general, it's impossible to even round operations involving transcendental functions to an arbitrary fixed- precision, you may need effectively infinite precision in order to the computation. In practice, this means the error induced by a lossy input conversion (assuming you hadn't already lost information) is entirely negligible compared to inherent inability to do the necessary calculations. -- http://mail.python.org/mailman/listinfo/python-list
Writing Out from 2 Lists
Hi,
I am trying to find a python solution for an informatics problem I have at
work. Generalized equivalent of my problem is:
I have an excel sheet with column 1 and column 2 having corresponding
information (much like a dictionary, however with repeating keys). Its like
if
you read down column 1: a,b,a,a,b,c and if you read down column 2: 1,2,3,4,5,6.
I would like to write unique files such as a.txt, b.txt and c.txt with
matching information (from column 2) in a.txt in separate rows like: 1,3,4 and
in b.txt like: 2,5, etc.
What I have been able to do until now is the following:
import sys
sys.path.append(C:/Downloads/Python/xlrd-0.6.1)
import xlrd
wb = xlrd.open_workbook(myexcelsheet.xls)
sheet = wb.sheet_by_index(0)
clmn1 = sheet.col_values(1,1)
clmn2 = sheet.col_values(2,1)
#NOW WHAT IS HAVE ARE TWO LISTS WITH CORRESPONDING INDICES
#My thought is now to write a counter and for each value in clmn1, write a text
file with the name of the value and add data from clmn2 in this file. I want to
start with index[0], i.e. value 1 of clmn1 and then go to 2nd value and compare
(==) it with 1st value and if its equal then append to the same file 2nd value
from clmn2...like this with 3rd value in clmn1, i want to compare it to 1st and
2nd valuewrap this whole in to a nice loop.
#Don't know if this is the easy way, but this is what my mind came up with.
Now I am stuck in colored line below, where I am unable to create a new
filename with string coming from a variable:
l = len(clmn1)
c = 0
while (c l):
filename = clmn1[c]
fhdl1 = open('/mypythonfolder/%(filename)s_appendsomename.txt','w')
fhdl1.write(clmn2(c)+'\n')
print filename
...
c = c + 1
...
...
...
I would appreciate comments and suggestions on how can I get through this
problem and also would like to know if there are any methods for lists to solve
this problem in a different way.
Many Thanks,
Ronald.
--
http://mail.python.org/mailman/listinfo/python-list
Re: simple python deployment tool
King wrote: On Jul 8, 2:21 pm, Alexander Kapps alex.ka...@web.de wrote: King wrote: Hi, I am writing a python package deployment tool for linux based platforms. I have tried various existing tool sets but none of them is up to the mark and they have their own issues. Initially I'll start with simple approach. I'm sorry, but your approach is not going to work. The Right Way(tm) is to not ship any external libraries, but let the user install the dependencies with the distro's package manager. And to not try to build custom packages, but to let the package maintainers of the different distros package your application for you. Yes, you an do this by creating a .deb package that will take care of installing the required libraries. It may possible that either required version is not available in the distro's repository or the one available is older one then required. Yes, that may happen. So best deal would be to ship at least all the python's libs along with your package. No, IMHO, the best way to cope with this, is to be a little conservative on what library versions you use. Don't use the most cutting edge versions, but those who are most wildly available. Even if you must use the most recent versions, you should leave the packaging to the distro maintainers. They know their distro a lot better and know how to maintain compatiblity with the rest of the system. 1. Find all the modules/packages and copy to lib directory. 2. Find python's *.so dependencies(system libs) and copy them to syslib That would include all the way down to libc, your archive is going to be huge, but the actual problem is, what if the libc isn't compatible with the kernel, what if the WX, GTK, X11, etc libraries aren't compatible with the running Xserver? You are right here. It seems there is no standard definition of system libraries on linux. To over come the problem of binary compatibility, I am using ubuntu (hardy) which is on glibc 2.7. Every other external python library including python is compiled on hardy. This is would give you maximum chances that libs would be compatible in case if you run on any other distro which is using glibc =2.7 Just because any other disto is based on glibc 2.7 doesn't ensure, that the other parts (like gtk libs vs. X11) are compatible. Actually by doing so, you are limiting your package to Hardy only. Any compatiblity with other Ubuntu versions or other distros would be purely by accident. End of the story is, you would need to package a minimal (but almost complete) Linux system into your package, which of course is not want you want. 3. Copy python(executable) and libpython2.6.so.1.0 into dist directory. 4. Set up temp environment 5. Run main script using python main script.py The idea is to produce a cleaner directory structure. Neither I am creating a boot loader nor I am converting main script file into executable. It's plain vanilla stuff. It's not vanilla stuff, but a very hard problem. In fact you are fighting against the whole system structure. Ok, forget about system libs(libX*.* etc.), I don't know why it sounds too complicated. I am hoping it should work and eventually it's easier then tools that create an executable using bootloader. The problem is in setting the correct python environment. It sounds complicated, because it is complicated. :-) Now, imagine, everybody would do this. There would be dozens, maybe hundreds of redundant copies of Python, libXYZ, etc. I don't know what you mean by create an executable using bootloader, but really, the correct way is to leave the dependency management to the distro and it's package manager. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On 07/07/2010 08:08 PM, Raymond Hettinger wrote: On Jul 7, 5:55 am, Mark Dickinson dicki...@gmail.com wrote: On Jul 7, 1:05 pm, david mainzer d...@tu-clausthal.de wrote: Dear Python-User, today i create some slides about floating point arithmetic. I used an example from http://docs.python.org/tutorial/floatingpoint.html so i start the python shell on my linux machine: d...@maxwell $ python Python 2.6.5 (release26-maint, May 25 2010, 12:37:06) [GCC 4.3.4] on linux2 Type help, copyright, credits or license for more information. sum = 0.0 for i in range(10): ... sum += 0.1 ... sum 0.99989 But thats looks a little bit wrong for me ... i must be a number greater then 1.0 because 0.1 = 0.155511151231257827021181583404541015625000 in python ... if i print it. [Mark Dickinson] So you've identified one source of error here, namely that 0.1 isn't exactly representable (and you're correct that the value stored internally is actually a little greater than 0.1). But you're forgetting about the other source of error in your example: when you do 'sum += 0.1', the result typically isn't exactly representable, so there's another rounding step going on. That rounding step might produce a number that's smaller than the actual exact sum, and if enough of your 'sum += 0.1' results are rounded down instead of up, that would easily explain why the total is still less than 1.0. One key for understanding floating point mysteries is to look at the actual binary sums rather that their approximate representation as a decimal string. The hex() method can make it easier to visualize Mark's explanation: s = 0.0 for i in range(10): ... s += 0.1 ... print s.hex(), repr(s) 0x1.ap-4 0.10001 0x1.ap-3 0.20001 0x1.4p-2 0.30004 0x1.ap-2 0.40002 0x1.0p-1 0.5 0x1.3p-1 0.59998 0x1.6p-1 0.69996 0x1.9p-1 0.79993 0x1.cp-1 0.89991 0x1.fp-1 0.99989 Having used hex() to understand representation error (how the binary partial sums are displayed), you can use the Fractions module to gain a better understanding of rounding error introduced by each addition: s = 0.0 for i in range(10): exact = Fraction.from_float(s) + Fraction.from_float(0.1) s += 0.1 actual = Fraction.from_float(s) error = actual - exact print '%-35s%-35s\t%s' % (actual, exact, error) 3602879701896397/36028797018963968 3602879701896397/36028797018963968 0 3602879701896397/18014398509481984 3602879701896397/18014398509481984 0 1351079888211149/4503599627370496 10808639105689191/36028797018963968 1/36028797018963968 3602879701896397/9007199254740992 14411518807585589/36028797018963968 -1/36028797018963968 1/2 18014398509481985/36028797018963968 -1/36028797018963968 5404319552844595/9007199254740992 21617278211378381/36028797018963968 -1/36028797018963968 3152519739159347/4503599627370496 25220157913274777/36028797018963968 -1/36028797018963968 7205759403792793/9007199254740992 28823037615171173/36028797018963968 -1/36028797018963968 2026619832316723/2251799813685248 32425917317067569/36028797018963968 -1/36028797018963968 9007199254740991/9007199254740992 36028797018963965/36028797018963968 -1/36028797018963968 Hope this helps your slides, Raymond Thanks to all for your help :) All your comments helped me and now i know how it works in python ! Best David -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing Out from 2 Lists
On Thu, Jul 8, 2010 at 4:30 AM, Dlanor Slegov
dlanorsle...@rocketmail.com wrote:
Hi,
I am trying to find a python solution for an informatics problem I have at
work. Generalized equivalent of my problem is:
I have an excel sheet with column 1 and column 2 having corresponding
information (much like a dictionary, however with repeating keys). Its
like if you read down column 1: a,b,a,a,b,c and if you read down column 2:
1,2,3,4,5,6. I would like to write unique files such as a.txt, b.txt and
c.txt with matching information (from column 2) in a.txt in separate rows
like: 1,3,4 and in b.txt like: 2,5, etc.
What I have been able to do until now is the following:
import sys
sys.path.append(C:/Downloads/Python/xlrd-0.6.1)
import xlrd
wb = xlrd.open_workbook(myexcelsheet.xls)
sheet = wb.sheet_by_index(0)
clmn1 = sheet.col_values(1,1)
clmn2 = sheet.col_values(2,1)
#NOW WHAT IS HAVE ARE TWO LISTS WITH CORRESPONDING INDICES
#My thought is now to write a counter and for each value in clmn1, write a
text file with the name of the value and add data from clmn2 in this file. I
want to start with index[0], i.e. value 1 of clmn1 and then go to 2nd value
and compare (==) it with 1st value and if its equal then append to the same
file 2nd value from clmn2...like this with 3rd value in clmn1, i want to
compare it to 1st and 2nd valuewrap this whole in to a nice loop.
#Don't know if this is the easy way, but this is what my mind came up
with. Now I am stuck in colored line below, where I am unable to create a
new filename with string coming from a variable:
l = len(clmn1)
c = 0
while (c l):
filename = clmn1[c]
fhdl1 = open('/mypythonfolder/%(filename)s_appendsomename.txt','w')
fhdl1 = open('/mypythonfolder/%s_appendsomename.txt' % filename,'w')
fhdl1.write(clmn2(c)+'\n')
print filename
...
c = c + 1
Personally, I'd rewrite the loop like this, using zip() and a for-loop:
for name, value in zip(clmn1, clmn2):
filepath = '/mypythonfolder/' + name + '_appendsomename.txt'
with open(filepath, 'a') as f:
f.write(str(value))
f.write('\n')
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Argh! Name collision!
On Jul 7, 6:47 pm, Alf P. Steinbach /Usenet alf.p.steinbach +use...@gmail.com wrote: Hm, for pure shock value I think I'll use the acronym PYthon Native Interface Support. pynis! :-) Well as long as you don't put your pynis *pointers* in pynie then everything will be Ok! ;-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 7:23 am, Mark Dickinson dicki...@gmail.com wrote: On Jul 8, 11:58 am, Adam Skutt ask...@gmail.com wrote: accurately. Moreover, in general, it's impossible to even round operations involving transcendental functions to an arbitrary fixed- precision, you may need effectively infinite precision in order to the computation. Impossible? Can you explain what you mean by this? Doesn't the decimal module do exactly that, giving correctly-rounded exp() and log() results to arbitrary precision? You run into the table-maker's dilemma: there's no way to know in advance how many digits you need in order to have n bits of precision in the result. For some computations, the number of bits required to get the desired precision can quickly overwhelm the finite limitations of your machine (e.g., you run out of RAM first or the time to compute the answer is simply unacceptable). Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
Wolfram Hinderer wrote: On 7 Jul., 19:32, Ethan Furman et...@stoneleaf.us wrote: Nobody wrote: On Wed, 07 Jul 2010 15:08:07 +0200, Thomas Jollans wrote: you should never rely on a floating-point number to have exactly a certain value. Never is an overstatement. There are situations where you can rely upon a floating-point number having exactly a certain value. It's not much of an overstatement. How many areas are there where you need the number 0.155511151231257827021181583404541015625000? If I'm looking for 0.1, I will *never* (except by accident ;) say if var == 0.1: it'll either be = or =. The following is an implementation of a well-known algorithm. Why would you want to replace the floating point comparisons? With what? snip code Interesting. I knew when I posted my above comment that I was ignoring such situations. I cannot comment on the code itself as I am unaware of the algorithm, and haven't studied numbers extensively (although I do find them very interesting). So while you've made your point, I believe mine still stands -- comparing floats using == to absolute numbers is almost always a mistake. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
On Wed, Jul 7, 2010 at 2:22 PM, Stephen Hansen me+list/pyt...@ixokai.iowrote:
First, its always best to be explicit with insert statements. Meaning,
don't rely on the underlining structure of a table, as in:
INSERT INTO YourRandomTable VALUES (my, value, here);
Instead, do:
INSERT INTO YourRandomTable (field1, field2, field3) VALUES (my,
value, here);
By following this advice, I realized I didn't need to do that fancy
multiplying out the '%s' which was screwing me up anyway, and then I didn't
need to create an sql using a '%', and then I didn't need to open the door
to injection attack!
However, I now have another error. Here is my current command:
cursor.execute(insert into personalDataKeys (Store, User, useFirstName,
useLastName, usePhone, useCell, useFax, useAddress, useShippingAddress,
useDOB, useEmail, usePW) values (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s,
%s), ([store, user] + col_vals))
I get this error from MySQL which I am having a hard time understanding:
LATEST FOREIGN KEY ERROR
100708 6:15:01 Transaction:
TRANSACTION 0 9382, ACTIVE 0 sec, process no 5326, OS thread id 1169992000
inserting, thread declared inside InnoDB 500
mysql tables in use 1, locked 1
3 lock struct(s), heap size 368, undo log entries 1
MySQL thread id 1502, query id 23700 localhost beno update
insert into personalDataKeys (Store, User, useFirstName, useLastName,
usePhone, useCell, useFax, useAddress, useShippingAddress, useDOB, useEmail,
usePW) values ('specialty', 'patients', 1, 1, 1, 1, 1, 1, 0, 1, 1, 1)
Foreign key constraint fails for table `test/personalDataKeys`:
,
CONSTRAINT `personalDataKeys_ibfk_1` FOREIGN KEY (`Store`) REFERENCES
`products` (`Store`)
Trying to add in child table, in index `Store` tuple:
DATA TUPLE: 2 fields;
0: len 9; hex 7370656369616c7479; asc specialty;; 1: len 6; hex
03b7; asc ;;
But in parent table `test/products`, in index `Store`,
the closest match we can find is record:
PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 696e66696d756d00; asc infimum ;;
What is this tuple?
TIA,
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 2:00 pm, Adam Skutt ask...@gmail.com wrote: On Jul 8, 7:23 am, Mark Dickinson dicki...@gmail.com wrote: On Jul 8, 11:58 am, Adam Skutt ask...@gmail.com wrote: accurately. Moreover, in general, it's impossible to even round operations involving transcendental functions to an arbitrary fixed- precision, you may need effectively infinite precision in order to the computation. Impossible? Can you explain what you mean by this? Doesn't the decimal module do exactly that, giving correctly-rounded exp() and log() results to arbitrary precision? You run into the table-maker's dilemma: there's no way to know in advance how many digits you need in order to have n bits of precision in the result. Sure. But it's a bit of a stretch to go from not knowing what resources you'll need in advance to calling something 'impossible'. :) For some computations, the number of bits required to get the desired precision can quickly overwhelm the finite limitations of your machine (e.g., you run out of RAM first or the time to compute the answer is simply unacceptable). Perhaps in theory. In practice, though, it's very rare to need to increase precision more than once or twice beyond an initial first guesstimate, and the amount of extra precision needed is small. That increase is unlikely to cause problems unless you were operating right up against your machine's limits in the first place. -- Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: The real problem with Python 3 - no business case for conversion (was I strongly dislike Python 3)
On Jul 7, 2010, at 11:26 PM, Terry Reedy wrote: On 7/7/2010 5:29 AM, geremy condra wrote: On Tue, Jul 6, 2010 at 1:37 AM, Terry Reedytjre...@udel.edu wrote: On 7/5/2010 9:00 PM, Philip Semanchuk wrote: On Jul 5, 2010, at 6:41 PM, Chris Rebert wrote: On Mon, Jul 5, 2010 at 3:38 PM, Philip Semanchu I ported two pure C extensions from 2 to 3 and was even able to keep a single C codebase. I'd be willing to contribute my experiences to a document somewhere. (Is there a Wiki?) Indeed there is: http://wiki.python.org/moin/ Thanks. I don't want to appear ungrateful, but I was hoping for something specific to the 2-to-3 conversion. I guess someone has to start somewhere... There is an existing 2to3 and other pages for Python code conversion. I do not know of any for CAPI conversion. The need for such has been acknowledged among the devs but if there is nothing yet, we need someone with specialized experience and a bit of time to make a first draft. If you start one, give it an easy to remember name C2to3? 2to3Capi? You choose. And link to it from the 2to3 pag In his post on this thread, Martin Loewis volunteered to list what he knows from psycopg2 if someone else will edit. I'm not sure why I don't have this post, but I'm happy to help edit etc if Martin wants to put together a rough draft. Since I wrote that, someone pointed out the the Python How-to collection includes Porting Extension Modules to 3.0 by Benjamim Peterson. So all Pyilip or Martin need to do is read that and suggest additions. That document is here: http://wiki.python.org/moin/PortingExtensionModulesToPy3k It took me a while to find. It's a shame it's not better known; I looked for such a document before I started porting sysv_ipc and posix_ipc and didn't find this one. Thanks for the pointer. Cheers Philip -- http://mail.python.org/mailman/listinfo/python-list
Debugging a segmentation fault
Hi,
my python project crashes in a non reproducible way. With gdb I got
the backtraces given below.
How can I possibly figure out the reason for the segfaults that occur
under Linux and Windows, using Python 2.6 in both cases.
Thanks
Dierk
Program received signal SIGSEGV, Segmentation fault.
0x080902bf in dict_dealloc (mp=0xbfbf0b4) at ../Objects/dictobject.c:
911
911 ../Objects/dictobject.c: No such file or directory.
in ../Objects/dictobject.c
(gdb) bt
#0 0x080902bf in dict_dealloc (mp=0xbfbf0b4) at ../Objects/
dictobject.c:911
#1 0x08090304 in dict_dealloc (mp=0xa805d74) at ../Objects/
dictobject.c:911
#2 0x08090304 in dict_dealloc (mp=0xbc2768c) at ../Objects/
dictobject.c:911
#3 0x08090304 in dict_dealloc (mp=0xbc27c64) at ../Objects/
dictobject.c:911
#4 0x080ab611 in subtype_dealloc (self=
Graph(node={u'(truncated)) at ../Objects/typeobject.c:1004
#5 0x08090304 in dict_dealloc (mp=0x9d3624c) at ../Objects/
dictobject.c:911
#6 0x0806c88b in instance_dealloc (inst=0x96641ec)
at ../Objects/classobject.c:668
#7 0x080dd1ad in PyEval_EvalFrameEx (f=
---Type return to continue, or q return to quit---
Frame 0x969ddb4, for file analyse.py, line 398, in analyseall
(options=Values(create_diagrams=True) at remote 0x966406c,
TIMEFMT='%d.%m.%Y', TIMESPAN_DAYS=365, projects=[('ant', 2002, 2008),
('apache', 1995, 2009), ('gcc', 2003, 2009), ('gimp', 2003, 2009),
('gnucash', 2001, 2009), ('gnumeric', 2000, 2009), ('gtk', 2000,
2009), ('kde', 2003, 2009), ('maemo', 2006, 2009), ('python', 1999,
2009), ('samba', 2004, 2009), ('tomcat', 2000, 2009), ('wine', 2003,
2009)],
ra=RegrAnalyzer(tseries=[Timeseries(results_hash={datetime.date at
remote 0x9773b00: {'grClusterCoeff': float at remote 0x97e0b8c,
'grDensity': float at remote 0x97e0b4c, 'grNumberMails': 1593,
'mailsPerDay': float at remote 0x97e0b7c, 'grMedianDegree':
numpy.float64 at remote 0x9695a50, 'productivity': float at remote
0x97e0b6c, 'grDistKSTest': numpy.float64 at remote 0x947e410,
'grEdges': 269, 'grBetweenCentNetw': float at remote 0x97e0b5c,
'grNumberNodes': 122, 'grBetweenCentNodeMedian': numpy.float64 at
remote 0x93581e0, 'gr...(truncated), throwflag=0) at ../Python/
ceval.c:1010
#8 0x080e1bb0 in fast_function (f=
Frame 0x82eda8c, for file analyse.py, line 472, in module (),
throwflag=0) at ../Python/ceval.c:3836
#9 call_function (f=
Frame 0x82eda8c, for file analyse.py, line 472, in module (),
throwflag=0) at ../Python/ceval.c:3771
#10 PyEval_EvalFrameEx (f=
Frame 0x82eda8c, for file analyse.py, line 472, in module (),
throwflag=0) at ../Python/ceval.c:2412
Program received signal SIGSEGV, Segmentation fault.
visit_decref (op=unknown at remote 0x2, data=0x0) at ../Modules/
gcmodule.c:271
271 ../Modules/gcmodule.c: No such file or directory.
in ../Modules/gcmodule.c
(gdb) bt
#0 visit_decref (op=unknown at remote 0x2, data=0x0) at ../
Modules/gcmodule.c:271
#1 0x0809223d in dict_traverse (op=
{'_sa_adapter': CollectionAdapter(_data=weakref at remote
0x1db90734, owner_state=InstanceState(obj=weakref at remote
0xae4de3c, _strong_obj=None, callables={}, session_id=158970732,
modified=False,
class_=EntityMeta(mapper=Mapper(primary_key_argument=None,
inherits=None, _all_tables=set([...]), _validators={},
_inheriting_mappers=set([]), _with_polymorphic_selectable=...,
single=False, allow_partial_pks=True,
_dependency_processors=[DetectKeySwitch(mapper=...,
direction=symbol(name='MANYTOONE') at remote 0x8605b2c,
parent=Mapper(primary_key_argument=None, inherits=None,
_all_tables=set([...]), _validators={}, _inheriting_mappers=set([]),
single=False, allow_partial_pks=True, _dependency_processors=[],
tables=[...], order_by=False,
primary_key=ColumnSet(_list=[Column(proxy_set=set([...]),
index=None, server_onupdate=None, name='id', is_literal=False,
nullable=False, default=None, quote=None, autoincrement=True,
onupdate=None, foreign_keys=OrderedSet(_list=[]) at remote
0x96d7554, _...(truncated), visit=0x810c460 visit_decref, arg=0x0)
at ../Objects/dictobject.c:2003
#2 0x0810cebc in subtract_refs (generation=value optimised out)
at ../Modules/gcmodule.c:296
#3 collect (generation=value optimised out) at ../Modules/
gcmodule.c:817
#4 0x0810d8eb in collect_generations (basicsize=28) at ../Modules/
gcmodule.c:924
#5 _PyObject_GC_Malloc (basicsize=28) at ../Modules/gcmodule.c:1363
#6 0x080ab65b in PyType_GenericAlloc (type=0x8236be0, nitems=0) at ../
Objects/typeobject.c:758
#7 0x080bc9a7 in weakref___new__ (type=0x8236be0, args=
(Message at remote 0x2526e38c, instancemethod at remote
0x2526ab6c), kwargs=0x0)
at ../Objects/weakrefobject.c:300
#8 0x080ad0dd in type_call (type=0x8236be0, args=
(Message at remote 0x2526e38c, instancemethod at remote
0x2526ab6c), kwds=0x0)
at ../Objects/typeobject.c:726
#9 0x0806245a in PyObject_Call (func=type at remote 0x8236be0, arg=
(Message at remote 0x2526e38c, instancemethod at remote
0x2526ab6c), kw=0x0)
Re: Storing a callback function as a class member
Hey, Sorry, I tried to sent only the relevant parts of the example, but the part where the error was, was left out. I defined the function, used as callback like this: class SomeClass: def callback(param): ... So I forgot the self parameter, and therefor the callback had a different number of parameters than I expected. Thanks for the effort! Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
Adam Skutt ask...@gmail.com wrote:
On Jul 8, 7:23 am, Mark Dickinson dicki...@gmail.com wrote:
On Jul 8, 11:58 am, Adam Skutt ask...@gmail.com wrote:
accurately. Moreover, in general, it's impossible to even round
operations involving transcendental functions to an arbitrary fixed-
precision, you may need effectively infinite precision in order to the
computation.
Impossible? Can you explain what you mean by this? Doesn't the
decimal module do exactly that, giving correctly-rounded exp() and
log() results to arbitrary precision?
You run into the table-maker's dilemma: there's no way to know in
advance how many digits you need in order to have n bits of precision
in the result. For some computations, the number of bits required to
get the desired precision can quickly overwhelm the finite limitations
of your machine (e.g., you run out of RAM first or the time to compute
the answer is simply unacceptable).
Yes, this appears to be unsolved yet, see also:
http://www.cs.berkeley.edu/~wkahan/LOG10HAF.TXT
Is it time to quit yet? That's the Table-Maker's Dilemma. No general
way exists to predict how many extra digits will have to be carried to
compute a transcendental expression and round it _correctly_ to some
preassigned number of digits. Even the fact (if true) that a finite
number of extra digits will ultimately suffice may be a deep theorem.
However, in practice, mpfr rounds correctly and seems to be doing fine.
In addition to this, I've been running at least 6 months of continuous
tests comparing cdecimal and decimal, and neither log() nor exp() poses
a problem.
pow() is trickier. Exact results have to be weeded out before
attempting the correction loop for correct rounding, and this is
complicated.
For example, in decimal this expression takes a long time (in cdecimal
the power function is not correctly rounded):
Decimal('100.0') ** Decimal('-557.71e-74288')
Stefan Krah
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 9:22 am, Mark Dickinson dicki...@gmail.com wrote: On Jul 8, 2:00 pm, Adam Skutt ask...@gmail.com wrote: On Jul 8, 7:23 am, Mark Dickinson dicki...@gmail.com wrote: On Jul 8, 11:58 am, Adam Skutt ask...@gmail.com wrote: accurately. Moreover, in general, it's impossible to even round operations involving transcendental functions to an arbitrary fixed- precision, you may need effectively infinite precision in order to the computation. Impossible? Can you explain what you mean by this? Doesn't the decimal module do exactly that, giving correctly-rounded exp() and log() results to arbitrary precision? You run into the table-maker's dilemma: there's no way to know in advance how many digits you need in order to have n bits of precision in the result. Sure. But it's a bit of a stretch to go from not knowing what resources you'll need in advance to calling something 'impossible'. :) For some computations, the number of bits required to get the desired precision can quickly overwhelm the finite limitations of your machine (e.g., you run out of RAM first or the time to compute the answer is simply unacceptable). Perhaps in theory. In practice, though, it's very rare to need to increase precision more than once or twice beyond an initial first guesstimate, and the amount of extra precision needed is small. That increase is unlikely to cause problems unless you were operating right up against your machine's limits in the first place. I suspect your platitude isn't especially comforting for those who need more computing capability than we can currently construct. However, I wouldn't call the amount of extra needed precision small for most transcendental functions, as it's frequently more than double in the worse-case situations and increases non-linearly as the number of desired digits increases. Which almost brings us full circle to where I was originally pointing: the rounding problem is inherent in the finite nature of a physical computer, so you cannot make the rounding problem go away. As such, talking about differences in rounding between decimal and binary representations is somewhat of a corner case. Replacing float with decimal won't get rid of the problems that floating-point brings to the table in the first place. The issues that come up all have to do with lack of human understanding of what the computer is doing. Take even as something as innocent as equality between two floating-point numbers: even exact rounding of all operations doesn't solve this entirely common problem. Accordingly, once we explain why this doesn't work, we frequently don't need the enhanced functionality decimal provides and hopefully can make the determination on our own. If you want to make elementary arithmetic (add, subtract, multiple, divide) behave intuitively then you (arguably) want an arbitrary- precision fractional/rational number class. After that, the right solution is education. Adam -- http://mail.python.org/mailman/listinfo/python-list
Python script to install python
I would like to have a python script that would download the most recent svn of python, configure, make, install and cleanup after itself. I am not replacing the python version I would be using to run the script. I was struggling to get this to work and I assume someone else has done it better. Any pointers? Thanks Vincent -- http://mail.python.org/mailman/listinfo/python-list
Issues compiling 2.6.5 on AIX 6.1
I've seen other threads on this issue, but the resolution still doesn't seem to exist for me. I'm running the configure script with these parameters: ./configure --prefix=/build/tools \ --exec-prefix=/build/tools \ --enable-shared \ --enable-ipv6 \ --with-gcc \ --with-pth I also want to state that I already edited all of the configure config.guess scripts to comprehend AIX6.1 (they're not updated yet). And as you can see, I'm running this using the -with-gcc option. The configure seems to have no problems; so I run a: make clean make This results in a completed compile. If I go back and look I see a lot of this: building '_struct' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/build/tools/src/Python-2.6.5/./Include -I/build/tools/include -I. -IInclude -I./Include -I/usr/local/include -I/build/tools/src/Python-2.6.5/Include -I/build/tools/src/Python-2.6.5 -c /build/tools/src/Python-2.6.5/Modules/_struct.c -o build/temp.aix-6.1-2.6/build/tools/src/Python-2.6.5/Modules/_struct.o ./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/temp.aix-6.1-2.6/build/tools/src/Python-2.6.5/Modules/_struct.o -L/build/tools/lib -L/usr/local/lib -lpython2.6 -o build/lib.aix-6.1-2.6/_struct.so collect2: library libpython2.6 not found Notice that it can't find the Python lib.. well, it's not installed yet. It's built and in the current directory I'm working out of, but that's not in my LiBPATH. I'll come back to that. If I don't do anything and try to make install it pushes the python libpython libraries to the appropriate locations, and then it appears it tries to re-make the modules: building '_struct' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/build/tools/src/Python-2.6.5/./Include -I/build/tools/include -I. -IInclude -I./Include -I/usr/local/include -I/build/tools/src/Python-2.6.5/Include -I/build/tools/src/Python-2.6.5 -c /build/tools/src/Python-2.6.5/Modules/_struct.c -o build/temp.aix-6.1-2.6/build/tools/src/Python-2.6.5/Modules/_struct.o ./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/temp.aix-6.1-2.6/build/tools/src/Python-2.6.5/Modules/_struct.o -L/build/tools/lib -L/usr/local/lib -lpython2.6 -o build/lib.aix-6.1-2.6/_struct.so However this time it results in a mass of: ld: 0711-224 WARNING: Duplicate symbol: some symbol I mean I get like 1000+ of these warnings. But they all culminate in: ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. Fatal Python error: Interpreter not initialized (version mismatch?) make: The signal code from the last command is 6. Now, I said I'd get back to the LIBPATH issue. I thought that if it was looking for that libpython file to compile the modules, it just wasn't finding it. So I executed a: LIBPATH=.:$LIBPATH make This resulted in: building '_struct' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/build/tools/src/Python-2.6.5/./Include -I/build/tools/include -I. -IInclude -I./Include -I/usr/local/include -I/build/tools/src/Python-2.6.5/Include -I/build/tools/src/Python-2.6.5 -c /build/tools/src/Python-2.6.5/Modules/_struct.c -o build/temp.aix-6.1-2.6/build/tools/src/Python-2.6.5/Modules/_struct.o ./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/temp.aix-6.1-2.6/build/tools/src/Python-2.6.5/Modules/_struct.o -L/build/tools/lib -L/usr/local/lib -lpython2.6 -o build/lib.aix-6.1-2.6/_struct.so ld: 0706-006 Cannot find or open library file: -l python2.6 ld:open(): No such file or directory collect2: ld returned 255 exit status Huh? Can't find the library now? In any event, I can't get Python to compile and/or install on my machine. Does anyone have any insight into what is happening to cause this problem? -B PRIVILEGED AND CONFIDENTIAL This email transmission contains privileged and confidential information intended only for the use of the individual or entity named above. If the reader of the email is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any use, dissemination or copying of this email transmission is strictly prohibited by the sender. If you have received this transmission in error, please delete the email and immediately notify the sender via the email return address or mailto:postmas...@argushealth.com. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
Ian hobso...@gmaiil.com wrote:
On 07/07/2010 19:38, Victor Subervi wrote:
Hi;
I have this code:
sql = 'insert into personalDataKeys values (%s, %s, %s)' %
(store,
user, ', %s'.join('%s' * len(col_vals))
cursor.execute(sql, col_vals)
Is this open to injection attacks? If so, how correct?
TIA,
beno
Yes, it is trivially open to injection attacks.
What would happen if someone enters the next line into one of your
col_vals
x,y);DROP DATABASE personalDataKeys; ha ha
Your sql statement would be closed early by the semicolon, and the
DROP TABLE personalDataKeys is then executed and would cause some
unexpected data loss.
Things could be more serious - DROP DATABASE mysql; for a mysql
installation for example.
You must always always every time and without any exceptions
what-so-ever, put all and every piece of data that comes from outside
the program through the appropriate routine to make whatever has been
entered into storable data and not part of the sql statement.
In php this is mysql_real_escape_string(). In your favourite language
there will be an equivalent.
If you miss just one occurrence its like leaving the side window
unlocked! Someone will get in one day.
Try reading the code more carefully. You don't need to escape the
strings so long as you use cursor.execute properly. In this case it is
being used properly: there is no danger of injection attacks from
col_vals.
If `store` or `user` come from user input there is a danger as they are
being inserted into the generated sql.
As has already been pointed out the code to generate the sql is broken,
but the principle behind it is sound. In this case the correct thing to
do would appear to be:
sql = 'insert into personalDataKeys values (%%s, %%s, %s)' % (','.join
(['%s'] * len(col_vals)))
cursor.execute(sql, (store, user) + col_vals)
which safely sanitises all of the data passed to the database.
--
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to test/troubshoot an extension (pylibconfig)?
On 2010-07-07, Grant Edwards inva...@invalid.invalid wrote:
Oops. Those Python bindings are for version 1.3.2 of libconfig (which
does work). They don't work with the current version of libconfig.
I've stripped the python bindings down to a minimal point, and I've
decided there may be a memory corruption problem in the 1.4.5 version
of the library. Here's the current boost binding:
-pylibconfig.cc--
#include boost/python.hpp
#include libconfig.h++
using namespace boost::python;
using namespace libconfig;
class pyConfig
{
public:
pyConfig()
{
config = new Config();
}
~pyConfig ()
{
delete config;
}
private:
Config *config;
};
BOOST_PYTHON_MODULE(pylibconfig)
{
class_pyConfig(Config);
}
-pylibconfig.cc--
That builds without warnings and installs fine, but attempting to
actually create an instance of the class in Python causes a glibc
memory corruption message:
Python 2.6.5 (release26-maint, Jun 22 2010, 12:58:11)
[GCC 4.3.4] on linux2
Type help, copyright, credits or license for more information.
import pylibconfig
conf = pylibconfig.Config()
*** glibc detected *** /usr/bin/python2.6: corrupted double-linked list:
0x08065c48 ***
Am I correct in concluding it has to be a memory corruption problem in
the library itself?
--
Grant Edwards grant.b.edwardsYow! I want the presidency
at so bad I can already taste
gmail.comthe hors d'oeuvres.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 released
In article 1450078b-d5ee-437f-bd8b-8da26900f...@x27g2000yqb.googlegroups.com, imageguy imageguy1...@gmail.com wrote: Sorry to be daft here, but what do you mean by a hardlink ? A windows Shortcut ? Just to be clear, a hardlink on NTFS functions almost exactly the same as a hardlink on a Unix filesystem -- it's a pointer to the same underlying file. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Normal is what cuts off your sixth finger and your tail... --Siobhan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 released
On 2010-07-08, Aahz a...@pythoncraft.com wrote: In article 1450078b-d5ee-437f-bd8b-8da26900f...@x27g2000yqb.googlegroups.com, imageguy imageguy1...@gmail.com wrote: Sorry to be daft here, but what do you mean by a hardlink ? A windows Shortcut ? Just to be clear, a hardlink on NTFS functions almost exactly the same as a hardlink on a Unix filesystem -- it's a pointer to the same underlying file. A windows shortcut is more like a Unix symlink (symbolic link), where the real destination path is a string contained in the link/shortcut file. That destination path is then evaluated and dereferenced when the link/shortcut is accessed. -- Grant Edwards grant.b.edwardsYow! Of course, you at UNDERSTAND about the PLAIDS gmail.comin the SPIN CYCLE -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Python script to install python
I would like to have a python script that would download the most recent svn of python, configure, make, install and cleanup after itself. I am not replacing the python version I would be using to run the script. I was struggling to get this to work and I assume someone else has done it better. Any pointers? Assuming you are on linux I recommend not using a python script for this but rather a shell script. From a python script you would most of the time be calling shell commands anyway. In a shell script you would do something like this: #!/bin/bash svn checkout cd whatever ./configure --whatever-options-you-like make # you probably want to run this as root make install # you probably don't want to be root anymore cd .. rm -rf whatever If you are on windows I assume a similar strategy is best. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
On 7/8/10 6:20 AM, Victor Subervi wrote:
However, I now have another error. Here is my current command:
cursor.execute(insert into personalDataKeys (Store, User,
useFirstName, useLastName, usePhone, useCell, useFax, useAddress,
useShippingAddress, useDOB, useEmail, usePW) values (%s, %s, %s, %s, %s,
%s, %s, %s, %s, %s, %s, %s), ([store, user] + col_vals))
Quick point: why the parens around [store, user] + col_vars? They're
redundant.
I get this error from MySQL which I am having a hard time understanding:
LATEST FOREIGN KEY ERROR
100708 6:15:01 Transaction:
TRANSACTION 0 9382, ACTIVE 0 sec, process no 5326, OS thread id
1169992000 inserting, thread declared inside InnoDB 500
mysql tables in use 1, locked 1
3 lock struct(s), heap size 368, undo log entries 1
MySQL thread id 1502, query id 23700 localhost beno update
insert into personalDataKeys (Store, User, useFirstName, useLastName,
usePhone, useCell, useFax, useAddress, useShippingAddress, useDOB,
useEmail, usePW) values ('specialty', 'patients', 1, 1, 1, 1, 1, 1, 0,
1, 1, 1)
Foreign key constraint fails for table `test/personalDataKeys`:
,
CONSTRAINT `personalDataKeys_ibfk_1` FOREIGN KEY (`Store`) REFERENCES
`products` (`Store`)
A foreign key is a constraint, a restriction, which says that rows in
TableA (personalDataKeys) depend on certain *matching* rows to already
exist and always be valid in TableB (products); the exact match is a
column they have in common (Store).
The purpose of foreign keys is to keep data consistent. Here, it appears
as if you have established a key such that the 'store' column in your
personalDataKeys table must point to a certain row in the products table
which has a 'store' column of the exact same value.
This error message is indicating that when you do this INSERT, there is
no corresponding row in the products table.
--
Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/
signature.asc
Description: OpenPGP digital signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
Zooko O'Whielacronx zo...@zooko.com wrote: I'm starting to think that one should use Decimals by default and reserve floats for special cases. Only if one has Power6 (or 7) which has hardware support for BCD. Otherwise you will have slow applications. Victor. -- Victor Eijkhout -- eijkhout at tacc utexas edu -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 released
On 7/8/10 8:07 AM, Grant Edwards wrote: On 2010-07-08, Aahz a...@pythoncraft.com wrote: In article 1450078b-d5ee-437f-bd8b-8da26900f...@x27g2000yqb.googlegroups.com, imageguy imageguy1...@gmail.com wrote: Sorry to be daft here, but what do you mean by a hardlink ? A windows Shortcut ? Just to be clear, a hardlink on NTFS functions almost exactly the same as a hardlink on a Unix filesystem -- it's a pointer to the same underlying file. A windows shortcut is more like a Unix symlink (symbolic link), where the real destination path is a string contained in the link/shortcut file. That destination path is then evaluated and dereferenced when the link/shortcut is accessed. This is true, but a windows shortcut is more limited: its a feature of higher level code in the UI (I don't want to say just Explorer, as the standard dialogs deal with it too), and not the filesystem. So it only really works if there's a user specifically clicking through it -- or if you have code made to look for the .lnk files, parse them (they're really simple INI files) and deference it manually. At least, IIUC. -- Stephen Hansen ... Also: Ixokai ... Mail: me+list/python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/ signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.7 released
On 08/07/2010 16:07, Grant Edwards wrote: On 2010-07-08, Aahza...@pythoncraft.com wrote: In article1450078b-d5ee-437f-bd8b-8da26900f...@x27g2000yqb.googlegroups.com, imageguyimageguy1...@gmail.com wrote: Sorry to be daft here, but what do you mean by a hardlink ? A windows Shortcut ? Just to be clear, a hardlink on NTFS functions almost exactly the same as a hardlink on a Unix filesystem -- it's a pointer to the same underlying file. A windows shortcut is more like a Unix symlink (symbolic link), where the real destination path is a string contained in the link/shortcut file. That destination path is then evaluated and dereferenced when the link/shortcut is accessed. Goodness knows I'm probably teaching my grandmother etc. etc. but I would clarify that a Windows shortcut is a *shell* concept: from the NTFS point of view, it's just a something.lnk with some opaque contents. A (= Vista) NTFS smbolic link is documented as designed to function just like Unix links. TJG -- http://mail.python.org/mailman/listinfo/python-list
Issue with logging.config
Hi Python Help:
I'm doing some work with logging.config and I'm running into an
interesting situation. I've run this by python-help, but that didn't help so I
thought I would send to the list. Here is the config file
[loggers]
keys=root,log,syslog
[handlers]
keys=console,log,syslog
[formatters]
keys=rootFormat,logFormat,syslogFormat
[logger_root]
level=DEBUG
handlers=console
[logger_log]
level=DEBUG
handlers=log
qualname=log
[logger_syslog]
level=DEBUG
handlers=syslog
qualname=syslog
[handler_console]
class=StreamHandler
level=DEBUG
formatter=rootFormat
args=(sys.stdout,)
[handler_log]
class=handlers.RotatingFileHandler
level=DEBUG
formatter=logFormat
args=('E:\local\Logs\syslog_interface.txt', 'a', 1000, 10)
propagate=0
[handler_syslog]
class=handlers.SysLogHandler
level=DEBUG
formatter=syslogFormat
host=141.232.41.205
port=handlers.SYSLOG_UDP_PORT
facility=LOG_LOCAL0
args=(('141.232.41.205',handlers.SYSLOG_UDP_PORT),handlers.SysLogHandler.LOG_LOCAL0)
[formatter_rootFormat]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[formatter_logFormat]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[formatter_syslogFormat]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
and the python code
# Imports
import logging
import logging.config
import os
import re
from threading import Thread
# Constants
CONFIG_FILENAME = 'E:\local\Config\syslog_interface.conf'
MCU_LIST_FILENAME = 'E:\local\Scada_Devices\mcu.txt'
# Classes
# PingIt
class PingIt(Thread):
def __init__(self, mcuIP):
Thread.__init__(self)
self.ip = mcuIP
self.status = -1
def run(self):
pinging = os.popen(ping -n 2 + self.ip, 'r')
while 1:
line = pinging.readline()
if not line:
break
gotResponse = re.findall(PingIt.lifeline, line)
if gotResponse:
self.status = int(gotResponse[0])
# Main Routine
#
# Get the logger configuration information
#
logging.config.fileConfig(CONFIG_FILENAME)
#
# Check if running from command line and create logger
#
if os.environ.get('PROMPT'):
#
# create logger for output to stdout
#
logger = logging.getLogger('root')
else:
#
# create logger for output to logfile
#
logger = logging.getLogger('log')
#
# Create logger for syslog output
#
syslog = logging.getLogger('syslog')
#
# Declare variables
#
PingIt.lifeline = re.compile(rReceived = (\d))
mcu_dict = {}
ping_list = []
status = (Not responding, Responded to only 1 ping of 2, Alive)
#
# Open the MCU file
#
mcu_file = open(MCU_LIST_FILENAME, 'r')
#
# Loop through the contents of the MCU file and ping the IPs
#
for mcu in mcu_file:
#
# mcu file contents example
# 192.168.97.227 MCU_HMSTD
#
# mcu_info[0] = MCU IP Address
# mcu_info[1] = MCU Name
#
mcu_info = mcu.split()
mcu_dict[mcu_info[0]] = mcu_info[1]
current = PingIt(mcu_info[0])
logger.info(Pinging + mcu_info[1])
ping_list.append(current)
current.start()
#
# Loop through ping list and print the response
#
for pinged in ping_list:
pinged.join()
logger.info(Status - + mcu_dict[pinged.ip] + is +
status[pinged.status])
syslog.info(Status - + mcu_dict[pinged.ip] + is +
status[pinged.status])
This is the output from the code
2010-07-06 14:43:58,280 - log - INFO - Status - netboss2 is Not responding
msg = 1342010-07-06 14:43:58,312 - syslog - INFO - Status - netboss2 is Not
responding
address = ('141.232.41.205', 514)
Traceback (most recent call last):
File C:\Python31\lib\logging\handlers.py, line 786, in emit
self.socket.sendto(msg, self.address)
TypeError: sendto() takes exactly 3 arguments (2 given)
2010-07-06 14:43:58,312 - syslog - INFO - Status - netboss2 is Not responding
This is the handlers.py code from the library. I added the print statement to
figure out why it is asking for three args but only getting two.
Line 777 of handlers.py
try:
if self.unixsocket:
try:
self.socket.send(msg)
except socket.error:
self._connect_unixsocket(self.address)
self.socket.send(msg)
else:
print('msg = ', msg, '\naddress = ', self.address)
self.socket.sendto(msg, self.address)
Re: Python -- floating point arithmetic
On Jul 8, 3:29 pm, Adam Skutt ask...@gmail.com wrote: On Jul 8, 9:22 am, Mark Dickinson dicki...@gmail.com wrote: On Jul 8, 2:00 pm, Adam Skutt ask...@gmail.com wrote: For some computations, the number of bits required to get the desired precision can quickly overwhelm the finite limitations of your machine (e.g., you run out of RAM first or the time to compute the answer is simply unacceptable). Perhaps in theory. In practice, though, it's very rare to need to increase precision more than once or twice beyond an initial first guesstimate, and the amount of extra precision needed is small. That increase is unlikely to cause problems unless you were operating right up against your machine's limits in the first place. I suspect your platitude isn't especially comforting for those who need more computing capability than we can currently construct. However, I wouldn't call the amount of extra needed precision small I think that's because we're talking at cross-purposes. To clarify, suppose you want to compute some value (pi; log(2); AGM(1, sqrt(2)); whatever...) to 1000 significant decimal places. Then typically the algorithm (sometimes known as Ziv's onion-peeling method) looks like: (1) Compute an initial approximation to 1002 digits (say), with known absolute error (given by a suitable error analysis); for the sake of argument, let's say that you use enough intermediate precision to guarantee an absolute error of 0.6 ulps. (2) Check to see whether that approximation unambiguously gives you the correctly-rounded 1000 digits that you need. (3) If not, increase the target precision (say by 3 digits) and try again. It's the precision increase in (3) that I was calling small, and similarly it's step (3) that isn't usually needed more than once or twice. (In general, for most functions and input values; I dare say there are exceptions.) Step (1) will often involve using significantly more than the target precision for intermediate computations, depending very much on what you happen to be trying to compute. IIUC, it's the extra precision in step (1) that you don't want to call 'small', and I agree. IOW, I'm saying that the extra precision required *due to the table- maker's dilemma* generally isn't a concern. I actually agree with much of what you've said. It was just the impossible claim that went over the top (IMO). The MPFR library amply demonstrates that computing many transcendental functions to arbitrary precision, with correctly rounded results, is indeed possible. -- Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: Python script to install python
On Thu, Jul 8, 2010 at 9:11 AM, Daniel Fetchinson fetchin...@googlemail.com wrote: I would like to have a python script that would download the most recent svn of python, configure, make, install and cleanup after itself. I am not replacing the python version I would be using to run the script. I was struggling to get this to work and I assume someone else has done it better. Any pointers? Assuming you are on linux I recommend not using a python script for this but rather a shell script. From a python script you would most of the time be calling shell commands anyway. In a shell script you would do something like this: #!/bin/bash svn checkout cd whatever ./configure --whatever-options-you-like make # you probably want to run this as root make install # you probably don't want to be root anymore cd .. rm -rf whatever Ok I'll take your advice and just use a shell script. I am on osx by the way. Thanks Vincent If you are on windows I assume a similar strategy is best. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
Zooko O'Whielacronx zo...@zooko.com writes: I'm starting to think that one should use Decimals by default and reserve floats for special cases. would you kindly lend me your Decimals ruler? i need to measure the sides of the triangle whose area i have to compute -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
On Thu, Jul 8, 2010 at 10:45 AM, Stephen Hansen me+list/pyt...@ixokai.iowrote:
On 7/8/10 6:20 AM, Victor Subervi wrote:
However, I now have another error. Here is my current command:
cursor.execute(insert into personalDataKeys (Store, User,
useFirstName, useLastName, usePhone, useCell, useFax, useAddress,
useShippingAddress, useDOB, useEmail, usePW) values (%s, %s, %s, %s, %s,
%s, %s, %s, %s, %s, %s, %s), ([store, user] + col_vals))
Quick point: why the parens around [store, user] + col_vars? They're
redundant.
I get this error from MySQL which I am having a hard time understanding:
LATEST FOREIGN KEY ERROR
100708 6:15:01 Transaction:
TRANSACTION 0 9382, ACTIVE 0 sec, process no 5326, OS thread id
1169992000 inserting, thread declared inside InnoDB 500
mysql tables in use 1, locked 1
3 lock struct(s), heap size 368, undo log entries 1
MySQL thread id 1502, query id 23700 localhost beno update
insert into personalDataKeys (Store, User, useFirstName, useLastName,
usePhone, useCell, useFax, useAddress, useShippingAddress, useDOB,
useEmail, usePW) values ('specialty', 'patients', 1, 1, 1, 1, 1, 1, 0,
1, 1, 1)
Foreign key constraint fails for table `test/personalDataKeys`:
,
CONSTRAINT `personalDataKeys_ibfk_1` FOREIGN KEY (`Store`) REFERENCES
`products` (`Store`)
A foreign key is a constraint, a restriction, which says that rows in
TableA (personalDataKeys) depend on certain *matching* rows to already
exist and always be valid in TableB (products); the exact match is a
column they have in common (Store).
The purpose of foreign keys is to keep data consistent. Here, it appears
as if you have established a key such that the 'store' column in your
personalDataKeys table must point to a certain row in the products table
which has a 'store' column of the exact same value.
This error message is indicating that when you do this INSERT, there is
no corresponding row in the products table.
mysql describe products Store;
+---+-+--+-+-+---+
| Field | Type| Null | Key | Default | Extra |
+---+-+--+-+-+---+
| Store | varchar(40) | NO | MUL | NULL| |
+---+-+--+-+-+---+
1 row in set (0.00 sec)
mysql describe personalDataKeys Store;
+---+-+--+-+-+---+
| Field | Type| Null | Key | Default | Extra |
+---+-+--+-+-+---+
| Store | varchar(40) | NO | MUL | NULL| |
+---+-+--+-+-+---+
1 row in set (0.00 sec)
They both use innodb. They're both indexed. I was thinking after getting
your email that maybe I'd set the varchars to different lengths, but no.
However...
mysql select * from products;
Empty set (0.00 sec)
Is it that I can't insert into personalDataKeys until I've first done so in
products? After rethinking this, it occurred to me that I probably made a
mistake in copying my create table command from personalData to
personalDataKeys, both of which had the foreign key of Store referenced to
table products. That wasn't necessary, since personalDataKeys only needs to
be associated with personalData, so I dropped and recreated the table,
updating personalDataKeys foreign key to reference personalData; however,
once again:
mysql select * from personalData;
Empty set (0.00 sec)
Here's the deal:
mysql describe personalData;
+---+--+--+-+++
| Field | Type | Null | Key | Default|
Extra |
+---+--+--+-+++
| ID| int(10) unsigned | NO | PRI | NULL |
auto_increment |
| Store | varchar(40) | NO | MUL | NULL
||
| User | varchar(50) | NO | MUL | NULL
||
| FirstName | varchar(100) | NO | | NULL
||
| LastName | varchar(100) | NO | | NULL
||
| Phone | varchar(13) | YES | | NULL
||
| Cell | varchar(13) | YES | | NULL
||
| Fax | varchar(13) | YES | | NULL
||
| AddressID | int(11) | NO | MUL | NULL
||
| ShippingAddressID | int(11) | NO | MUL | NULL
||
| DOB | date | YES | | 2000-01-01
||
| Email | varchar(100) | NO | | NULL
||
| PW| varchar(12) | NO | | NULL
||
+---+--+--+-+++
13 rows in set (0.00 sec)
mysql describe personalDataKeys;
Re: Python -- floating point arithmetic
On Thu, Jul 8, 2010 at 8:52 AM, Giacomo Boffi giacomo.bo...@polimi.it wrote: Zooko O'Whielacronx zo...@zooko.com writes: I'm starting to think that one should use Decimals by default and reserve floats for special cases. would you kindly lend me your Decimals ruler? i need to measure the sides of the triangle whose area i have to compute If your ruler doesn't have a [second] set of marks for centimeters and millimeters, that's really one cheap/cruddy ruler you're using. Cheers, Chris -- Metrication! http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Thu, Jul 8, 2010 at 4:58 AM, Adam Skutt ask...@gmail.com wrote: I can't think of any program I've ever written where the inputs are actually intended to be decimal. Consider a simple video editing program, and the user specifies a frame rate 23.976 fps. Is that what they really wanted? No, they wanted 24000/1001 but didn't feel like typing that. Okay, so there was a lossy conversion from the user's intention (24000/1001) to what they typed in (23.976). instr = '23.976' Now as a programmer you have two choices: 1. accept what they typed in and losslessly store it in a decimal: from decimal import Decimal as D x = D(instr) print x 23.976 2. accept what they typed in and lossily convert it to a float: x = float(instr) print %.60f % (x,) 23.97509050529822707176208496093750 option 2 introduces further error between what you have stored in your program and what the user originally wanted and offers no advantages except for speed, right? I'm sorry, what will never be true? Are you saying that decimals have a disadvantage compared to floats? If so, what is their disadvantage? He's saying that once you get past elementary operations, you quickly run into irrational numbers, which you will not be representing accurately. Moreover, in general, it's impossible to even round operations involving transcendental functions to an arbitrary fixed- precision, you may need effectively infinite precision in order to the computation. In practice, this means the error induced by a lossy input conversion (assuming you hadn't already lost information) is entirely negligible compared to inherent inability to do the necessary calculations. But this is not a disadvantage of decimal compared to float is it? These problems affect both representations. Although perhaps they affect them differently, I'm not sure. I think sometimes people conflate the fact that decimals can easily have higher and more variable precision than floats with the fact that decimals are capable of losslessly storing decimal values but floats aren't. Regards, Zooko -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
On 7/7/2010 11:52 AM, Stephen Hansen wrote:
On 7/7/10 11:38 AM, Victor Subervi wrote:
Hi;
I have this code:
sql = 'insert into personalDataKeys values (%s, %s, %s)' % (store,
user, ', %s'.join('%s' * len(col_vals))
cursor.execute(sql, col_vals)
Bad approach. Don't put actual data into an SQL statement using
string parameter substitution. Try this:
values = (store, user) + tuple(col_vals) # all values to be inserted
valuesql = ,.join([%s]*len(values)) # '%s,%s,%s,%s,%s,%s'
sql = INSERT INTO personaldatakeys VALUES ( + valuesql + )
cursor.execute(sql, values) # execute INSERT
valuefields is always some number of repeats of comma-separated %s
Anything in values will be escaped properly. No SQL injection
vulnerability.
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
How is Unladen Swallow coming along?
How is Unladen Swallow coming along? Looking at the site, code is being checked in and issues are being reported, but the last quarterly release was 2009 Q3. They missed their January 2010 release date for 2009 Q4, so they're now about 6 months behind their project plan. (http://code.google.com/p/unladen-swallow/wiki/ProjectPlan;) John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
On 7/8/10 9:03 AM, Victor Subervi wrote: mysql describe products Store; +---+-+--+-+-+---+ | Field | Type| Null | Key | Default | Extra | +---+-+--+-+-+---+ | Store | varchar(40) | NO | MUL | NULL| | +---+-+--+-+-+---+ 1 row in set (0.00 sec) mysql describe personalDataKeys Store; +---+-+--+-+-+---+ | Field | Type| Null | Key | Default | Extra | +---+-+--+-+-+---+ | Store | varchar(40) | NO | MUL | NULL| | +---+-+--+-+-+---+ 1 row in set (0.00 sec) They both use innodb. They're both indexed. I was thinking after getting your email that maybe I'd set the varchars to different lengths, but no. A foreign key isn't about the schema, per se; its not about the varchar's being different lengths (as they discard trailing padding)-- its about *data*. True, if you had varchar(20) and varchar(40), then if any string longer then 20 wouldn't ever pass -- but that's really secondary. (That's not saying a database may refuse to accept a FK if data types are mismatched) If personalDataKeys has a foreign key connecting it to products, then you can't add something to personalDataKeys with store = specialty unless something already exists in products with store = speciality; However... mysql select * from products; Empty set (0.00 sec) Is it that I can't insert into personalDataKeys until I've first done so in products? Yes, that's precisely what foreign keys do. That wasn't necessary, since personalDataKeys only needs to be associated with personalData, so I dropped and recreated the table, updating personalDataKeys foreign key to reference personalData; however, once again: Are you certain this is what you want? It sounds like you may be using foreign keys without fully understanding what they are. Think of them like a big arrow. If you define a foreign key in personalDataKeys, referencing personalData, you should picture a large arrow pointing from personalDataKeys to personalData. It's pointing because the constraint created by the foreign key means, Every record in this table, personalDataKeys, has a column which *must* exist in its referenced table, personalData, before that record is allowed to be added. A foreign key isn't just a description of a relationship: its a strict rule, declaring that a certain field in one table *actually* refers directly to a *specific* row in *another* table: therefore, this field can't be allowed to be any value which doesn't exist already in that other table. A primary key lets you uniquely identify a certain row in one table. A foreign key lets you identify a certain row in *another table*, that this table ultimately depends on. In personalDataKeys I store which fields will be required for a given store as it relates to personal data. For example, if there is a pharmacy with users 'doctors' and 'patients', certain fields in personalData will be required for one but not the other, and this needs to be inserted into personalDataKeys. My concern here is that you're making *columns* which are store-dependent, such as sometimes in one store, personalData will have a column/field named foo, but another store it won't use foo but instead use bar, depending on how the store itself is configured. I'd refer you back to my previous email which described two schemes to record store-dependant data: one using a separate table for each store type, another using a generic key/value table. Having one big table with a big mix of columns that various store configurations pick and choose seems like a very inflexible design. Additionally (if you do keep this design), these two tables you described seem to make the columns that are used tied to *users*, not *stores*. The key for personalDataKeys is (Store, User): but isn't it the case that for a certain kind of store (i.e., a pharmacy), /all/ users in that store will have the same fields in personalData be relevant? So shouldn't personalDataKeys really be storeDataKeys? I.e., a configuration of the store itself of what data keys it considers relevant. All of this, however, obviously happens before any data is actually entered into either personalData or products. Yeah, the constraints and such happen before data is entered. But once they are created, you have to actual enter data in the correct order. The constraints enforce consistency so programmer-error can't introduce data into the tables which is out of whack with the data layout. -- Stephen Hansen ... Also: Ixokai ... Mail: me+list/python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/ signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Issues compiling 2.6.5 on AIX 6.1
On 07/08/2010 04:36 PM, Stopp, Bryan wrote: I’ve seen other threads on this issue, but the resolution still doesn’t seem to exist for me. I’m running the configure script with these parameters: ./configure --prefix=/build/tools \ --exec-prefix=/build/tools \ --enable-shared \ --enable-ipv6 \ --with-gcc \ --with-pth I also want to state that I already edited all of the configure config.guess scripts to comprehend AIX6.1 (they’re not updated yet). This smells horribly like rather risky business. I don't know what you changed or how experienced you are when it comes to editing these auto-generated, system-specific files. Have you tried with Python 2.7, or do you need 2.6? Python 2.7 might work here? Since configure and config.guess are auto-generated, maybe you can regenerate them with a newer/updated version of autoconf that supports your system. Quite frankly, it surprises me that changes to configure would be necessary. Is the system that unusual? What happens if you --disable-shared ? I expect you want the shared library, but - does it work? Just some random thoughts -- I know nothing about AIX and little about cross-UNIX portability beyond, to an extent, GNU vs BSD. Cheers, Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
On 7/7/10 11:52 AM, Stephen Hansen wrote:
On 7/7/10 11:38 AM, Victor Subervi wrote:
Hi;
I have this code:
sql = 'insert into personalDataKeys values (%s, %s, %s)' % (store,
user, ', %s'.join('%s' * len(col_vals))
cursor.execute(sql, col_vals)
First, its always best to be explicit with insert statements. Meaning,
don't rely on the underlining structure of a table, as in:
INSERT INTO YourRandomTable VALUES (my, value, here);
Instead, do:
INSERT INTO YourRandomTable (field1, field2, field3) VALUES (my,
value, here);
I suddenly feel a need to come back and explain *why* I make the claim
to 'best' above: the explicit naming of the fields in the INSERT,
specifically, since others have shown how to do the INSERT safely while
keeping the essentially variable number of items in the values clause.
I still would advise against that approach even if it is safe from a SQL
Injection standpoint: but for a different reason entirely, that of
long-term maintainability.
No design is perfect; no customer specification (no matter how vetted,
analyzed, and approved by stakeholders) survives implementation and
real-life usage.
If you always select specific columns in a specific order (i.e., always
SELECT this, that, other; and never SELECT *), and always insert with
your columns specified (i.e., always INSERT INTO blah (this, that,
other) and never INSERT INTO blah VALUES (..)), then it lets you adapt
your application in the future when something comes up.
Specifically, it lets you add new columns without breaking everything :)
Now, those new columns would need to either allow NULL's or have a
default value of course.
But some day down the road you can go and do an ALTER TABLE to add say,
my_whatever to the above, and you don't suddenly have to vet every
single piece of code which accesses that table. All the existing code
will still work: its getting the pieces of data it knows how to use. As
you need to, you can adjust that code to take into account this new
piece of data. But by making any new additions optional in your SQL,
and making all your other accesses explicit, it just eases migration and
maintenance in future updates.
Some may disagree, I dunno. I just find in my experience that following
that practice has saved a lot of time and effort down the road.
(Especially during migrations from old versions to new versions, and
running versions concurrently during some test-phase, etc, or rolling
back a new version if a critical bug is found: the changes made to the
database to support the new versions can safely persist without you
having to do a much more expensive / time-consuming restoration of the
database from a backup).
--
Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/
signature.asc
Description: OpenPGP digital signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: Debugging a segmentation fault
dierkerdm...@mail.com dierkerdm...@mail.com writes: my python project crashes in a non reproducible way. With gdb I got the backtraces given below. How can I possibly figure out the reason for the segfaults that occur under Linux and Windows, using Python 2.6 in both cases. It's a big C program and you have to debug it as one. Well, first of all try upgrading to the newest versions and see if there is already a fix. Be especially suspicious of any C extension modules you're using. There are also some compile time options that let you rebuild Python with extra consistency checks for reference counts. Turn those on and see what happens. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 11:36 am, Mark Dickinson dicki...@gmail.com wrote: I think that's because we're talking at cross-purposes. To clarify, suppose you want to compute some value (pi; log(2); AGM(1, sqrt(2)); whatever...) to 1000 significant decimal places. Then typically the algorithm (sometimes known as Ziv's onion-peeling method) looks like: (1) Compute an initial approximation to 1002 digits (say), with known absolute error (given by a suitable error analysis); for the sake of argument, let's say that you use enough intermediate precision to guarantee an absolute error of 0.6 ulps. (2) Check to see whether that approximation unambiguously gives you the correctly-rounded 1000 digits that you need. (3) If not, increase the target precision (say by 3 digits) and try again. It's the precision increase in (3) that I was calling small, and similarly it's step (3) that isn't usually needed more than once or twice. (In general, for most functions and input values; I dare say there are exceptions.) Step (1) will often involve using significantly more than the target precision for intermediate computations, depending very much on what you happen to be trying to compute. IIUC, it's the extra precision in step (1) that you don't want to call 'small', and I agree. IOW, I'm saying that the extra precision required *due to the table- maker's dilemma* generally isn't a concern. Yes, though I think attributing only the precision added in step 3 to the table-maker's dilemma isn't entirely correct. While it'd be certainly less of a dilemma if we could precompute the necessary precision, it doesn't' help us if the precision is generally unbounded. As such, I think it's really two dilemmas for the price of one. I actually agree with much of what you've said. It was just the impossible claim that went over the top (IMO). The MPFR library amply demonstrates that computing many transcendental functions to arbitrary precision, with correctly rounded results, is indeed possible. That's because you're latching onto that word instead of the whole sentence in context and making a much bigger deal out of than is appropriate. The fact that I may not be able to complete a given calculation for an arbitrary precision is not something that can be ignored. It's the same notional problem with arbitrary-precision integers: is it better to run out of memory or overflow the calculation? The answer, of course, is a trick question. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 2:59 pm, Stefan Krah stefan-use...@bytereef.org wrote:
pow() is trickier. Exact results have to be weeded out before
attempting the correction loop for correct rounding, and this is
complicated.
For example, in decimal this expression takes a long time (in cdecimal
the power function is not correctly rounded):
Decimal('100.0') ** Decimal('-557.71e-74288')
Hmm. So it does. Luckily, this particular problem is easy to deal
with. Though I dare say that you have more up your sleeve. :)?
--
Mark
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 12:38 pm, Zooko O'Whielacronx zo...@zooko.com wrote: On Thu, Jul 8, 2010 at 4:58 AM, Adam Skutt ask...@gmail.com wrote: I can't think of any program I've ever written where the inputs are actually intended to be decimal. Consider a simple video editing program, and the user specifies a frame rate 23.976 fps. Is that what they really wanted? No, they wanted 24000/1001 but didn't feel like typing that. Okay, so there was a lossy conversion from the user's intention (24000/1001) to what they typed in (23.976). instr = '23.976' Now as a programmer you have two choices: 1. accept what they typed in and losslessly store it in a decimal: from decimal import Decimal as D x = D(instr) print x 23.976 2. accept what they typed in and lossily convert it to a float: x = float(instr) print %.60f % (x,) 23.97509050529822707176208496093750 option 2 introduces further error between what you have stored in your program and what the user originally wanted and offers no advantages except for speed, right? No, you have a third choice, and it's the only right choice: 3. Convert the input to user's desired behavior and behave accordingly. Anything else, here, will result in A/V sync issues. Which is really my point, just because we write '23.976' on the command-line doesn't necessarily mean that's what we meant. Humans are pretty lazy, and we write rational numbers as incomplete decimals all of the time. But this is not a disadvantage of decimal compared to float is it? These problems affect both representations. Although perhaps they affect them differently, I'm not sure. I think sometimes people conflate the fact that decimals can easily have higher and more variable precision than floats with the fact that decimals are capable of losslessly storing decimal values but floats aren't. No, it's not a specific disadvantage of decimal compared to float. I'm not sure why David C. choose to phrase it in those specific terms, though I'm not sure it matters all that much. What I believe is one must understand that the underlying issues are fundamental, and the only way to solve the issues is to educate programmers so they can write code that behaves correctly in the face of rounding. And I do believe you're correct that programmers frequently see one desirable behavior of decimal FP over binary FP and therefore assume all the badness must have gone away. Adam -- http://mail.python.org/mailman/listinfo/python-list
Re: How to test/troubshoot an extension (pylibconfig)?
On 2010-07-08, Grant Edwards inva...@invalid.invalid wrote: On 2010-07-07, Grant Edwards inva...@invalid.invalid wrote: Oops. Those Python bindings are for version 1.3.2 of libconfig (which does work). They don't work with the current version of libconfig. Python 2.6.5 (release26-maint, Jun 22 2010, 12:58:11) [GCC 4.3.4] on linux2 Type help, copyright, credits or license for more information. import pylibconfig conf = pylibconfig.Config() *** glibc detected *** /usr/bin/python2.6: corrupted double-linked list: 0x08065c48 *** Am I correct in concluding it has to be a memory corruption problem in the library itself? Nope. That problem was cause by having two versions of the library installed -- one under /usr and the other under /usr/local. -- Grant Edwards grant.b.edwardsYow! Gee, I feel kind of at LIGHT in the head now, gmail.comknowing I can't make my satellite dish PAYMENTS! -- http://mail.python.org/mailman/listinfo/python-list
ANN: ActivePython 2.6.5.14 is now available
We are pleased to announce the availability of ActivePython 2.6.5.14. http://www.activestate.com/activepython This is a minor release with several bug fixes. As usual, it includes an updated Python Package Manager (PyPM) with updates to essential packages such as Distribute (a compatible fork of setuptools), virtualenv, pip and SQLAlchemy. See the release notes for full details: http://docs.activestate.com/activepython/2.6/relnotes.html#changes What is ActivePython? - ActivePython is ActiveState's binary distribution of Python. Builds for Windows, Mac OS X, Linux are made freely available. Solaris, HP-UX and AIX builds, and access to older versions are available in ActivePython Business, Enterprise and OEM editions: http://www.activestate.com/python ActivePython includes the Python core and the many core extensions: zlib and bzip2 for data compression, the Berkeley DB (bsddb) and SQLite (sqlite3) database libraries, OpenSSL bindings for HTTPS support, the Tix GUI widgets for Tkinter, ElementTree for XML processing, ctypes (on supported platforms) for low-level library access, and others. The Windows distribution ships with PyWin32 -- a suite of Windows tools developed by Mark Hammond, including bindings to the Win32 API and Windows COM. ActivePython 2.6 and 2.7 also include a binary package manager for Python (PyPM) that can be used to install packages much easily. For example: C:\pypm install mysql-python [...] C:\python import MySQLdb See this page for full details: http://docs.activestate.com/activepython/2.6/whatsincluded.html As well, ActivePython ships with a wealth of documentation for both new and experienced Python programmers. In addition to the core Python docs, ActivePython includes the What's New in Python series, Dive into Python, the Python FAQs HOWTOs, and the Python Enhancement Proposals (PEPs). An online version of the docs can be found here: http://docs.activestate.com/activepython/2.6/ We would welcome any and all feedback to: activepython-feedb...@activestate.com Please file bugs against ActivePython at: http://bugs.activestate.com/enter_bug.cgi?product=ActivePython On what platforms does ActivePython run? ActivePython includes installers for the following platforms: - Windows/x86 - Windows/x64 (aka AMD64) - Mac OS X - Linux/x86 - Linux/x86_64 (aka AMD64) - Solaris/SPARC (Business, Enterprise or OEM edition only) - Solaris/x86 (Business, Enterprise or OEM edition only) - HP-UX/PA-RISC (Business, Enterprise or OEM edition only) - HP-UX/IA-64 (Enterprise or OEM edition only) - AIX/PowerPC (Business, Enterprise or OEM edition only) - AIX/PowerPC 64-bit (Business, Enterprise or OEM edition only) Custom builds are available in Enterprise Edition: http://www.activestate.com/enterprise-edition Thanks, and enjoy! The Python Team -- Sridhar Ratnakumar sridharr at activestate.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Debugging a segmentation fault
my python project crashes in a non reproducible way. With gdb I got the backtraces given below. How can I possibly figure out the reason for the segfaults that occur under Linux and Windows, using Python 2.6 in both cases. One of your third party C extension has a reference count bug. It looks like it has an Py_INCREF() to little or a Py_DECREF() too much. The first segfaults occurs in a decref macro: Objects/dictobject.c:911 Py_XDECREF(ep-me_value); , the second is a bit more obscure and hidden in the cyclic GC. The error is either introduced by the same reference counting bug or in error in the type definition and initialization. Which third party products with C extensions do you use? Have you updated your database adapter and NumPy yet? Christian -- http://mail.python.org/mailman/listinfo/python-list
Call for Applications - PSF Sponsored Sprints
The PSF is happy to open our first call for applications for sprint funding! Have you ever had a group of people together to hack towards a common goal? You've hosted a sprint! Have you ever wanted to get a group of like minded Pythonistas together to hack for a day? You're going to want to hold a sprint! Whether you call them Sprints, Hackfests, Hack-a-thons, or any other name, they're a great way to hang out with like-minded developers and work on common code. Sprints are an unbeatable way to build friendships and contacts that will last for years to come, and they're a great way to learn about something new if you're just starting out. The Python Software Foundation has set aside funds to be distributed to world-wide sprint efforts. We're anticipating 2-3 events per month focused on covering topics to help the entire community: - Python Core bug triage and patch submission (on-boarding new contributors) - Python Core documentation (including process documentation) improvements - Porting libraries/applications to Python 3 - Python website/wiki content improvements - PyPI packaging hosting site improvements - Contribution to other core projects, such as packaging related issues. If you are interested in holding a sprint on any of the topics above and you're looking for some money to help out with sprint costs, we can help (up to a max of $250 USD). Prepare an application including the following information: - Date and Location: Where will the event be? What day and time? - Organizers: Who are the event organizers and sprint coach? Is the sprint being run by a Python user group? - Attendees: How many participants do you expect? - Goal: What is the focus and goal of the sprint? - Budget: How much funding you are requesting, and what will you use it for? - Applications should be sent to: spri...@python.org with the subject Sprint Funding Application - location We encourage anyone - even those who have never held, or been to a sprint - to consider holding one. We will help you as much as we can with welcome packets, advertising, and hooking you up with required resources - anything to make it possible. As part of being approved, the you will need to agree to deliver a report (hopefully, with pictures!) of the sprint to the Sprint Committee, so we can post it on the sprint blog and site: http://www.pythonsprints.com If you have any questions or need more information, contact us by email at spri...@python.org. More information is up on our blog: http://pythonsprints.com/2010/07/8/call-applications-now-open/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Is This Open To SQL Injection?
I've come to the realization that I don't need FKs at all here. Essentially, what I need to do is consult personalDataKeys simply to determine what data should be loaded into and retrieved from personalData. I was mistaken because the data are not interdependent, it only appeared that way superficially. No, the data aren't loaded into one big table. There are about 10 fields each in these tables. Yes, I should rename it to storeUserDataKeys. Thanks again, Stephen. beno -- http://mail.python.org/mailman/listinfo/python-list
pySimpleSessions - PHP sessions implemented in Python
Hey, since there is no standalone sessions module for python (at least a properly working one), I created one and thought i'd share it with you. This is the project: http://code.google.com/p/pysimplesessions/ This is the introduction: http://code.google.com/p/pysimplesessions/wiki/Introduction And finally the module source: http://code.google.com/p/pysimplesessions/source/browse/trunk/sessions.py If you need, there are some examples (partly working *g*): http://code.google.com/p/pysimplesessions/source/browse/trunk/#trunk/examples It would be great if I could get some feedback from you. ;) cheers, -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
Adam Skutt ask...@gmail.com wrote: I actually agree with much of what you've said. It was just the impossible claim that went over the top (IMO). The MPFR library amply demonstrates that computing many transcendental functions to arbitrary precision, with correctly rounded results, is indeed possible. That's because you're latching onto that word instead of the whole sentence in context and making a much bigger deal out of than is appropriate. The fact that I may not be able to complete a given calculation for an arbitrary precision is not something that can be ignored. It's the same notional problem with arbitrary-precision integers: is it better to run out of memory or overflow the calculation? The answer, of course, is a trick question. In the paper describing his strategy for correct rounding Ziv gives an estimate for abnormal cases, which is very low. This whole argument is a misunderstanding. Mark and I argue that correct rounding is quite feasible in practice, you argue that you want guaranteed execution times and memory usage. This is clear now, but was not so apparent in the impossible paragraph that Mark responded to. I think asking for strictly bounded resource usage is reasonable. In cdecimal there is a switch to turn off correct rounding for exp() and log(). Stefan Krah -- http://mail.python.org/mailman/listinfo/python-list
RE: Issues compiling 2.6.5 on AIX 6.1
These are good points, but I've fixed the configure config.guess on about 10 other OSS projects that I needed to compile on this machine. So I know that my changes are correct. The changes boil down to: There are a number of lines in the files that reference AIX versions for configuring details about the system. Specifically, it'll be if or case statements that look like: *-ibm-aix5*) *aix4*|*aix5*) I just add the aix6 clause: *-ibm-aix5* | *-ibm-aix6*) *aix4*|*aix5*|*aix6*) As Aix6 is binary compatible with 5, but the configure defaults it to an unknown AIX version and completely mis-configures the compiler. As for using the autoconf tool for my system, I'm not really sure if it's even installed or if the latest autoconf supports this version. Oddly enough I'm not the sys admin, but I do play one on TV. I do want to stick with Python 2.6 if I can, but I will give compiling 2.7 a shot to see what, if anything, changes/is fixed. Thanks for the feedback! -B -Original Message- From: python-list-bounces+cbds=argushealth@python.org [mailto:python-list-bounces+cbds=argushealth@python.org] On Behalf Of Thomas Jollans Sent: Thursday, July 08, 2010 11:51 AM To: python-list@python.org Subject: Re: Issues compiling 2.6.5 on AIX 6.1 On 07/08/2010 04:36 PM, Stopp, Bryan wrote: I've seen other threads on this issue, but the resolution still doesn't seem to exist for me. I'm running the configure script with these parameters: ./configure --prefix=/build/tools \ --exec-prefix=/build/tools \ --enable-shared \ --enable-ipv6 \ --with-gcc \ --with-pth I also want to state that I already edited all of the configure config.guess scripts to comprehend AIX6.1 (they're not updated yet). This smells horribly like rather risky business. I don't know what you changed or how experienced you are when it comes to editing these auto-generated, system-specific files. Have you tried with Python 2.7, or do you need 2.6? Python 2.7 might work here? Since configure and config.guess are auto-generated, maybe you can regenerate them with a newer/updated version of autoconf that supports your system. Quite frankly, it surprises me that changes to configure would be necessary. Is the system that unusual? What happens if you --disable-shared ? I expect you want the shared library, but - does it work? Just some random thoughts -- I know nothing about AIX and little about cross-UNIX portability beyond, to an extent, GNU vs BSD. Cheers, Thomas -- http://mail.python.org/mailman/listinfo/python-list PRIVILEGED AND CONFIDENTIAL This email transmission contains privileged and confidential information intended only for the use of the individual or entity named above. If the reader of the email is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any use, dissemination or copying of this email transmission is strictly prohibited by the sender. If you have received this transmission in error, please delete the email and immediately notify the sender via the email return address or mailto:postmas...@argushealth.com. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
Mark Dickinson dicki...@gmail.com wrote:
On Jul 8, 2:59 pm, Stefan Krah stefan-use...@bytereef.org wrote:
pow() is trickier. Exact results have to be weeded out before
attempting the correction loop for correct rounding, and this is
complicated.
For example, in decimal this expression takes a long time (in cdecimal
the power function is not correctly rounded):
Decimal('100.0') ** Decimal('-557.71e-74288')
Hmm. So it does. Luckily, this particular problem is easy to deal
with. Though I dare say that you have more up your sleeve. :)?
Not at the moment, but I'll keep trying. :)
Stefan Krah
--
http://mail.python.org/mailman/listinfo/python-list
RE: Issues compiling 2.6.5 on AIX 6.1
I just wanted to follow up my previous email: I tried compiling 2.7 (without editing any config.guess or configure files as they are up to date for AIX6) and it failed with the exact same errors. So I'm still stuck and not sure what I should to do get this to compile. Any other ideas out there? -B -Original Message- From: python-list-bounces+cbds=argushealth@python.org [mailto:python-list-bounces+cbds=argushealth@python.org] On Behalf Of Thomas Jollans Sent: Thursday, July 08, 2010 11:51 AM To: python-list@python.org Subject: Re: Issues compiling 2.6.5 on AIX 6.1 On 07/08/2010 04:36 PM, Stopp, Bryan wrote: I've seen other threads on this issue, but the resolution still doesn't seem to exist for me. I'm running the configure script with these parameters: ./configure --prefix=/build/tools \ --exec-prefix=/build/tools \ --enable-shared \ --enable-ipv6 \ --with-gcc \ --with-pth I also want to state that I already edited all of the configure config.guess scripts to comprehend AIX6.1 (they're not updated yet). This smells horribly like rather risky business. I don't know what you changed or how experienced you are when it comes to editing these auto-generated, system-specific files. Have you tried with Python 2.7, or do you need 2.6? Python 2.7 might work here? Since configure and config.guess are auto-generated, maybe you can regenerate them with a newer/updated version of autoconf that supports your system. Quite frankly, it surprises me that changes to configure would be necessary. Is the system that unusual? What happens if you --disable-shared ? I expect you want the shared library, but - does it work? Just some random thoughts -- I know nothing about AIX and little about cross-UNIX portability beyond, to an extent, GNU vs BSD. Cheers, Thomas -- http://mail.python.org/mailman/listinfo/python-list PRIVILEGED AND CONFIDENTIAL This email transmission contains privileged and confidential information intended only for the use of the individual or entity named above. If the reader of the email is not the intended recipient or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any use, dissemination or copying of this email transmission is strictly prohibited by the sender. If you have received this transmission in error, please delete the email and immediately notify the sender via the email return address or mailto:postmas...@argushealth.com. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: Lua is faster than Fortran???
On Jul 4, 5:58 pm, John Nagle na...@animats.com wrote: TheUnladenSwallowpeople should in theory be able to reach that level of performance. (Both groups are employed at Google. So their effectiveness will be compared.) John Nagle No. Collin Winter said that they will never be as fast as Chrome's V8 or similar JS engines, since they were created from scratch to be super fast above all else. On the other hand, US is a project to enhance an existing interpreter, carrying a lot of the burden of early design decisions. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Thu, Jul 8, 2010 at 11:22 AM, Adam Skutt ask...@gmail.com wrote: On Jul 8, 12:38 pm, Zooko O'Whielacronx zo...@zooko.com wrote: Now as a programmer you have two choices: … 1. accept what they typed in and losslessly store it in a decimal: … 2. accept what they typed in and lossily convert it to a float: No, you have a third choice, and it's the only right choice: 3. Convert the input to user's desired behavior and behave accordingly. Anything else, here, will result in A/V sync issues. Okay, please tell me about how this is done. I've never dealt with this sort of issue directly. As far as I can imagine, any way to implement option 3 will involve accepting the user's input and storing it in memory somehow and then computing on it. As far as I can tell, doing so with a decimal instead of a float will never be worse for your outcome and will often be better. But, please explain in more detail how this works. Thanks! Regards, Zooko -- http://mail.python.org/mailman/listinfo/python-list
Python Multi-Channel Audio
I'm looking for some module or system of modules that can help me do a few things with audio 1. Playback of files (at least .wavs, other codecs would be nice but, hey, converting to a wav is easy) 2. Seek within the file 3. Control volume 3. Do all of these things by channel, e.g. play sound effect 1 on channels 12 at equal volume and play effect #2 on only channel 2 I've got a setup with gstreamer and alsa right now, but I don't have any way of getting control of individual channels at the gstreamer level. I can easily do it at the alsa level and just manipulate the master left/right output, but that's not very helpful if it modifies all currently playing effects. I'm not even sure if this is possible to do with python, butsome pointers would really be great. (For the curious: This is part of the audio system for a larger immersive theater project, ) -- http://mail.python.org/mailman/listinfo/python-list
Re: How is Unladen Swallow coming along?
On Jul 8, 1:42 pm, John Nagle na...@animats.com wrote: How is Unladen Swallow coming along? Looking at the site, code is being checked in and issues are being reported, but the last quarterly release was 2009 Q3. They missed their January 2010 release date for 2009 Q4, so they're now about 6 months behind their project plan. (http://code.google.com/p/unladen-swallow/wiki/ProjectPlan;) John Nagle Don't be shy. Ask this question in Unladen Swallow's google group. They don't bite! Luis -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On Jul 8, 2:00 pm, Stefan Krah stefan-use...@bytereef.org wrote: This whole argument is a misunderstanding. Mark and I argue that correct rounding is quite feasible in practice, you argue that you want guaranteed execution times and memory usage. This is clear now, but was not so apparent in the impossible paragraph that Mark responded to. No, that's what I'm arguing for, though such a feature is important. I'm pointing out that the feasibility of correct rounding is entirely dependent on what you're doing. For IEEE-754 double, it's feasible for the elementary functions if you can tolerate intermediate calculations that are more than twice as large as your double in the corner cases. Certainly, for a single calculation, this is acceptable, but at how many calculations is it no longer acceptable? Adam -- http://mail.python.org/mailman/listinfo/python-list
Question about odd code in libconfig bindings
I'm playign with Python bindings for libconfig, and one of the methods
does something that seems very odd to me. The purpose of the method
is to fetch the value found a a particular path within the
configuration namespace.
If the path exists and has a scalar value the method returns the tuple
(value,True). Otherwise it returns ('',False). AFAICT, the latter
happens in either of two cases: the path doesn't exist at all, or the
path exists but doesn't have a scalar value (it's an aggretate object
such as a list, group, or array).
This seems distinctly un-pythonic.
I would think that the right thing to do would be to either return the
requested value or raise an exception. The libconfig C++ API defines
a SettingNotFound exception and SettingTypeException that seem
appropriate in the two failure cases.
Is there something about implementing bindings using Boost that makes
this sort of (value,success) return tuple a desirable way to do
things?
FWIW, here's the method I find puzzling:
tuple value ( const char * path )
{
std::string v_string;
if ( config-lookupValue ( path, v_string ) )
return make_tuple ( v_string.c_str(), true );
unsigned int v_uint;
if ( config-lookupValue ( path, v_uint ) )
return make_tuple ( v_uint, true );
int v_int;
if ( config-lookupValue ( path, v_int ) )
return make_tuple ( v_int, true );
bool v_bool;
if ( config-lookupValue ( path, v_bool ) )
return make_tuple ( v_bool, true );
unsigned long long v_ulonglong;
if ( config-lookupValue ( path, v_ulonglong ) )
return make_tuple ( v_ulonglong, true );
long long v_longlong;
if ( config-lookupValue ( path, v_longlong ) )
return make_tuple ( v_longlong, true );
float v_float;
if ( config-lookupValue ( path, v_float ) )
return make_tuple ( v_float, true );
double v_double;
if ( config-lookupValue ( path, v_double ) )
return make_tuple ( v_double, true );
return make_tuple ( , false );
}
--
Grant Edwards grant.b.edwardsYow! Don't SANFORIZE me!!
at
gmail.com
--
http://mail.python.org/mailman/listinfo/python-list
is not operator?
What happens here? Does Python (2.6.5) have an is not operator? a = 5 print (a is not False) True print (a is (not False)) False print (not (a is False)) True It seems y is not x fits well with spoken English, but it is also a bit surprising that y is not x does not mean y is (not x) but not (y is x). Why does Python reorder is and not operators, and what are the formal rules for this behavior? -- http://mail.python.org/mailman/listinfo/python-list
Re: Lua is faster than Fortran???
On 4 Jul, 21:59, Stefan Behnel stefan...@behnel.de wrote: I have already said I don't care about unladen swallow. What I meant, was: which of these benchmarks would have to be better to make you care? Because your decision not to care seems to be based on exactly these benchmarks. Those are the only one I've seen. -- http://mail.python.org/mailman/listinfo/python-list
Re: is not operator?
On 7/8/10 4:10 PM, sturlamolden wrote: What happens here? Does Python (2.6.5) have an is not operator? Yes. From Grammar/Grammar: comp_op: ''|''|'=='|'='|'='|''|'!='|'in'|'not' 'in'|'is'|'is' 'not' -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: is not operator?
On Thu, 2010-07-08 at 13:10 -0700, sturlamolden wrote: What happens here? Does Python (2.6.5) have an is not operator? a = 5 print (a is not False) True print (a is (not False)) False print (not (a is False)) True It seems y is not x fits well with spoken English, but it is also a bit surprising that y is not x does not mean y is (not x) but not (y is x). Why does Python reorder is and not operators, and what are the formal rules for this behavior? Don't forget about the similar not in, as in: 'a' not in 'abc' False This is probably the section of documentation you want: http://docs.python.org/reference/expressions.html#notin -- John Krukoff jkruk...@ltgc.com Land Title Guarantee Company -- http://mail.python.org/mailman/listinfo/python-list
Re: is not operator?
On 8 Jul, 22:32, Robert Kern robert.k...@gmail.com wrote: What happens here? Does Python (2.6.5) have an is not operator? Yes. From Grammar/Grammar: comp_op: ''|''|'=='|'='|'='|''|'!='|'in'|'not' 'in'|'is'|'is' 'not' Thanks :) -- http://mail.python.org/mailman/listinfo/python-list
Re: How is Unladen Swallow coming along?
On 7/8/2010 12:19 PM, Luis M. González wrote: On Jul 8, 1:42 pm, John Naglena...@animats.com wrote: How is Unladen Swallow coming along? Looking at the site, code is being checked in and issues are being reported, but the last quarterly release was 2009 Q3. They missed their January 2010 release date for 2009 Q4, so they're now about 6 months behind their project plan. (http://code.google.com/p/unladen-swallow/wiki/ProjectPlan;) John Nagle Don't be shy. Ask this question in Unladen Swallow's google group. They don't bite! Found this: http://www.python.org/dev/peps/pep-3146/#performance-retrospective; It's starting to work, but the performance improvement is tiny, well under 2x faster than CPython. Only 1.08x on html5lib. That's far less than they expected. They were going for 5x, which is far less than Shed Skin (which restricts Python) already achieves. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: is not operator?
On 8 Jul, 22:29, John Krukoff jkruk...@ltgc.com wrote: Don't forget about the similar not in, as in: I noticed that in the grammar Robert posted. It never occurred to me as being a special operator too, but it is. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python -- floating point arithmetic
On 8 Jul., 15:10, Ethan Furman et...@stoneleaf.us wrote: Interesting. I knew when I posted my above comment that I was ignoring such situations. I cannot comment on the code itself as I am unaware of the algorithm, and haven't studied numbers extensively (although I do find them very interesting). So while you've made your point, I believe mine still stands -- comparing floats using == to absolute numbers is almost always a mistake. JFTR, it works because a+b == a+b (while I don't think that a+b == b+a holds for all a and b). In general, I totally agree with you. -- http://mail.python.org/mailman/listinfo/python-list
object exported through manager from multiprocess module
Hi all,
have troubles with exporting objects through managers from multiprocess
module, see example:
Worker.py:
###
from multiprocessing import Process
from multiprocessing.managers import BaseManager
import pcapy
from impacket.ImpactDecoder import EthDecoder
__all__ = ['Worker']
class Worker(Process):
'''
Class for sniffing packets, runnig as root
'''
public = ['go', 'terminate']
def __init__(self):
super(Worker, self).__init__()
self.iface = ''
self.expr = ''
self.pcap = ''
# define packet decoder
self.decoder = EthDecoder()
# key for queue daemon, remotely on localhost:5000
self._keyQ = '10b222970537b97919db36ec757370d2'
class QueueManager(BaseManager): pass
QueueManager.register('get_dataQueue')
self._m = QueueManager(address=('127.0.0.1', 5000),
authkey=self._keyQ)
self._m.connect()
self.dataQueue = self._m.get_dataQueue()
def go(self, iface, expr):
'''
start sniffer
'''
print Starting sniffer
self.iface = iface
self.expr = expr
super(Worker, self).start()
def terminate(self):
'''
terminate sniffer
'''
super(Worker, self).terminate()
def run(self):
print sniffing ...
print self.iface
print self.expr
self.pcap = pcapy.open_live(self.iface, 1500, 1, 0)
self.pcap.setfilter(self.expr)
self.pcap.loop(0, self.__packetHandler)
print ... done
def __packetHandler(self, hdr, data):
'''
handles packets and put them in to the queue
'''
print Handling packets
#print data
print Queue size: %i % self.dataQueue.qsize()
print self.decoder.decode(data)
self.dataQueue.put(data)
Export object (Worker):
###
from Worker import Worker
class SniffManager(BaseManager): pass
SniffManager.register('Worker', callable=Worker)
Sm = SniffManager(address=('127.0.0.1', 5001),
authkey='f1f16683f3e0208131b46d37a79c8921')
Ss = Sm.get_server()
Ss.serve_forever()
Call object methods remotely:
###
# get remote object
class WorkerManager(BaseManager): pass
WorkerManager.register('Worker')
w = WorkerManager(address=('127.0.0.1', 5001),
authkey='f1f16683f3e0208131b46d37a79c8921')
w.connect()
worker = w.Worker()
worker.go(iface=ethx, expr=whatever) # WORKS FINE
but
worker.terminate()
File /home/tom/web2py/applications/init/controllers/sniffer.py, line
143, in index
worker.terminate()
File string, line 2, in terminate
File /usr/lib/python2.6/multiprocessing/managers.py, line 740, in
_callmethod
raise convert_to_error(kind, result)
AttributeError: 'NoneType' object has no attribute 'terminate'
Which is strange from my point of view, don't you think?
Thanks for advices,
cheers
--
Tomas Pelka
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Multi-Channel Audio
On 07/08/2010 09:17 PM, Alex Karpinski wrote: I'm looking for some module or system of modules that can help me do a few things with audio 1. Playback of files (at least .wavs, other codecs would be nice but, hey, converting to a wav is easy) 2. Seek within the file 3. Control volume 3. Do all of these things by channel, e.g. play sound effect 1 on channels 12 at equal volume and play effect #2 on only channel 2 I've got a setup with gstreamer and alsa right now, but I don't have any way of getting control of individual channels at the gstreamer level. I can easily do it at the alsa level and just manipulate the master left/right output, but that's not very helpful if it modifies all currently playing effects. I'm not even sure if this is possible to do with python, butsome pointers would really be great. First of all, allow me to point to a something I hacked together (mostly) a few weeks ago: http://bitbucket.org/jollybox/pyaudiogen/wiki/Home It's something of a library, VERY minimal/skeletal at the moment, for audio generation (and/or processing) in Python. No idea if this is of any help to you. Python 3 only. Multiple channel support is there, though to be useful it'd still need a bit of glue. Not much to see there yet all in all. (For the curious: This is part of the audio system for a larger immersive theater project, ) Though really, what I think you want is jack. It basically allows you to plug things together in any way, and I'm sure you can control the volume of individual plugs. I expect there is a Python API, but I haven't checked. -- http://mail.python.org/mailman/listinfo/python-list
ANN: winreg_unicode 0.5.0
I'm pleased to announce the release of winreg_unicode 0.5.0, the first release of winreg_unicode. The winreg_unicode package aims to be a drop-in replacement for Python 2's _winreg module. However, it returns unicode values where possible, similar to Python 3's winreg module. To illustrate the need for the winreg_unicode package, suppose an application must query the registry to discover a filename and the registry contains the string međuresorna.txt. Python 2's _winreg module will return meduresorna.txt instead, which is not the actual name of the file. The winreg_unicode package does not yet contain all of _winreg's functions. In particular, functions that write to the registry are not yet included. Code contributions are welcome. PyPi: http://pypi.python.org/pypi/winreg_unicode Bug tracker: http://github.com/DanielStutzbach/winreg_unicode/issues Source code: http://github.com/DanielStutzbach/winreg_unicode -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com -- http://mail.python.org/mailman/listinfo/python-list
Re: C interpreter in Lisp/scheme/python
On Thu, 08 Jul 2010 10:39:45 +0200, p...@informatimago.com (Pascal J. Bourguignon) wrote: Nick Keighley nick_keighley_nos...@hotmail.com writes: Nick Keighley nick_keighley_nos...@hotmail.com wrote: Rivka Miller rivkaumil...@gmail.com wrote: Anyone know what the first initial of L. Peter Deutsch stand for ? Laurence according to wikipedia (search time 2s) oops! He was born Laurence but changed it legally to L. including the dot Too bad, Laurence is a nice name. He probably hates the nickname Larry. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Multi-Channel Audio
On Thu, Jul 8, 2010 at 2:11 PM, Thomas Jollans tho...@jollans.com wrote: On 07/08/2010 09:17 PM, Alex Karpinski wrote: I'm looking for some module or system of modules that can help me do a few things with audio 1. Playback of files (at least .wavs, other codecs would be nice but, hey, converting to a wav is easy) 2. Seek within the file 3. Control volume 3. Do all of these things by channel, e.g. play sound effect 1 on channels 12 at equal volume and play effect #2 on only channel 2 I've got a setup with gstreamer and alsa right now, but I don't have any way of getting control of individual channels at the gstreamer level. I can easily do it at the alsa level and just manipulate the master left/right output, but that's not very helpful if it modifies all currently playing effects. I'm not even sure if this is possible to do with python, butsome pointers would really be great. First of all, allow me to point to a something I hacked together (mostly) a few weeks ago: http://bitbucket.org/jollybox/pyaudiogen/wiki/Home It's something of a library, VERY minimal/skeletal at the moment, for audio generation (and/or processing) in Python. No idea if this is of any help to you. Python 3 only. Multiple channel support is there, though to be useful it'd still need a bit of glue. Not much to see there yet all in all. (For the curious: This is part of the audio system for a larger immersive theater project, ) Though really, what I think you want is jack. It basically allows you to plug things together in any way, and I'm sure you can control the volume of individual plugs. I expect there is a Python API, but I haven't checked. Untried: http://sourceforge.net/projects/py-jack/ Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
