Re: [Numpy-discussion] PyArray_DescrConverter - alignment / trailingunused bytes

2006-09-15 Thread Albert Strasheim 
Hello all

In [1]: import numpy as N

In [3]: N.dtype({'names' : ['x', 'y'],
 'formats' : [N.intc, N.float64]},
align=True)
Out[3]: dtype([('x', 'i4'), ('', '|V4'), ('y', 'f8')])

The reason you might not have discovered this:

In [2]: N.dtype?
Type:   type
Base Class: type 'type'
String Form:type 'numpy.dtype'
Namespace:  Interactive
File:   c:\python24\lib\site-packages\numpy\__init__.py
Docstring:
no docstring

Cheers,

Albert

 -Original Message-
 From: [EMAIL PROTECTED] [mailto:numpy-
 [EMAIL PROTECTED] On Behalf Of Martin Wiechert
 Sent: 15 September 2006 16:14
 To: numpy-discussion
 Subject: [Numpy-discussion] PyArray_DescrConverter - alignment /
 trailingunused bytes
 
 Hi list,
 
 I'm using PyArray_DescrConverter with a dict object to create a struct-
 like
 dtype from C.
 As the struct contains different data types I run into unaligned access
 problems.
 Is there a way to force alignment or to get trailing unused bytes in the
 dtpye?
 
 Thanks, Martin


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] PyArray_DescrConverter - alignment / trailingunused bytes

2006-09-15 Thread Martin Wiechert 
On Friday 15 September 2006 17:53, Albert Strasheim wrote:
 Hello all

 In [1]: import numpy as N

 In [3]: N.dtype({'names' : ['x', 'y'],
  'formats' : [N.intc, N.float64]},
 align=True)
 Out[3]: dtype([('x', 'i4'), ('', '|V4'), ('y', 'f8')])

 The reason you might not have discovered this:

 In [2]: N.dtype?
 Type:   type
 Base Class: type 'type'
 String Form:type 'numpy.dtype'
 Namespace:  Interactive
 File:   c:\python24\lib\site-packages\numpy\__init__.py
 Docstring:
 no docstring


Thanks Albert! Do you also know the corresponding C-API function? It cannot be 
PyArray_DescrConverter (PyObject *, PyArray_Descr **), whose signature has no 
align, right?

 Cheers,

 Albert

  -Original Message-
  From: [EMAIL PROTECTED] [mailto:numpy-
  [EMAIL PROTECTED] On Behalf Of Martin Wiechert
  Sent: 15 September 2006 16:14
  To: numpy-discussion
  Subject: [Numpy-discussion] PyArray_DescrConverter - alignment /
  trailingunused bytes
 
  Hi list,
 
  I'm using PyArray_DescrConverter with a dict object to create a struct-
  like
  dtype from C.
  As the struct contains different data types I run into unaligned access
  problems.
  Is there a way to force alignment or to get trailing unused bytes in the
  dtpye?
 
  Thanks, Martin

 -
 Using Tomcat but need to do more? Need to support web services, security?
 Get stuff done quickly with pre-integrated technology to make your job
 easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
 Geronimo
 http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
 ___
 Numpy-discussion mailing list
 Numpy-discussion@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] PyArray_DescrConverter - alignment / trailingunused bytes

2006-09-15 Thread Albert Strasheim 
Hello all

 -Original Message-
 From: [EMAIL PROTECTED] [mailto:numpy-
 [EMAIL PROTECTED] On Behalf Of Martin Wiechert
 Sent: 15 September 2006 19:53
 To: Discussion of Numerical Python
 Subject: Re: [Numpy-discussion]PyArray_DescrConverter - alignment /
 trailingunused bytes
 
 On Friday 15 September 2006 19:14, Travis Oliphant wrote:
  Martin Wiechert wrote:
   Thanks Albert! Do you also know the corresponding C-API function? It
   cannot be PyArray_DescrConverter (PyObject *, PyArray_Descr **), whose
   signature has no align, right?
 
  The DescrConverter function is meant for O-style conversions.  It
  can't accept an align function.  We could possibly add something to the
  converter to allow specification of alignment through the object to be
  converted.
 

 snip

 One final question. To me the repr of a dtype with gaps is a little bit
 puzzling:
 
  dtype ({'names': ['a', 'b', 'c'], 'formats': ['a4', 'f8', 'f4'],
 'offsets': [0, 16, 24]})
 dtype([('a', '|S4'), ('', '|V12'), ('b', 'f8'), ('', '|V12'), ('c',
 'f4')])
 
 There should be no gap between b and c but still the repr has ('',
 '|V12')
 between them. Am I missing something?

For performance reasons, compilers will typically align integers (and
probably floats) on 4-byte boundaries and apparently, doubles on 16-byte
boundaries.

Because compilers align like this, so does NumPy. This allows you to:

1. Take any kind of C struct definition
2. Convert it to a dtype
3. Create a NumPy array with this dtype
4. Pass the array's data pointer to C code
5. Cast the data pointer to a pointer to your C struct
6. Operate on the pointer to struct as if it were allocated in C

Cheers,

Albert


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] PyArray_DescrConverter - alignment/ trailingunused bytes

2006-09-15 Thread Albert Strasheim 
Argh

  snip
 
  One final question. To me the repr of a dtype with gaps is a little bit
  puzzling:
 
   dtype ({'names': ['a', 'b', 'c'], 'formats': ['a4', 'f8', 'f4'],
  'offsets': [0, 16, 24]})
  dtype([('a', '|S4'), ('', '|V12'), ('b', 'f8'), ('', '|V12'), ('c',
  'f4')])
 
  There should be no gap between b and c but still the repr has ('',
  '|V12')
  between them. Am I missing something?

I see you're not specifying align=True here. Ignore my last message.

Cheers,

Albert


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] PyArray_DescrConverter - alignment / trailingunused bytes

2006-09-15 Thread Travis Oliphant 
Martin Wiechert wrote:
 On Friday 15 September 2006 19:14, Travis Oliphant wrote:
   
 Martin Wiechert wrote:
 
 Thanks Albert! Do you also know the corresponding C-API function? It
 cannot be PyArray_DescrConverter (PyObject *, PyArray_Descr **), whose
 signature has no align, right?
   
 The DescrConverter function is meant for O-style conversions.  It
 can't accept an align function.  We could possibly add something to the
 converter to allow specification of alignment through the object to be
 converted.

 

 I begin to see the light

 For dictionaries one could maybe just add an optional key align.
 Also an optional key elsize or itemsize to force the total size of the 
 record may sometimes be useful. E.g. one may want to faithfully map a given C 
 struct. (That's why I was asking for trailing unused bytes.)

 But I understand that other things have higher priority.

   
 Or, you can just call the __new__ method of the PyArrayDescr_Type object

 res = PyObject_CallMethod((PyObject *)PyArrayDescr_Type, __new__,
 Oi, dict_object, 1))

 or call the tp-new method directly:

 args = Py_BuildValue(Oi, dict_object, 1);
 PyArrayDescr_Type-tp_new(PyArrayDescr_Type, args, NULL);
 Py_DECREF(args);

 

 Thank you! I'll try this.

   
 (I think passing in NULL for the keywords is O.K., but I haven't checked
 it).

 -Travis

 

 One final question. To me the repr of a dtype with gaps is a little bit 
 puzzling:

   
 dtype ({'names': ['a', 'b', 'c'], 'formats': ['a4', 'f8', 'f4'], 
 
 'offsets': [0, 16, 24]})
 dtype([('a', '|S4'), ('', '|V12'), ('b', 'f8'), ('', '|V12'), ('c', 'f4')])

 There should be no gap between b and c but still the repr has ('', 
 '|V12') 
 between them. Am I missing something?
   
There was a bug I just fixed in the representation of these structures 
with gaps.  It should be fixed in SVN, now.


-Travis


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] PyArray_DescrConverter - alignment / trailingunused bytes

2006-09-15 Thread Martin Wiechert 
On Friday 15 September 2006 20:27, Travis Oliphant wrote:
 Martin Wiechert wrote:
  On Friday 15 September 2006 19:14, Travis Oliphant wrote:
  Martin Wiechert wrote:
  Thanks Albert! Do you also know the corresponding C-API function? It
  cannot be PyArray_DescrConverter (PyObject *, PyArray_Descr **), whose
  signature has no align, right?
 
  The DescrConverter function is meant for O-style conversions.  It
  can't accept an align function.  We could possibly add something to the
  converter to allow specification of alignment through the object to be
  converted.
 
  I begin to see the light
 
  For dictionaries one could maybe just add an optional key align.
  Also an optional key elsize or itemsize to force the total size of
  the record may sometimes be useful. E.g. one may want to faithfully map a
  given C struct. (That's why I was asking for trailing unused bytes.)
 
  But I understand that other things have higher priority.
 
  Or, you can just call the __new__ method of the PyArrayDescr_Type object
 
  res = PyObject_CallMethod((PyObject *)PyArrayDescr_Type, __new__,
  Oi, dict_object, 1))
 
  or call the tp-new method directly:
 
  args = Py_BuildValue(Oi, dict_object, 1);
  PyArrayDescr_Type-tp_new(PyArrayDescr_Type, args, NULL);
  Py_DECREF(args);
 
  Thank you! I'll try this.
 
  (I think passing in NULL for the keywords is O.K., but I haven't checked
  it).
 
  -Travis
 
  One final question. To me the repr of a dtype with gaps is a little bit
 
  puzzling:
  dtype ({'names': ['a', 'b', 'c'], 'formats': ['a4', 'f8', 'f4'],
 
  'offsets': [0, 16, 24]})
  dtype([('a', '|S4'), ('', '|V12'), ('b', 'f8'), ('', '|V12'), ('c',
  'f4')])
 
  There should be no gap between b and c but still the repr has ('',
  '|V12') between them. Am I missing something?

 There was a bug I just fixed in the representation of these structures
 with gaps.  It should be fixed in SVN, now.


 -Travis



Thanks again.

 -
 Using Tomcat but need to do more? Need to support web services, security?
 Get stuff done quickly with pre-integrated technology to make your job
 easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
 Geronimo
 http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
 ___
 Numpy-discussion mailing list
 Numpy-discussion@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion