[Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Maccarthy, Jonathan K 
Hi everyone,

I've looked in the mailing list archives and with the googles, but haven't yet 
found any hints with this question...

I have a float field in a NumPy record that looks like it's being substituted 
as a string in the Python {:f}.format() mini-language, thus throwing an error:


In [1]: tmp = np.rec.array([('XYZZ', 2001123, -23.82396)], 
dtype=np.dtype([('sta', '|S6'), ('ondate', 'i8'), ('lat', 'f4')]))[0]

In [2]: type(tmp)
Out[3]: numpy.core.records.record

In [3]: tmp
Out[3]: ('XYZZ', 2001123, -23.823917388916016)

In [4]: tmp.sta, tmp.ondate, tmp.lat
Out[4]: ('XYZZ', 2001123, -23.823917)

# strings and integers work
In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp)
Out[5]: 'XYZZ2001123'

# lat is a float, but it seems to be coerced to a string first, and failing
In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
---
ValueErrorTraceback (most recent call last)
/Users/jkmacc/ipython-input-312-bff8066cfde8 in module()
 1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)

ValueError: Unknown format code 'f' for object of type 'str'

# string formatting for doesn't fail
In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6s}'.format(tmp)
Out[7]: 'XYZZ2001123  -23.82'


This also fails:

In [7]: {:f}.format(np.array(3.2, dtype='f4'))
---
ValueErrorTraceback (most recent call last)
/Users/jkmacc/ipython-input-314-33119128e3e6 in module()
 1 {:f}.format(np.array(3.2, dtype='f4'))

ValueError: Unknown format code 'f' for object of type 'str'



Does anyone understand what's happening?

Thanks for your help.

Best,
Jon
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Benjamin Root 
You can treat a record in a record array like a tuple or a dictionary when
it comes to formatting.  So, either refer to the index element you want
formatted as a float, or refer to it by name (in the formatting language).
By just doing {:f}, you are just grabbing the first one, which is XXYYZZ
and trying to format that.  But remember, you can only do this to a single
record at a time, not the entire record array at once.

Regular numpy arrays can not be formatted in this manner, hence your other
attempt failures.

Cheers!
Ben Root


On Thu, Jun 6, 2013 at 3:48 PM, Maccarthy, Jonathan K jkm...@lanl.govwrote:

 Hi everyone,

 I've looked in the mailing list archives and with the googles, but haven't
 yet found any hints with this question...

 I have a float field in a NumPy record that looks like it's being
 substituted as a string in the Python {:f}.format() mini-language, thus
 throwing an error:


 In [1]: tmp = np.rec.array([('XYZZ', 2001123, -23.82396)],
 dtype=np.dtype([('sta', '|S6'), ('ondate', 'i8'), ('lat', 'f4')]))[0]

 In [2]: type(tmp)
 Out[3]: numpy.core.records.record

 In [3]: tmp
 Out[3]: ('XYZZ', 2001123, -23.823917388916016)

 In [4]: tmp.sta, tmp.ondate, tmp.lat
 Out[4]: ('XYZZ', 2001123, -23.823917)

 # strings and integers work
 In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp)
 Out[5]: 'XYZZ2001123'

 # lat is a float, but it seems to be coerced to a string first, and
 failing
 In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
 ---
 ValueErrorTraceback (most recent call last)
 /Users/jkmacc/ipython-input-312-bff8066cfde8 in module()
  1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)

 ValueError: Unknown format code 'f' for object of type 'str'

 # string formatting for doesn't fail
 In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6s}'.format(tmp)
 Out[7]: 'XYZZ2001123  -23.82'


 This also fails:

 In [7]: {:f}.format(np.array(3.2, dtype='f4'))
 ---
 ValueErrorTraceback (most recent call last)
 /Users/jkmacc/ipython-input-314-33119128e3e6 in module()
  1 {:f}.format(np.array(3.2, dtype='f4'))

 ValueError: Unknown format code 'f' for object of type 'str'



 Does anyone understand what's happening?

 Thanks for your help.

 Best,
 Jon
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Robert Kern 
On Thu, Jun 6, 2013 at 8:48 PM, Maccarthy, Jonathan K jkm...@lanl.gov wrote:
 Hi everyone,

 I've looked in the mailing list archives and with the googles, but haven't 
 yet found any hints with this question...

 I have a float field in a NumPy record that looks like it's being substituted 
 as a string in the Python {:f}.format() mini-language, thus throwing an 
 error:


 In [1]: tmp = np.rec.array([('XYZZ', 2001123, -23.82396)], 
 dtype=np.dtype([('sta', '|S6'), ('ondate', 'i8'), ('lat', 'f4')]))[0]

 In [2]: type(tmp)
 Out[3]: numpy.core.records.record

 In [3]: tmp
 Out[3]: ('XYZZ', 2001123, -23.823917388916016)

 In [4]: tmp.sta, tmp.ondate, tmp.lat
 Out[4]: ('XYZZ', 2001123, -23.823917)

 # strings and integers work
 In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp)
 Out[5]: 'XYZZ2001123'

 # lat is a float, but it seems to be coerced to a string first, and failing
 In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
 ---
 ValueErrorTraceback (most recent call last)
 /Users/jkmacc/ipython-input-312-bff8066cfde8 in module()
  1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)

 ValueError: Unknown format code 'f' for object of type 'str'

 # string formatting for doesn't fail
 In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6s}'.format(tmp)
 Out[7]: 'XYZZ2001123  -23.82'


 This also fails:

 In [7]: {:f}.format(np.array(3.2, dtype='f4'))
 ---
 ValueErrorTraceback (most recent call last)
 /Users/jkmacc/ipython-input-314-33119128e3e6 in module()
  1 {:f}.format(np.array(3.2, dtype='f4'))

 ValueError: Unknown format code 'f' for object of type 'str'



 Does anyone understand what's happening?

numpy.ndarray does not implement the __format__() method. Thus,
str.format() method falls back to object.__format__(). This is the
exception that object.__format__() raises. Why it says object of type
'str' is not clear to me. Similarly, numpy.float32 does not implement
the __format__() method. The string scalar type and the native integer
scalar type (I assume you are on a 64-bit platform, so Python ints are
64-bit to match your 'i8' field) inherit from the corresponding native
Python types, so they inherit their __format__() methods.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Maccarthy, Jonathan K 
Ah, so np.int64 and np.str inherit the native Python __format__(), but 
np.float32/64 doesn't get __builtin__.float.__format__().  That's not 
intuitive, but I see now why this works:

In [8]: '{:6.6s} {:8d} {:11.6f}'.format(tmp.sta, tmp.ondate, float(tmp.lat))
Out[8]: 'XYZZ2001123  -23.82'

Thanks!

-Jon

On Jun 6, 2013, at 1:56 PM, Robert Kern wrote:

 On Thu, Jun 6, 2013 at 8:48 PM, Maccarthy, Jonathan K jkm...@lanl.gov wrote:
 Hi everyone,
 
 I've looked in the mailing list archives and with the googles, but haven't 
 yet found any hints with this question...
 
 I have a float field in a NumPy record that looks like it's being 
 substituted as a string in the Python {:f}.format() mini-language, thus 
 throwing an error:
 
 
 In [1]: tmp = np.rec.array([('XYZZ', 2001123, -23.82396)], 
 dtype=np.dtype([('sta', '|S6'), ('ondate', 'i8'), ('lat', 'f4')]))[0]
 
 In [2]: type(tmp)
 Out[3]: numpy.core.records.record
 
 In [3]: tmp
 Out[3]: ('XYZZ', 2001123, -23.823917388916016)
 
 In [4]: tmp.sta, tmp.ondate, tmp.lat
 Out[4]: ('XYZZ', 2001123, -23.823917)
 
 # strings and integers work
 In [5]: '{0.sta:6.6s} {0.ondate:8d}'.format(tmp)
 Out[5]: 'XYZZ2001123'
 
 # lat is a float, but it seems to be coerced to a string first, and failing
 In [6]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
 ---
 ValueErrorTraceback (most recent call last)
 /Users/jkmacc/ipython-input-312-bff8066cfde8 in module()
  1 '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6f}'.format(tmp)
 
 ValueError: Unknown format code 'f' for object of type 'str'
 
 # string formatting for doesn't fail
 In [7]: '{0.sta:6.6s} {0.ondate:8d} {0.lat:11.6s}'.format(tmp)
 Out[7]: 'XYZZ2001123  -23.82'
 
 
 This also fails:
 
 In [7]: {:f}.format(np.array(3.2, dtype='f4'))
 ---
 ValueErrorTraceback (most recent call last)
 /Users/jkmacc/ipython-input-314-33119128e3e6 in module()
  1 {:f}.format(np.array(3.2, dtype='f4'))
 
 ValueError: Unknown format code 'f' for object of type 'str'
 
 
 
 Does anyone understand what's happening?
 
 numpy.ndarray does not implement the __format__() method. Thus,
 str.format() method falls back to object.__format__(). This is the
 exception that object.__format__() raises. Why it says object of type
 'str' is not clear to me. Similarly, numpy.float32 does not implement
 the __format__() method. The string scalar type and the native integer
 scalar type (I assume you are on a 64-bit platform, so Python ints are
 64-bit to match your 'i8' field) inherit from the corresponding native
 Python types, so they inherit their __format__() methods.
 
 --
 Robert Kern
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Robert Kern 
On Thu, Jun 6, 2013 at 9:18 PM, Maccarthy, Jonathan K jkm...@lanl.gov wrote:
 Ah, so np.int64 and np.str inherit the native Python __format__(), but 
 np.float32/64 doesn't get __builtin__.float.__format__().  That's not 
 intuitive, but I see now why this works:

 In [8]: '{:6.6s} {:8d} {:11.6f}'.format(tmp.sta, tmp.ondate, float(tmp.lat))
 Out[8]: 'XYZZ2001123  -23.82'

np.float64 works because it inherits from the Python float type
(Python floats are 64-bit floats). np.float32 doesn't inherit from the
Python float type because it can't; they don't represent the same kind
of data, so their memory layouts at the C level cannot coincide. Since
you are on a 64-bit platform, np.int64 represents the same kind of
integer as the Python int type, so it can subclass, but an np.int32
couldn't.

It's not necessarily intuitive, but it's the best we can do under the
constraints. The only thing more intuitive would be to disallow
subclassing from the Python builtin types entirely, but that's
*really* annoying.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Maccarthy, Jonathan K 
I'm really showing my ignorance now, I think;  so, the old-style fmt % 
(tuple) must use a different mechanism, and perhaps that's why np.savetxt 
never choked on a float32 for me before (yes, I am on a 64-bit system).

In [8]: type(tmp.lat)
Out[8]: numpy.float32

In [9]: '%6s %8i %11.6f' % (tmp.sta, tmp.ondate, tmp.lat)
Out[9]: '  XYZZ  2001123  -23.82'

Thanks again for your patience.

On Jun 6, 2013, at 2:26 PM, Robert Kern wrote:

 
 np.float64 works because it inherits from the Python float type
 (Python floats are 64-bit floats). np.float32 doesn't inherit from the
 Python float type because it can't; they don't represent the same kind
 of data, so their memory layouts at the C level cannot coincide. Since
 you are on a 64-bit platform, np.int64 represents the same kind of
 integer as the Python int type, so it can subclass, but an np.int32
 couldn't.
 
 It's not necessarily intuitive, but it's the best we can do under the
 constraints. The only thing more intuitive would be to disallow
 subclassing from the Python builtin types entirely, but that's
 *really* annoying.
 
 --
 Robert Kern
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] floats coerced to string with {:f}.format() ?

2013-06-06 Thread Robert Kern 
On Thu, Jun 6, 2013 at 9:50 PM, Maccarthy, Jonathan K jkm...@lanl.gov wrote:
 I'm really showing my ignorance now, I think;  so, the old-style fmt % 
 (tuple) must use a different mechanism, and perhaps that's why np.savetxt 
 never choked on a float32 for me before (yes, I am on a 64-bit system).

 In [8]: type(tmp.lat)
 Out[8]: numpy.float32

 In [9]: '%6s %8i %11.6f' % (tmp.sta, tmp.ondate, tmp.lat)
 Out[9]: '  XYZZ  2001123  -23.82'

 Thanks again for your patience.

Yes, the mechanism is quite different. With %f, str.__mod__() is in
control. When it is handling the %f code, it is expecting a Python
float object *or* one that can be converted into one via float(). All
of the numpy floating and integer types can be converted to a Python
float via float().

With str.format(), the control is inverted. The value being
represented is asked via its __format__() method to handle the 'f'
format code.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion