Question about Reading from text file with Python's Array class

[traveller3141]

I've been trying to use the Array class to read 32-bit integers from a
file. There is one integer per line and the integers are stored as text.
For problem specific reasons, I only am allowed to read 2 lines (2 32-bit
integers) at a time.

To test this, I made a small sample file (sillyNums.txt) as follows;
109
345
2
1234556

To read this file I created the following test script (trying to copy
something I saw on Guido's blog -
http://neopythonic.blogspot.com/2008/10/sorting-million-32-bit-integers-in-2mb.html
):

import array
assert array.array('i').itemsize == 4

bufferSize = 2

f=open(sillyNums.txt,r)
data = array.array('i')
data.fromstring(f.read(data.itemsize* bufferSize))
print data


The output was nonsense:
array('i', [171520049, 171258931])

I assume this has something to do with my incorrectly specifying how the
various bit/bytes line up. Does anyone know if there's a simple explanation
of how I can do this correctly, or why I can't do it at all?

Thanks,
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about Reading from text file with Python's Array class

[Peter Otten]

traveller3141 wrote:

 I've been trying to use the Array class to read 32-bit integers from a
 file. There is one integer per line and the integers are stored as text.
 For problem specific reasons, I only am allowed to read 2 lines (2 32-bit
 integers) at a time.
 
 To test this, I made a small sample file (sillyNums.txt) as follows;
 109
 345
 2
 1234556

These are numbers in ascii not in binary. To read these you don't have to 
use the array class:

from itertools import islice
data = []
with open(sillyNums.txt) as f:
for line in islice(f, 2):
data.append(int(line))

(If you insist on using an array you of course can)

Binary denotes the numbers as they are stored in memory and typically used 
by lowlevel languages like C. A binary 32 bit integer always consists of 4 
bytes. With your code

 To read this file I created the following test script (trying to copy
 something I saw on Guido's blog -
 http://neopythonic.blogspot.com/2008/10/sorting-million-32-bit-integers-
in-2mb.html
 ):
 
 import array
 assert array.array('i').itemsize == 4
 
 bufferSize = 2
 
 f=open(sillyNums.txt,r)
 data = array.array('i')
 data.fromstring(f.read(data.itemsize* bufferSize))
 print data
 
 
 The output was nonsense:
 array('i', [171520049, 171258931])

you are telling Python to interpret the first 8 bytes in the file as two 
integers. The first 4 bytes are 1, 0, 9, \n (newline) when 
interpreted as characters or 49, 48, 57, 10 when looking at the bytes' 
numerical values. A 32 bit integer is calculated with

 49+48*2**8+57*2**16+10*2**24
171520049

Looks familiar...

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about Reading from text file with Python's Array class

[Tim Chase]

On 12/18/11 12:33, traveller3141 wrote:

To test this, I made a small sample file (sillyNums.txt) as follows;
109
345
2
1234556

f=open(sillyNums.txt,r)
data = array.array('i')
data.fromstring(f.read(data.itemsize* bufferSize))
print data

The output was nonsense:
array('i', [171520049, 171258931])

I assume this has something to do with my incorrectly specifying how the
various bit/bytes line up. Does anyone know if there's a simple explanation
of how I can do this correctly, or why I can't do it at all?


It reads the bytes directly as if using struct.unpack() as in

 data = '109\n345\n2\n123456'
 print ' '.join(hex(ord(c))[2:] for c in data)
31 30 39 a 33 34 35 a 32 a 31 32 33 34 35 36
 0x0a393031 # 4 bytes
171520049

It sounds like you want something like

  from itertools import islice
  a = array.array('i')
  a.fromlist(map(int, islice(f, bufferSize)))

which will read the first 2 lines of the file (using islice() on 
the file-object and slicing the first 2 items), and map the 
string representations into integers, then pass the resulting 
list of integer data to the array.array.fromlist() method.


-tkc



--
http://mail.python.org/mailman/listinfo/python-list

Reading from text file

[A. Joseph]

I want to read from text file, 25 lines each time i press enter key,
just like the python documentation.

i`m using Python 2.5, Windows XP

Thank you
--
http://mail.python.org/mailman/listinfo/python-list

Re: Reading from text file

[Fredrik Lundh]

A. Joseph wrote:


I want to read from text file, 25 lines each time i press enter key,
just like the python documentation.


you can use pydoc's pager from your program:

import pydoc

text = open(filename).read()
pydoc.pager(text)

/F

--
http://mail.python.org/mailman/listinfo/python-list