Re: [Tutor] Help with building bytearray arrays
Chip Wachob wrote: > Peter, > > I see that clue "[[". > > The thread history pretty much sums up what is going on up to this point. > > I'll cover it once more: [snip] > I hope this helps. Unfortunately it doesn't as the problem is in my_transfer. > I'm beginning to wonder if Python was the right choice for this > project.. You mean you'd rather debug a segfault in C than an exception with a traceback and an error message that is spot on in Python? You have an error in your program logic, and you'll eventually run into those no matter what language you choose. > Thanks to everyone for your comments and patience. If you are too frustrated to look into the actual problem at the moment you can change the following loop > Earlier discussions here indicated that the best way was to : > > results = [] > > for i in range (0, slice_size): >results.append(transfer(data_out)) by replacing the append() with the extend() method results = [] for i in range (0, slice_size): results.extend(transfer(data_out)) for now. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 10/09/18 19:15, Chip Wachob wrote: > So I see why my .join() isn't working. I'm not sure how to fix it though. I already showed you the sum() function. It can take a list of lists and add them together end_array = sum(results,[]) > My background is in C and other 'historical' languages, so I'm trying > to get a hold of the way Python handles arrays, which is different > that the way I've thought for 20+ years.. :) Yes, as per my other message, Python data structures are very powerful. They are not just bare chunks of memory. > I'm beginning to wonder if Python was the right choice for this > project.. but it's too late for me to switch now. Looks like a very good choice to me. Almost any other language would be much more work. :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 10Sep2018 10:23, Chip Wachob wrote:
So, without all the fluff associated with wiggling lines, my function
now looks like this:
def RSI_size_the_loop():
results = []
all_together = [] # not certain if I need this, put it in in an
attempt to fix the incompatibility if it existed
You don't need this. all_together doesn't need to be mentioned until you
initiate it with your bytearray.join.
for x in range (0, MAX_LOOP_COUNT, slice_size):
results.append(my_transfer(disp, data_out, slice_size)
print " results ", x, " = ", results # show how results grows
on each iteration
all_together = bytearray().join(results)
print " all together ", all_together
I can observe results increasing in size and the last time through the loop:
results 48 =
[[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]]
Peter has pointed out that you have a list of list-of-bytearray instead of a
flat list-of-bytearray.
The inference here is that your my_transfer function is returning a single
element list-of-bytearray.
So, now when I hit the line:
all_together = bytearray().join(results)
I'm getting the Traceback :
[...]
Traceback (most recent call last):
File "SW8T_5.py", line 101, in # this is my main script
loop_size = RSI_size_the_loop(Print)
File "/home/temp/Python_Scratch/examples/RSI.py", line 359, in
RSI_size_the_loop
all_together = bytearray().join(results)
TypeError: can only join an iterable of bytes (item 0 has type 'list')
So because you have a list-of-list, item[0] is indeed a list, not a bytearray.
If you change this:
results.append(my_transfer(disp, data_out, slice_size)
into:
result = my_transfer(disp, data_out, slice_size)
print("result =", repr(result))
results.append(result)
this should be apparent. So this issue lies with your my_transfer function; the
main loop above now looks correct.
I've even added in print statements for the types, so I could double
check, and I get:
results returns
all_together returns
So both are type 'list' which is referred to here :
https://infohost.nmt.edu/tcc/help/pubs/python/web/sequence-types.html
as a valid sequence type but apparently there's a detail I'm still missing...
Yeah. bytearray().join wants bytes or bytearrays in the list/iterable you hand
it. You've got lists, with the bytearrays further in.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
Peter, I see that clue "[[". The thread history pretty much sums up what is going on up to this point. I'll cover it once more: I'm using Adafruit FT232H Breakout board and Adafruit's library. https://github.com/adafruit/Adafruit_Python_GPIO Per Adafruit's example code, I create an SPI interface: https://learn.adafruit.com/adafruit-ft232h-breakout?view=all I can access this interface with the spi.transfer() spi.write() and spi.read() functions. The transfer() function (see link above) accepts and returns bytearray objects data. My application requires me to send a large payload down the SPI port to my circuit, and read back the same in full duplex. Due to a limitation of the part, I can't send my whole bytearray at one time. It crashes out. So, my solution was to send as large of a chunk of data at one time as I can inside a loop until I get to the end of the data to be sent. Meanwhile, the transfer() function is returning 'chunks' of bytearrays. Earlier discussions here indicated that the best way was to : results = [] for i in range (0, slice_size): results.append(transfer(data_out)) Then, concatenate the results into a long bytearray so I can use it elsewhere in my code. all_together = bytearray().join(results) I _thought_ that this was going to create a concatenated list of all the returned results: all_together = [results[0] + results[1] + results [2] + results[3]] -- for example but, as you point out, I got: all_together = [[results[0]], [results[1]], [results[2]]. [results[3]] -- I'm sure that the brackets and braces are not syntactically correct, but I think you get the idea. So I see why my .join() isn't working. I'm not sure how to fix it though. Related to this, but I'm not yet at that point until I get this resolved, I need to walk through the all_together data and find where the data changes from one value to another.. My background is in C and other 'historical' languages, so I'm trying to get a hold of the way Python handles arrays, which is different that the way I've thought for 20+ years.. :) I hope this helps. I'm beginning to wonder if Python was the right choice for this project.. but it's too late for me to switch now. Thanks to everyone for your comments and patience. On Mon, Sep 10, 2018 at 1:42 PM, Peter Otten <__pete...@web.de> wrote: > Chip Wachob wrote: > >> Cameron, >> >> Thank you again for the insight. >> >> Yes, data_out is an equivalently-sized 'chunk' of a larger array. >> >> I'm 'getting' this now.. >> >> So, without all the fluff associated with wiggling lines, my function >> now looks like this: >> >> def RSI_size_the_loop(): >>results = [] >>all_together = [] # not certain if I need this, put it in in an >> attempt to fix the incompatibility if it existed >> >>for x in range (0, MAX_LOOP_COUNT, slice_size): >> results.append(my_transfer(disp, data_out, slice_size) >> >> print " results ", x, " = ", results # show how results grows >> on each iteration >> >>all_together = bytearray().join(results) >> >>print " all together ", all_together >> >> >> I can observe results increasing in size and the last time through the >> loop: >> >> results 48 = >> > [[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')], > > Note that there are two '[' at the start of the list. This means that the > first list item is another list. In fact you seem to have a list of single > item lists like > > [["foo"], ["bar"], ...] > > when you need > > ["foo", "bar", ...] > > Of course join will fail with that: > "".join(["foo", "bar"]) > 'foobar' "".join([["foo"], ["bar"]]) > Traceback (most recent call last): > File "", line 1, in > TypeError: sequence item 0: expected string, list found > > I think the error message is pretty clear ;) > > Have a look into your my_transfer() function to find out why it returns a > list (or show us the code). > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
Chip Wachob wrote: > Cameron, > > Thank you again for the insight. > > Yes, data_out is an equivalently-sized 'chunk' of a larger array. > > I'm 'getting' this now.. > > So, without all the fluff associated with wiggling lines, my function > now looks like this: > > def RSI_size_the_loop(): >results = [] >all_together = [] # not certain if I need this, put it in in an > attempt to fix the incompatibility if it existed > >for x in range (0, MAX_LOOP_COUNT, slice_size): > results.append(my_transfer(disp, data_out, slice_size) > > print " results ", x, " = ", results # show how results grows > on each iteration > >all_together = bytearray().join(results) > >print " all together ", all_together > > > I can observe results increasing in size and the last time through the > loop: > > results 48 = > [[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')], Note that there are two '[' at the start of the list. This means that the first list item is another list. In fact you seem to have a list of single item lists like [["foo"], ["bar"], ...] when you need ["foo", "bar", ...] Of course join will fail with that: >>> "".join(["foo", "bar"]) 'foobar' >>> "".join([["foo"], ["bar"]]) Traceback (most recent call last): File "", line 1, in TypeError: sequence item 0: expected string, list found I think the error message is pretty clear ;) Have a look into your my_transfer() function to find out why it returns a list (or show us the code). ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
Cameron,
Thank you again for the insight.
Yes, data_out is an equivalently-sized 'chunk' of a larger array.
I'm 'getting' this now..
So, without all the fluff associated with wiggling lines, my function
now looks like this:
def RSI_size_the_loop():
results = []
all_together = [] # not certain if I need this, put it in in an
attempt to fix the incompatibility if it existed
for x in range (0, MAX_LOOP_COUNT, slice_size):
results.append(my_transfer(disp, data_out, slice_size)
print " results ", x, " = ", results # show how results grows
on each iteration
all_together = bytearray().join(results)
print " all together ", all_together
I can observe results increasing in size and the last time through the loop:
results 48 =
[[bytearray(b'\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')],
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]]
So, now when I hit the line:
all_together = bytearray().join(results)
I'm getting the Traceback :
Traceback (most recent call last):
File "SW8T_5.py", line 101, in # this is my main script
loop_size = RSI_size_the_loop(Print)
File "/home/temp/Python_Scratch/examples/RSI.py", line 359, in
RSI_size_the_loop
all_together = bytearray().join(results)
TypeError: can only join an iterable of bytes (item 0 has type 'list')
I looked this up, and there's very few search engine results on this
type of TypeError.
I wanted to clarify my understanding of iterable, and looked that up here:
https://infohost.nmt.edu/tcc/help/pubs/python/web/iterable.html
And, as far as I can tell I'm using a compatible sequence type.
I've even added in print statements for the types, so I could double
check, and I get:
results returns
all_together returns
So both are type 'list' which is referred to here :
https://infohost.nmt.edu/tcc/help/pubs/python/web/sequence-types.html
as a valid sequence type but apparently there's a detail I'm still missing...
On Mon, Sep 10, 2018 at 5:22 AM, Cameron Simpson wrote:
> On 09Sep2018 23:00, Chip Wachob wrote:
>>
>> On Sat, Sep 8, 2018 at 9:14 PM, Cameron Simpson wrote:
>>>
>>> Actually he's getting back bytearray instances from transfer and wants to
>>> join them up (his function does a few small transfers to work around an
>>> issue with one big transfer). His earlier code is just confused. So he
>>> wants:
>>>
>>> bytearray().join(results)
>>>
>>> Hacked example:
>>>
>>> >>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
>>> bytearray(b'foobah')
>>
>>
>> I understand this example and I can replicate it in the interpreter..
>>
>> But, I'm still missing something here.
>> I presume that I need to instantiate an array of slice_size-sized
>> bytearrays.
>
>
> But no!
>
>> So, when I'm looping through, I can do:
>> for i in range (0, slice_count):
>> results[i] = spi.transfer(data_out)
>
>
> Python lists are a variable size data structure, so our example goess:
>
> results = []
>
> which allocates an empty list. Then:
>
> for i in range(slice_count):
>results.append(spi.transfer(data_out))
>
> suitably adjusted (data_out will be different pieces of the larger data,
> yes?) which grows the array by one item with each append. Your spi.transfer
> function allocates a new bytearray for each return value, so you end up with
> a list of distinct bytearrays.
>
>> Then I can :
>>
>> all_together = butearray().join(results)
>
>
> Yes.
>
>> But I can't seem to be able to find the proper syntax to create the
>> initial array.
>> And any attempt at filling the arrays to test some stand-alone code
>> only give me errors.
>
>
> You _can_ allocate a presized list, but there's almost no benefit and it
> isn't what people normally do.
>
>> Here's my code (all of it)
>
> [...]
>>
>> #results = bytearray( (bytearray(slice_size)*slice_count) )
>> results = bytearray(slice_size)# why isn't this 16 bytes long?
>
>
> It is 16 bytes long when I do it:
>
>>>> bs=bytearray(16)
>>>> bs
>
> bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
>
>> res_list = (results)*slice_count
>
>
> This is a single 64 byte bytearray. You may have intended to make a single
> element tuple with "(results)", but that is just the same as "results"
> (rundundant brackets). To make a single element tuple you go "(results,)" -
> see the trailing comma?
>
> Example of each:
>
>>>> bs*16
>
>
Re: [Tutor] Help with building bytearray arrays
On 10/09/18 04:00, Chip Wachob wrote:
> I presume that I need to instantiate an array of slice_size-sized bytearrays.
Cameron has already addressed this and explained
that you don't need to and if you did how to do it.
I'd only add that you need to readjust your thinking
when it comes to Python data structures versus C.
In C data structures are very primitive (little more
than an allocated chunk of memory).
In Python data structures are extremely rich and indeed
much of Python's power comes from exploiting the
richness of the standard data structures. It is well
worth while just playing around with these in the >>>
prompt using dir() and help() to explore all of the
functionality embodied in strings, lists, tuples,
sets and dictionaries.
> And any attempt at filling the arrays to test some stand-alone code
> only give me errors.
There are several ways of doing this and Cameron
showed you a couple. There is another option which
is very powerful, especially for generating more
complex data sets. Its called a generator expression
and the most commonly used version is a list
comprehension:
new_list = [expression for item in collection if condition]
The expression can be any valid Python expression.
The condition can be any valid Python expression that
evaluates to a boolean result (and is also optional).
It is equivalent to:
new_list = []
for item in collection:
if condition:
new_list.append(expression)
Some examples to clarify:
num_list = [num for num in range(5)]
which is the same result as
numlist = list(range(5))
odds = [num for num in range(5) if num % 2] ->[1,3]
evens = [num for num in range(5) if num %2 == 0] -> [0,2,4]
squares = [num*num for num in range(5)]
odd_squares = [n*n for n in odds]
You can make them as complex as you like by
creating helper functions too:
fancy_list = [func1(n) for n in my_data if func2(n)]
The syntax can look a little confusing at first
(its somewhat like set notation) but once you
get used to it its OK.
> results = bytearray(slice_size)# why isn't this 16 bytes long?
It is for me.
> res_list = (results)*slice_count
And this is 64 bytes long
> print " results ", [results]
And this prints a list with a single, 16 byte, bytearray inside it.
> print " results size ", sys.getsizeof(results)
This prints(on my system) 65 which may be confusing you.
But remember what I said about Python data structures
being rich, the bytearray has more than just the raw data
in it. The size reflects allthe other stuff that Python
uses to create a bytearray object. Try
>>> sys.getsizeof("")
and
>>> sys.getsizeof(bytearray())
>>> sys.getsizeof(bytearray("1"))
>>> sys.getsizeof(bytearray("12"))
Do the results surprize you?
Remember, Python data structures are not just chunks of memory.
> print " res_list ", [res_list]
Note that you are putting res_list into a list here,
but it is already a bytearray... There is no adavantage
in enclosing it in a list.
> Again, I feel like I'm circling the target, but not able to divine the
> proper syntax
Its not so much the syntax but your mental model
of what's going on under the covers. It's not C
(at least not at the first level down).
HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 09Sep2018 23:00, Chip Wachob wrote:
On Sat, Sep 8, 2018 at 9:14 PM, Cameron Simpson wrote:
Actually he's getting back bytearray instances from transfer and wants to
join them up (his function does a few small transfers to work around an
issue with one big transfer). His earlier code is just confused. So he
wants:
bytearray().join(results)
Hacked example:
>>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
bytearray(b'foobah')
I understand this example and I can replicate it in the interpreter..
But, I'm still missing something here.
I presume that I need to instantiate an array of slice_size-sized bytearrays.
But no!
So, when I'm looping through, I can do:
for i in range (0, slice_count):
results[i] = spi.transfer(data_out)
Python lists are a variable size data structure, so our example goess:
results = []
which allocates an empty list. Then:
for i in range(slice_count):
results.append(spi.transfer(data_out))
suitably adjusted (data_out will be different pieces of the larger data, yes?)
which grows the array by one item with each append. Your spi.transfer function
allocates a new bytearray for each return value, so you end up with a list of
distinct bytearrays.
Then I can :
all_together = butearray().join(results)
Yes.
But I can't seem to be able to find the proper syntax to create the
initial array.
And any attempt at filling the arrays to test some stand-alone code
only give me errors.
You _can_ allocate a presized list, but there's almost no benefit and it isn't
what people normally do.
Here's my code (all of it)
[...]
#results = bytearray( (bytearray(slice_size)*slice_count) )
results = bytearray(slice_size)# why isn't this 16 bytes long?
It is 16 bytes long when I do it:
>>> bs=bytearray(16)
>>> bs
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
res_list = (results)*slice_count
This is a single 64 byte bytearray. You may have intended to make a single
element tuple with "(results)", but that is just the same as "results"
(rundundant brackets). To make a single element tuple you go "(results,)" - see
the trailing comma?
Example of each:
>>> bs*16
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x
00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
>>> (bs,)*16
(bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'),
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'))
But you don't need any of that.
First, the list "results" can start empty and just get things appended to it
and second, you don't need to preallocate any bytearrays because the internal,
primary, transfer allocates a new bytearray for each return chunk.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe
Re: [Tutor] Help with building bytearray arrays
On Sat, Sep 8, 2018 at 9:14 PM, Cameron Simpson wrote:
> On 08Sep2018 11:40, Alan Gauld wrote:
>>
>> On 08/09/18 03:15, Chip Wachob wrote:
>>>
>>> Ideally, I'd like to take the slice_size chunks that have been read
>>> and concatenate them back togetjer into a long MAX_LOOP_COUNT size
>>> array to pass back to the rest of my code. Eg:
>>
>>
>> You need to create a list of read_ary
>>
>> results = []
>>
>> then after creating each read_ary value append it
>> to results.
>>
>> results.append(read_ary)
>>
>> Then, at the very end, return the summation of all
>> the lists in results.
>>
>> return sum(results,[])
>
Cameron is correct (Gold Star for you Cameron)
>
> Actually he's getting back bytearray instances from transfer and wants to
> join them up (his function does a few small transfers to work around an
> issue with one big transfer). His earlier code is just confused. So he
> wants:
>
> bytearray().join(results)
>
> Hacked example:
>
> >>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
> bytearray(b'foobah')
I understand this example and I can replicate it in the interpreter..
But, I'm still missing something here.
I presume that I need to instantiate an array of slice_size-sized bytearrays.
So, when I'm looping through, I can do:
for i in range (0, slice_count):
results[i] = spi.transfer(data_out)
Then I can :
all_together = butearray().join(results)
But I can't seem to be able to find the proper syntax to create the
initial array.
And any attempt at filling the arrays to test some stand-alone code
only give me errors.
Here's my code (all of it)
#
#
#
import sys
slice_size = 16# in bytes
MAX_LOOP_COUNT = 64# in bytes
slice_count = MAX_LOOP_COUNT / slice_size
print " slice size = ", slice_size
print " MLC = ", MAX_LOOP_COUNT
print " slice count = ", slice_count
#results = bytearray( (bytearray(slice_size)*slice_count) )
results = bytearray(slice_size)# why isn't this 16 bytes long?
res_list = (results)*slice_count
print " results ", [results]
print " results size ", sys.getsizeof(results)
print " res_list ", [res_list]
print " res_list size ", sys.getsizeof(res_list)
print " type = ", type(results)
results[0] = ([1,3,5,7,9])
all_together = bytearray().join(results)
But I'm getting:
$ python merge_1.py
slice size = 16
MLC = 64
slice count = 4
results
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]
results size 65
res_list
[bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')]
res_list size 113
type =
Traceback (most recent call last):
File "merge_1.py", line 27, in
results[0] = ([1,3,5,7,9])
TypeError: an integer or string of size 1 is required
Again, I feel like I'm circling the target, but not able to divine the
proper syntax
I hope this makes sense.
>
> And he's working with bytearrays because the target library is Python 2,
> where there's no bytes type.
>
> Cheers,
> Cameron Simpson
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 09Sep2018 17:06, Chip Wachob wrote:
Before I jump in, the 1000 foot view is I have to send an array of 512
bytes down the SPI loop, and read back 512 bytes that were latched in
from a control interface. Unfortunately, there's a glitch in the FTDI
part and I can't just send the 512 bytes.. the part times out and and
causes the script to terminate early... So, my solution was to
'chunk' the data into smaller groups which the part _can_ handle.
This works fine until I come to the point where I concatenate the
'chunks' into my 512 byte array.. which other functions will then
process.
Sounds good to me.
The libraries that I'm using are from the project link below. I'm
learning from all of you how much code to post, I didn't want to post
the entire library, but the way it's set up it is hard not to.. as has
been pointed out.. methods of classes and then there's other files
that contain some of the low-level workings..
https://github.com/adafruit/Adafruit_Python_GPIO
Thanks.
Yes, I realize now that I left out important information. I have my
own transfer function which is the supplied transfer function with a
bunch of GPIO wrapped around it, and the 'chunking'. This way I can
call it when I need to send and receive data from the loop, which is
often. In the end, I will have to talk with several different SPI
devices, all of which will have to have different GPIO line wiggling
going on before and after they call the spi.transfer.
[...]
def transfer(self, data):
Ok, this is a method, a particular type of function associated with a class
instance. (All objects are class instances, and the class is their type.)
So to call this you would normally have a control object of some kind. [...]
Ah, it looks like you should have an SpiDev instance, inferring from this
code:
https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py
So suppose you've got such an object and a variable referring to it, let's
say it is named "spi". You'd normally call the transfer function like this:
spi.transfer(some_data)
[...]
When you call a Python method, the instance itself is implicitly passed as
the parameter "self" (well, the first parameter - in Python we always call
this "self" like C++ uses "this").
Ah, the one 'thorn' in my side is "this". I have considerable
difficulty with the 'this' concept. That probably means that my code
could be 'better', but I tend to avoid "this" like the plague.
It isn't as big a deal as you might imagine. An object method essentially gets
the object as a piece of context. So a method call like this:
spi.transfer(data)
just has the "spi" object available within the method for use, for example is
probably is already set up with all the hardware access you're going to make
use of.
So lower down in the function, when it
goes:
self._ft232h._write(str(bytearray(data)))
all you're doing is making use of the already initialised _ft232h object to do
the write, and that comes along with the "spi" object you called the transfer
method through.
So from your point of view it's just context for the transfer function.
logger.debug('SPI transfer with command {0:2X}.'.format(command))
Write a debugging message.
I saw this, but I don't know how to turn it 'on' so I could see the
log messages. I wouldn't mind doing this with my code as well.
Perhaps creating my own log? Right now, I have a verbose flag that I
pass to all my functions.. called disp. Then for each place where I
want to see a value, etc, I have a line that reads: if(disp): print "
What data is this ", data
Ah, logging. Yes, it is probably worth learning a little about, just to hook it
it. You can at least get away from passing your "disp" variable around - just
call the right logging call (warning, info, debug, error etc) and tune which
messages get out from your main programme.
The logging module is documented here:
https://docs.python.org/2.7/library/logging.html#module-logging
So your main programme would set up the logging, including a logging level. The
module comes with five predefined levels from DEBUG through to CRITICAL. Then
in your code you write messages like the debug one above. When you're debugging
you might set the systems level to DEBUG and see heaps of messages, or INFO to
see progress reporting, or WARNING to just see when things go wrong. Then your
code just writes messages are a suitable level and the global setting controls
which ones get out.
All you really need to do is "import logging" and then just call
"logging.warning" or "logging.debug" etc. There's a tutorial here:
https://docs.python.org/2.7/howto/logging.html#logging-basic-tutorial
Get off the ground like that and return to your main programme. The whole
logging system can be rather complicated if you get sucked into learning it
thoroughly.
[...]
# Send command and length.
self._assert_cs()
I would guess that this raises a control signal. Ah,
Re: [Tutor] Help with building bytearray arrays
Cameron, et al.
First off, thank you for being patient with me. I'm not used to the
email list communication style.
Since Cameron's response was the one that raised the most questions /
comments, I'm going to reply to it.
Inline.. now that I know that this is the preferred method...
Before I jump in, the 1000 foot view is I have to send an array of 512
bytes down the SPI loop, and read back 512 bytes that were latched in
from a control interface. Unfortunately, there's a glitch in the FTDI
part and I can't just send the 512 bytes.. the part times out and and
causes the script to terminate early... So, my solution was to
'chunk' the data into smaller groups which the part _can_ handle.
This works fine until I come to the point where I concatenate the
'chunks' into my 512 byte array.. which other functions will then
process.
The libraries that I'm using are from the project link below. I'm
learning from all of you how much code to post, I didn't want to post
the entire library, but the way it's set up it is hard not to.. as has
been pointed out.. methods of classes and then there's other files
that contain some of the low-level workings..
https://github.com/adafruit/Adafruit_Python_GPIO
Anyhow, on with the 'show'..
On Sat, Sep 8, 2018 at 6:49 AM, Cameron Simpson wrote:
> On 08Sep2018 20:01, Cameron Simpson wrote:
>>>
>>> So, if I'm understanding the transfer() function correctly, the
>>> function takes and returns a bytearray type.
>>
>>
>> It would be good to see the specification for the transfer function. They
>> we can adhere to its requirements. Can you supply a URL?
>
>
> I see you supplied the whole transfer function in your first message :-(
Yes, I realize now that I left out important information. I have my
own transfer function which is the supplied transfer function with a
bunch of GPIO wrapped around it, and the 'chunking'. This way I can
call it when I need to send and receive data from the loop, which is
often. In the end, I will have to talk with several different SPI
devices, all of which will have to have different GPIO line wiggling
going on before and after they call the spi.transfer.
You're walk-through is very helpful.
>
> I'll recite it here and walk through its contents to explain:
>
>def transfer(self, data):
>
> Ok, this is a method, a particular type of function associated with a class
> instance. (All objects are class instances, and the class is their type.)
>
> So to call this you would normally have a control object of some kind. [...]
> Ah, it looks like you should have an SpiDev instance, inferring from this
> code:
>
>
> https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py
>
> So suppose you've got such an object and a variable referring to it, let's
> say it is named "spi". You'd normally call the transfer function like this:
>
> spi.transfer(some_data)
>
> instead of calling transfer() "bare", as it were.
Again, I was generalizing and also being somewhat lazy and not typing
it all... my bad.
>
> When you call a Python method, the instance itself is implicitly passed as
> the parameter "self" (well, the first parameter - in Python we always call
> this "self" like C++ uses "this").
Ah, the one 'thorn' in my side is "this". I have considerable
difficulty with the 'this' concept. That probably means that my code
could be 'better', but I tend to avoid "this" like the plague.
>
>"""Full-duplex SPI read and write. The specified array of bytes will
> be
>clocked out the MOSI line, while simultaneously bytes will be read
> from
>the MISO line. Read bytes will be returned as a bytearray object.
>"""
>
> This is a docstring. It is like a comment but it gets attached to the
> function and can be inspected. Not your problem.
>
># Build command to read and write SPI data.
>command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve << 2) |
> self.write_clock_ve
>
> This constructs a value just as you would in C.
>
>logger.debug('SPI transfer with command {0:2X}.'.format(command))
>
> Write a debugging message.
I saw this, but I don't know how to turn it 'on' so I could see the
log messages. I wouldn't mind doing this with my code as well.
Perhaps creating my own log? Right now, I have a verbose flag that I
pass to all my functions.. called disp. Then for each place where I
want to see a value, etc, I have a line that reads: if(disp): print "
What data is this ", data
>
># Compute length low and high bytes.
># NOTE: Must actually send length minus one because the MPSSE engine
># considers 0 a length of 1 and a length of 65536
>length = len(data)
>len_low = (length-1) & 0xFF
>len_high = ((length-1) >> 8) & 0xFF
>
> All just like C.
>
># Send command and length.
>self._assert_cs()
>
> I would guess that this raises a control signal. Ah, RS-232? So
> clear-to-send then.
This is actually a chip
Re: [Tutor] Help with building bytearray arrays
On 08Sep2018 11:40, Alan Gauld wrote:
On 08/09/18 03:15, Chip Wachob wrote:
Ideally, I'd like to take the slice_size chunks that have been read
and concatenate them back togetjer into a long MAX_LOOP_COUNT size
array to pass back to the rest of my code. Eg:
You need to create a list of read_ary
results = []
then after creating each read_ary value append it
to results.
results.append(read_ary)
Then, at the very end, return the summation of all
the lists in results.
return sum(results,[])
Actually he's getting back bytearray instances from transfer and wants to join
them up (his function does a few small transfers to work around an issue with
one big transfer). His earlier code is just confused. So he wants:
bytearray().join(results)
Hacked example:
>>> bytearray().join( (bytearray("foo"),bytearray("bah")) )
bytearray(b'foobah')
And he's working with bytearrays because the target library is Python 2, where
there's no bytes type.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 08Sep2018 20:01, Cameron Simpson wrote:
So, if I'm understanding the transfer() function correctly, the
function takes and returns a bytearray type.
It would be good to see the specification for the transfer function.
They we can adhere to its requirements. Can you supply a URL?
I see you supplied the whole transfer function in your first message :-(
I'll recite it here and walk through its contents to explain:
def transfer(self, data):
Ok, this is a method, a particular type of function associated with a class
instance. (All objects are class instances, and the class is their type.)
So to call this you would normally have a control object of some kind. [...]
Ah, it looks like you should have an SpiDev instance, inferring from this code:
https://github.com/adafruit/Adafruit_Python_GPIO/blob/master/Adafruit_GPIO/SPI.py
So suppose you've got such an object and a variable referring to it, let's say
it is named "spi". You'd normally call the transfer function like this:
spi.transfer(some_data)
instead of calling transfer() "bare", as it were.
When you call a Python method, the instance itself is implicitly passed as the
parameter "self" (well, the first parameter - in Python we always call this
"self" like C++ uses "this").
"""Full-duplex SPI read and write. The specified array of bytes will be
clocked out the MOSI line, while simultaneously bytes will be read from
the MISO line. Read bytes will be returned as a bytearray object.
"""
This is a docstring. It is like a comment but it gets attached to the function
and can be inspected. Not your problem.
# Build command to read and write SPI data.
command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve << 2) |
self.write_clock_ve
This constructs a value just as you would in C.
logger.debug('SPI transfer with command {0:2X}.'.format(command))
Write a debugging message.
# Compute length low and high bytes.
# NOTE: Must actually send length minus one because the MPSSE engine
# considers 0 a length of 1 and a length of 65536
length = len(data)
len_low = (length-1) & 0xFF
len_high = ((length-1) >> 8) & 0xFF
All just like C.
# Send command and length.
self._assert_cs()
I would guess that this raises a control signal. Ah, RS-232? So clear-to-send
then.
self._ft232h._write(str(bytearray((command, len_low, len_high
Ok, it looks like this is Python 2, not Python 3. Let's unpack it.
"(command, len_low, len_high)" is a tuple of 3 values. A tuple is like a read
only list. We're passing that to the bytearray() constructor, which will accept
an iterable of values and make a bytes buffer. And we want to write that to
"self._ft232h". Which expects a str, which is why I think this is Python 2. In
Python 2 there's no "bytes" type and str is an immutable array of 8-bit
character values. So writing bytes uses str.
So this tabkes a tuple of values, to be bytes, makes those into a bytearray and
makes that into a str, and writes that str to the serial line.
self._ft232h._write(str(bytearray(data)))
We write that data itself.
self._ft232h._write('\x87')
self._deassert_cs()
And here we lower the control signal.
# Read response bytes.
return bytearray(self._ft232h._poll_read(length))
This calls the _poll_read method, presumably requesting "length" bytes, the
same length as the supplied data. I presume we get a str back: we make a new
bytearray with those bytes in it and return it.
So you do get a new bytearray back from each call, so we can accumulate them
into a list and then join them together later to make a single bytearray.
Why this faffing about with str and bytearray? Probably for Python 2/3
compatibility, and because you want to deal with bytes (small ints) instead of
characters. Ignore it: we're dealing with bytes.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 08/09/18 03:15, Chip Wachob wrote: > my function's main pieces are: It would probably be better to post the entire function, a partial code sample like this just gives us an inkling of what you are trying to do but not enough to be sure of where the errors lie. > def transfer_byte_array(): >MAX_LOOP_COUNT = 64 >slice_size = 16 >read_ary = bytearray(MAX_LOOP_COUNT) >scratch_ary = bytearray() > >for step in range (0, MAX_LOOP_COUNT, slice_size): > scratch_ary = transfer(data_to_send, slice_size) You pass in two arguments here but in your description of transfer below it is a method of an object that only takes one (self being the object). Something is broken here. I would expect, given the code below, to see something like scratch_ary = someObject.transfer(data_to_send) > for bytes in range (0, slice_size): > read_ary = scratch_ary[bytes] This is replacing rad_ary each time with a single value from scratch_ary. At the end of theloop read_ary wil;l cpontain the last valuye in scratch_ary. You can achieve the same result without a loop: raed_ary = scratch_ary[-1] But I suspect that's not what you want. I think what you really want is the first slice_size elements of scratch_ary: read_ary = scratch_ary[:slice_size] # use slicing > Ideally, I'd like to take the slice_size chunks that have been read > and concatenate them back togetjer into a long MAX_LOOP_COUNT size > array to pass back to the rest of my code. Eg: You need to create a list of read_ary results = [] then after creating each read_ary value append it to results. results.append(read_ary) Then, at the very end, return the summation of all the lists in results. return sum(results,[]) > def transfer(self, data): The self parameter indicates that this is a method definition from a class. That implies that to use it you must create an instance of the class and call the method on that instance. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 08/09/18 03:15, Chip Wachob wrote: > Admin, please remove my earlier messages. No can do, once the emails are sent by the server they are out there on the net, stored in people's mailboxes and in various internet archives. When using a mailing list always check before sending, there's no going back... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 08/09/18 04:57, Chip Wachob wrote: > was my attempt at 'setting' the type of the variable. A variable in Python is just a name. It has no type. Only values have types. So you can set (or change) the type of a value but not of a variable. > Coming from a 'C' background, I find the lack of typing in Python to > be confusing. There is no shortage of typing in Python its just applied to the values(objects) not to their labels. > I'm used to working with bytes / words signed and > unsigned for a reason. The same applies in Python. You just have to separate the concepts of variable name and variable value. In C those two concepts get kind of squished together but in Python the name/value duality is strong. You can always check the type of an object by using the type() function or if you think a type error is likely catch it when it happens with: try:... except TypeError: -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
Please try to adopt the inline reply style; we prefer it here. It lets us reply point by point and makes messages read like conversations. Anyway... On 07Sep2018 23:57, Chip Wachob wrote: Point taken on 'bytes'.. thanks. the scratch_ary = bytearray() was my attempt at 'setting' the type of the variable. I had hoped that it would help resolve the error messages telling me that the types didn't go together. Coming from a 'C' background, I find the lack of typing in Python to be confusing. I'm used to working with bytes / words signed and unsigned for a reason. Ok. Variables aren't typed. Variables are references to objects, which _are_ typed. So pointing a variable at a bytearray doesn't set its type in any persistent sense. Thus: x = 1 x = "a string" x = bytearray() All just point "x" at different objects. If you'd like a C metaphor, variables are _like_ (void*) pointers: they can reference any kind of object. Aside: they're not pointers, avoid the term - people will complain. As far as the language spec goes they're references. Which may in a particular implementation be _implemented internally_ using pointers. So, if I'm understanding the transfer() function correctly, the function takes and returns a bytearray type. It would be good to see the specification for the transfer function. They we can adhere to its requirements. Can you supply a URL? You mentioned about constructing a bytearray if I need one. Can you expand on how I approach that? Well, you were getting several bytes or bytearray objects back from transfer and wanted to join them together. The efficient way is to accumulate them in a list and then join them all together later using the bytearry (or bytes) .join method: https://docs.python.org/3/library/stdtypes.html#bytearray.join So: data_list = [] for # send some data, get some data back data = transfer(...) # add that buffer to the data_list data_list.append(data) # make a single bytes object being the concatenation of all the smaller data # chunks all_data = bytes.join(data_list) It looks to me like your transfer() function is handed a bytes or bytearray and returns one. Normally that would be a separate bytes or bytearray. Aside: bytes and bytearray objects are generally the way Python 3 deals with chunks of bytes. A "bytes" is readonly and a bytearray may have its contents modified. From a C background, they're like an array of unsigned chars. I'm going to try the experiment you mentioned in hopes of it giving me a better understanding of the 'types' and what happens with the variables. Sounds good. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
Point taken on 'bytes'.. thanks. the scratch_ary = bytearray() was my attempt at 'setting' the type of the variable. I had hoped that it would help resolve the error messages telling me that the types didn't go together. Coming from a 'C' background, I find the lack of typing in Python to be confusing. I'm used to working with bytes / words signed and unsigned for a reason. So, if I'm understanding the transfer() function correctly, the function takes and returns a bytearray type. You mentioned about constructing a bytearray if I need one. Can you expand on how I approach that? I'm going to try the experiment you mentioned in hopes of it giving me a better understanding of the 'types' and what happens with the variables. Thank you, On Fri, Sep 7, 2018 at 6:23 PM, Cameron Simpson wrote: > On 07Sep2018 15:45, Chip Wachob wrote: >> >> Basically I'm trying to write a block of unsigned bytes to the device >> and read back an equal sized block of unsigned bytes. There's a >> function that is provided called transfer(data_to_send, num_of_bytes) >> that handles the heavy lifting. Unfortunately there seems to be a bug >> in the part and if I attempt to send the entire block of bytes (64), >> the device will lock up. I've been able to determine that if I send >> 16 bytes at a time, I'm okay. >> >> So, I take my bytearray(64) and step through it 16 bytes at a time like >> this: >> >> my function's main pieces are: >> >> def transfer_byte_array(): >> MAX_LOOP_COUNT = 64 >> slice_size = 16 >> read_ary = bytearray(MAX_LOOP_COUNT) >> scratch_ary = bytearray() >> >> for step in range (0, MAX_LOOP_COUNT, slice_size): >> scratch_ary = transfer(data_to_send, slice_size) >> >> for bytes in range (0, slice_size): >> read_ary = scratch_ary[bytes] >> >> return(read_ary) >> >> >> Ideally, I'd like to take the slice_size chunks that have been read >> and concatenate them back togetjer into a long MAX_LOOP_COUNT size >> array to pass back to the rest of my code. Eg: >> >> read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3] > > > Minor remark: don't use the name "bytes" for a variable, it is a builtin > type name and you're shadowing it. > > It looks to me like "transfer" hands you back a buffer with the read data, > so this: > > scratch_ary = bytearray() > > don't do anything (it gets discarded). > > If you're getting back a bytes or bytearray object from transfer, just > gather them all up in an list: > > returned_buffers = [] > for .. > response = transfer(data_to_send, slice_size) > returned_buffers.append(response) > ... > read_ary = b''.join(returned_buffers) > > Note that that makes a new bytes object for read_ary to refer to. You don't > need the earlier initialisation of read_ary. > > Also note that the bytes object is read only; if that is a problem you'll > need to construct a bytearray instead. > > [...] >> >> The problem that I repeatedly run into is with the line: >> >> read_ary = scratch_ary[bytes] (or variants thereof) >> >> The traceback is this: >> >> Traceback (most recent call last): >> File "SW8T_5.py", line 101, in >>loop_size = RSI_size_the_loop(Print) >> File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in >> RSI_size_the_loop >>read_ary.append(scratch_ary[singles]) >> TypeError: an integer or string of size 1 is required > > > Yeah I thought that looked weird to me too. >> >> or, one of the other common ones that I've seen is >> >> TypeError: can't concat bytearray to list >> >> This one is confusing because both of the operands are bytearry >> types.. or at least I thought they should be... > > > No, one will be a list :-) putting a bunch of: > > print(repr(foo)) > > replacing "foo" with relevant variables will be illuminating to you; you can > see immediately where this are not what you expected. > >> I'm obviously missing something fundamental here. Problem is I can't >> seem to find any examples of people asking this question before on the >> inter-webs.. > > > You have the opposite of my problem. I can often find people asking the same > question, but less often an answer. Or a decent answer, anyway. > > Cheers, > Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Help with building bytearray arrays
Admin, please remove my earlier messages.
This message is a properly 'self contained' message.
Hello,
I've been struggling with this for the last day or so and I can't seem
to figure out how to make it work.
I'll start out by saying that if there's a better approach, then I'm all ears.
I'm using the Adafruit Breakout board for the FTDI FT232H part. Along
with this comes the Python libraries and Adafruit libraries which I'm
using. I don't think that the libraries are the real issue here, but
I thought I should mention it just for the sake of information.
Basically I'm trying to write a block of unsigned bytes to the device
and read back an equal sized block of unsigned bytes. There's a
function that is provided called transfer(data_to_send, num_of_bytes)
that handles the heavy lifting. Unfortunately there seems to be a bug
in the part and if I attempt to send the entire block of bytes (64),
the device will lock up. I've been able to determine that if I send
16 bytes at a time, I'm okay.
So, I take my bytearray(64) and step through it 16 bytes at a time like this:
my function's main pieces are:
def transfer_byte_array():
MAX_LOOP_COUNT = 64
slice_size = 16
read_ary = bytearray(MAX_LOOP_COUNT)
scratch_ary = bytearray()
for step in range (0, MAX_LOOP_COUNT, slice_size):
scratch_ary = transfer(data_to_send, slice_size)
for bytes in range (0, slice_size):
read_ary = scratch_ary[bytes]
return(read_ary)
Ideally, I'd like to take the slice_size chunks that have been read
and concatenate them back togetjer into a long MAX_LOOP_COUNT size
array to pass back to the rest of my code. Eg:
read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3]
I know that the + version doesn't work (or didn't for me) but it is
just my attempt at illustrating the overall goal.
The problem that I repeatedly run into is with the line:
read_ary = scratch_ary[bytes] (or variants thereof)
The traceback is this:
Traceback (most recent call last):
File "SW8T_5.py", line 101, in
loop_size = RSI_size_the_loop(Print)
File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in
RSI_size_the_loop
read_ary.append(scratch_ary[singles])
TypeError: an integer or string of size 1 is required
or, one of the other common ones that I've seen is
TypeError: can't concat bytearray to list
This one is confusing because both of the operands are bytearry
types.. or at least I thought they should be...
when I try to replace the same line with :
read_ary += scratch_ary[bytes]
or
read_ary.append(scratch[bytes])
or
read_ary = read_ary + scratch_ary[bytes]
I'm obviously missing something fundamental here. Problem is I can't
seem to find any examples of people asking this question before on the
inter-webs..
the transfer function expects an input of a bytearray and returns the same:
def transfer(self, data):
"""Full-duplex SPI read and write. The specified array of bytes will be
clocked out the MOSI line, while simultaneously bytes will be read from
the MISO line. Read bytes will be returned as a bytearray object.
"""
# Build command to read and write SPI data.
command = 0x30 | (self.lsbfirst << 3) | (self.read_clock_ve <<
2) | self.write_clock_ve
logger.debug('SPI transfer with command {0:2X}.'.format(command))
# Compute length low and high bytes.
# NOTE: Must actually send length minus one because the MPSSE engine
# considers 0 a length of 1 and a length of 65536
length = len(data)
len_low = (length-1) & 0xFF
len_high = ((length-1) >> 8) & 0xFF
# Send command and length.
self._assert_cs()
self._ft232h._write(str(bytearray((command, len_low, len_high
self._ft232h._write(str(bytearray(data)))
self._ft232h._write('\x87')
self._deassert_cs()
# Read response bytes.
return bytearray(self._ft232h._poll_read(length))
Thank you in advance to taking time to read.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with building bytearray arrays
On 07Sep2018 15:45, Chip Wachob wrote: Basically I'm trying to write a block of unsigned bytes to the device and read back an equal sized block of unsigned bytes. There's a function that is provided called transfer(data_to_send, num_of_bytes) that handles the heavy lifting. Unfortunately there seems to be a bug in the part and if I attempt to send the entire block of bytes (64), the device will lock up. I've been able to determine that if I send 16 bytes at a time, I'm okay. So, I take my bytearray(64) and step through it 16 bytes at a time like this: my function's main pieces are: def transfer_byte_array(): MAX_LOOP_COUNT = 64 slice_size = 16 read_ary = bytearray(MAX_LOOP_COUNT) scratch_ary = bytearray() for step in range (0, MAX_LOOP_COUNT, slice_size): scratch_ary = transfer(data_to_send, slice_size) for bytes in range (0, slice_size): read_ary = scratch_ary[bytes] return(read_ary) Ideally, I'd like to take the slice_size chunks that have been read and concatenate them back togetjer into a long MAX_LOOP_COUNT size array to pass back to the rest of my code. Eg: read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3] Minor remark: don't use the name "bytes" for a variable, it is a builtin type name and you're shadowing it. It looks to me like "transfer" hands you back a buffer with the read data, so this: scratch_ary = bytearray() don't do anything (it gets discarded). If you're getting back a bytes or bytearray object from transfer, just gather them all up in an list: returned_buffers = [] for .. response = transfer(data_to_send, slice_size) returned_buffers.append(response) ... read_ary = b''.join(returned_buffers) Note that that makes a new bytes object for read_ary to refer to. You don't need the earlier initialisation of read_ary. Also note that the bytes object is read only; if that is a problem you'll need to construct a bytearray instead. [...] The problem that I repeatedly run into is with the line: read_ary = scratch_ary[bytes] (or variants thereof) The traceback is this: Traceback (most recent call last): File "SW8T_5.py", line 101, in loop_size = RSI_size_the_loop(Print) File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in RSI_size_the_loop read_ary.append(scratch_ary[singles]) TypeError: an integer or string of size 1 is required Yeah I thought that looked weird to me too. or, one of the other common ones that I've seen is TypeError: can't concat bytearray to list This one is confusing because both of the operands are bytearry types.. or at least I thought they should be... No, one will be a list :-) putting a bunch of: print(repr(foo)) replacing "foo" with relevant variables will be illuminating to you; you can see immediately where this are not what you expected. I'm obviously missing something fundamental here. Problem is I can't seem to find any examples of people asking this question before on the inter-webs.. You have the opposite of my problem. I can often find people asking the same question, but less often an answer. Or a decent answer, anyway. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Help with building bytearray arrays
Hello, I've been struggling with this for the last day or so and I can't seem to figure out how to make it work. I'll start out by saying that if there's a better approach, then I'm all ears. I'm using the Adafruit Breakout board for the FTDI FT232H part. Along with this comes the Python libraries and Adafruit libraries which I'm using. I don't think that the libraries are the real issue here, but I thought I should mention it just for the sake of information. Basically I'm trying to write a block of unsigned bytes to the device and read back an equal sized block of unsigned bytes. There's a function that is provided called transfer(data_to_send, num_of_bytes) that handles the heavy lifting. Unfortunately there seems to be a bug in the part and if I attempt to send the entire block of bytes (64), the device will lock up. I've been able to determine that if I send 16 bytes at a time, I'm okay. So, I take my bytearray(64) and step through it 16 bytes at a time like this: my function's main pieces are: def transfer_byte_array(): MAX_LOOP_COUNT = 64 slice_size = 16 read_ary = bytearray(MAX_LOOP_COUNT) scratch_ary = bytearray() for step in range (0, MAX_LOOP_COUNT, slice_size): scratch_ary = transfer(data_to_send, slice_size) for bytes in range (0, slice_size): read_ary = scratch_ary[bytes] return(read_ary) Ideally, I'd like to take the slice_size chunks that have been read and concatenate them back togetjer into a long MAX_LOOP_COUNT size array to pass back to the rest of my code. Eg: read_ary = ary_slice[0] + ary_slice[1] + ary_slice[2] + ary_slice[3] I know that the + version doesn't work (or didn't for me) but it is just my attempt at illustrating the overall goal. The problem that I repeatedly run into is with the line: read_ary = scratch_ary[bytes] (or variants thereof) The traceback is this: Traceback (most recent call last): File "SW8T_5.py", line 101, in loop_size = RSI_size_the_loop(Print) File "/home/temp/Python_Scratch/examples/RSI.py", line 350, in RSI_size_the_loop read_ary.append(scratch_ary[singles]) TypeError: an integer or string of size 1 is required or, one of the other common ones that I've seen is TypeError: can't concat bytearray to list This one is confusing because both of the operands are bytearry types.. or at least I thought they should be... when I try to replace the same line with : read_ary += scratch_ary[bytes] or read_ary.append(scratch[bytes]) or read_ary = read_ary + scratch_ary[bytes] I'm obviously missing something fundamental here. Problem is I can't seem to find any examples of people asking this question before on the inter-webs.. Thank you in advance to taking time to read. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
