Re: Problem slicing a list with the C API
On Sun, 13 Mar 2022 at 10:41, Jen Kris wrote: > > > Thanks for PySequence_InPlaceConcat, so when I need to extend I'll know what > to use. But my previous email was based on incorrect information from > several SO posts that claimed only the extend method will work to add tuples > to a list. I found that's wrong -- even my own Python code uses the append > method. But my PyList_Append is not doing the job so that's where I'm > looking now. > Ah. I guess that's one of the fundamental vulnerabilities of Stack Overflow: people answer the question in front of them, which isn't necessarily the question that YOU are trying to answer. Sometimes, the solutions can be misleading, because your goal is actually different. Fortunately, tuples in Python are objects, like every other value. It's extremely convenient - to word that another way, it's extremely INconvenient to work with a language where that isn't the case, and you need to work differently depending on whether you're using integers or strings. This is how you work with a trie (actually an artifact name now, as it's implemented with a hashtable) in SourceMod - this is the equivalent of dictionary subscript assignment: https://sm.alliedmods.net/new-api/adt_trie/SetTrieValue - works for integers, floats, and handles (which are integers) https://sm.alliedmods.net/new-api/adt_trie/SetTrieString - works for strings Yup, it's annoying :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
Thanks for PySequence_InPlaceConcat, so when I need to extend I'll know what to use. But my previous email was based on incorrect information from several SO posts that claimed only the extend method will work to add tuples to a list. I found that's wrong -- even my own Python code uses the append method. But my PyList_Append is not doing the job so that's where I'm looking now. Thanks very much for your reply. Mar 12, 2022, 15:36 by ros...@gmail.com: > On Sun, 13 Mar 2022 at 10:30, Jen Kris wrote: > >> >> >> Chris, you were right to focus on the list pDictData itself. As I said, >> that is a list of 2-tuples, but I added each of the 2-tuples with >> PyList_Append, but you can only append a tuple to a list with the extend >> method. However, there is no append method in the C API as far as I can >> tell -- hence pDictData is empty. I tried with PyList_SetItem but that >> doesn't work. Do you know of way to "extend" a list in the C API. >> > > Hmm. Not entirely sure I understand the question. > > In Python, a list has an append method, which takes any object (which > may be a tuple) and adds that object to the end of the list: > x = ["spam", "ham"] x.append((1,2)) x > ['spam', 'ham', (1, 2)] > > A list also has an extend method, which takes any sequence (that also > includes tuples), and adds *the elements from it* to the end of the > list: > x = ["spam", "ham"] x.extend((1,2)) x > ['spam', 'ham', 1, 2] > > The append method corresponds to PyList_Append, as you mentioned. It > should be quite happy to append a tuple, and will add the tuple > itself, not the contents of it. So when you iterate over the list, > you'll get tuples. > > Extending a list can be done with the sequence API. In Python, you can > write extend() as +=, indicating that you're adding something onto the > end: > x = ["spam", "ham"] x += (1, 2) x > ['spam', 'ham', 1, 2] > > This corresponds to PySequence_InPlaceConcat, so if that's the > behaviour you want, that would be the easiest way to do it. > > Based on your other comments, I would suspect that appending the > tuples is probably what you want here? > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
On Sun, 13 Mar 2022 at 10:30, Jen Kris wrote: > > > Chris, you were right to focus on the list pDictData itself. As I said, > that is a list of 2-tuples, but I added each of the 2-tuples with > PyList_Append, but you can only append a tuple to a list with the extend > method. However, there is no append method in the C API as far as I can tell > -- hence pDictData is empty. I tried with PyList_SetItem but that doesn't > work. Do you know of way to "extend" a list in the C API. > Hmm. Not entirely sure I understand the question. In Python, a list has an append method, which takes any object (which may be a tuple) and adds that object to the end of the list: >>> x = ["spam", "ham"] >>> x.append((1,2)) >>> x ['spam', 'ham', (1, 2)] A list also has an extend method, which takes any sequence (that also includes tuples), and adds *the elements from it* to the end of the list: >>> x = ["spam", "ham"] >>> x.extend((1,2)) >>> x ['spam', 'ham', 1, 2] The append method corresponds to PyList_Append, as you mentioned. It should be quite happy to append a tuple, and will add the tuple itself, not the contents of it. So when you iterate over the list, you'll get tuples. Extending a list can be done with the sequence API. In Python, you can write extend() as +=, indicating that you're adding something onto the end: >>> x = ["spam", "ham"] >>> x += (1, 2) >>> x ['spam', 'ham', 1, 2] This corresponds to PySequence_InPlaceConcat, so if that's the behaviour you want, that would be the easiest way to do it. Based on your other comments, I would suspect that appending the tuples is probably what you want here? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
Chris, you were right to focus on the list pDictData itself. As I said, that is a list of 2-tuples, but I added each of the 2-tuples with PyList_Append, but you can only append a tuple to a list with the extend method. However, there is no append method in the C API as far as I can tell -- hence pDictData is empty. I tried with PyList_SetItem but that doesn't work. Do you know of way to "extend" a list in the C API. Thanks very much. Jen Mar 12, 2022, 13:57 by ros...@gmail.com: > On Sun, 13 Mar 2022 at 08:54, Jen Kris wrote: > >> >> >> pDictData, despite the name, is a list of 2-tuples where each 2-tuple is a >> dictionary object and a string. >> > > Ah, gotcha. In that case, yeah, slicing it will involve referencing > the tuples all the way down the line (adding to their refcounts, so if > there's a borked one, kaboom). > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
On Sun, 13 Mar 2022 at 08:54, Jen Kris wrote: > > > pDictData, despite the name, is a list of 2-tuples where each 2-tuple is a > dictionary object and a string. > Ah, gotcha. In that case, yeah, slicing it will involve referencing the tuples all the way down the line (adding to their refcounts, so if there's a borked one, kaboom). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
pDictData, despite the name, is a list of 2-tuples where each 2-tuple is a dictionary object and a string. Mar 12, 2022, 13:41 by ros...@gmail.com: > On Sun, 13 Mar 2022 at 08:25, Jen Kris via Python-list > wrote: > >> PyObject* slice = PySlice_New(PyLong_FromLong(0), half_slice, 0); >> PyObject* subdata_a = PyObject_GetItem(pDictddata, slice); >> >> On the final line (subdata_a) I get a segfault. I know that the second >> parameter of PyObject_GetItem is a “key” and I suspect that’s where the >> problem comes from, but I don’t understand what a key is in this context. >> > > The key is simply whatever would be in the square brackets in Python > code, so that part looks fine. > > But dictionaries aren't usually subscripted with slices, so I'm a bit > confused as to what's going on here. What exactly is > dictdata/pDictdata? > > Have you confirmed that pDictdata (a) isn't NULL, (b) is the object > you intend it to be, and (c) contains the objects you expect it to? > The segfault might not be from the slice object itself, it might be > from actually iterating over the thing being sliced and touching all > its elements. For instance, if dictdata is actually a list, that call > will be constructing a new list with references to the same elements, > so if one of them is broken (maybe NULL), it'll break badly. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
Thanks to you both. I am going to implement PySequence_Get_Slice now. If I
have trouble then, per comments from Chris Angelico, I will iterate through
pDictData to verify it because I haven't done that. It is not null, however.
Jen
Mar 12, 2022, 13:40 by pyt...@mrabarnett.plus.com:
> On 2022-03-12 21:24, Jen Kris via Python-list wrote:
>
>> I have a C API project where I have to slice a list into two parts.
>> Unfortunately the documentation on the slice objects is not clear enough for
>> me to understand how to do this, and I haven’t found enough useful info
>> through research. The list contains tuple records where each tuple consists
>> of a dictionary object and a string.
>>
>> The relevant part of the Python code is:
>>
>> half_slice = int(len(dictdata) * 0.5)
>> subdata_a = dictdata[half_slice:]
>> subdata_b = dictdata[:half_slice]
>>
>> This is what I’ve done so far with the C API:
>>
>> int64_t Calc_Slices(PyObject* pDictdata, int64_t records_count)
>> {
>> long round_half = records_count * 0.5;
>> PyObject* half_slice = PyLong_FromLong(round_half);
>>
>> PyObject* slice = PySlice_New(PyLong_FromLong(0), half_slice, 0);
>> PyObject* subdata_a = PyObject_GetItem(pDictddata, slice);
>>
>> return 0;
>> }
>>
>> On the final line (subdata_a) I get a segfault. I know that the second
>> parameter of PyObject_GetItem is a “key” and I suspect that’s where the
>> problem comes from, but I don’t understand what a key is in this context.
>>
>> The code shown above omits error handling but none of the objects leading up
>> to the final line is null, they all succeed.
>>
>> Thanks for any ideas.
>>
> Use PySequence_GetSlice to slice the list.
>
> Also, why use floats when you can use integers?
>
> long round_half = records_count / 2;
>
> (In Python that would be half_slice = len(dictdata) // 2.)
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
On Sun, 13 Mar 2022 at 08:25, Jen Kris via Python-list wrote: > PyObject* slice = PySlice_New(PyLong_FromLong(0), half_slice, 0); > PyObject* subdata_a = PyObject_GetItem(pDictddata, slice); > > On the final line (subdata_a) I get a segfault. I know that the second > parameter of PyObject_GetItem is a “key” and I suspect that’s where the > problem comes from, but I don’t understand what a key is in this context. > The key is simply whatever would be in the square brackets in Python code, so that part looks fine. But dictionaries aren't usually subscripted with slices, so I'm a bit confused as to what's going on here. What exactly is dictdata/pDictdata? Have you confirmed that pDictdata (a) isn't NULL, (b) is the object you intend it to be, and (c) contains the objects you expect it to? The segfault might not be from the slice object itself, it might be from actually iterating over the thing being sliced and touching all its elements. For instance, if dictdata is actually a list, that call will be constructing a new list with references to the same elements, so if one of them is broken (maybe NULL), it'll break badly. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem slicing a list with the C API
On 2022-03-12 21:24, Jen Kris via Python-list wrote:
I have a C API project where I have to slice a list into two parts.
Unfortunately the documentation on the slice objects is not clear enough for me
to understand how to do this, and I haven’t found enough useful info through
research. The list contains tuple records where each tuple consists of a
dictionary object and a string.
The relevant part of the Python code is:
half_slice = int(len(dictdata) * 0.5)
subdata_a = dictdata[half_slice:]
subdata_b = dictdata[:half_slice]
This is what I’ve done so far with the C API:
int64_t Calc_Slices(PyObject* pDictdata, int64_t records_count)
{
long round_half = records_count * 0.5;
PyObject* half_slice = PyLong_FromLong(round_half);
PyObject* slice = PySlice_New(PyLong_FromLong(0), half_slice, 0);
PyObject* subdata_a = PyObject_GetItem(pDictddata, slice);
return 0;
}
On the final line (subdata_a) I get a segfault. I know that the second
parameter of PyObject_GetItem is a “key” and I suspect that’s where the
problem comes from, but I don’t understand what a key is in this context.
The code shown above omits error handling but none of the objects leading up to
the final line is null, they all succeed.
Thanks for any ideas.
Use PySequence_GetSlice to slice the list.
Also, why use floats when you can use integers?
long round_half = records_count / 2;
(In Python that would be half_slice = len(dictdata) // 2.)
--
https://mail.python.org/mailman/listinfo/python-list
Problem slicing a list with the C API
I have a C API project where I have to slice a list into two parts.
Unfortunately the documentation on the slice objects is not clear enough for me
to understand how to do this, and I haven’t found enough useful info through
research. The list contains tuple records where each tuple consists of a
dictionary object and a string.
The relevant part of the Python code is:
half_slice = int(len(dictdata) * 0.5)
subdata_a = dictdata[half_slice:]
subdata_b = dictdata[:half_slice]
This is what I’ve done so far with the C API:
int64_t Calc_Slices(PyObject* pDictdata, int64_t records_count)
{
long round_half = records_count * 0.5;
PyObject* half_slice = PyLong_FromLong(round_half);
PyObject* slice = PySlice_New(PyLong_FromLong(0), half_slice, 0);
PyObject* subdata_a = PyObject_GetItem(pDictddata, slice);
return 0;
}
On the final line (subdata_a) I get a segfault. I know that the second
parameter of PyObject_GetItem is a “key” and I suspect that’s where the
problem comes from, but I don’t understand what a key is in this context.
The code shown above omits error handling but none of the objects leading up to
the final line is null, they all succeed.
Thanks for any ideas.
Jen
--
https://mail.python.org/mailman/listinfo/python-list
