Re: [Python-Dev] [Python-checkins] cpython: fix compiler warnings

[Vlad Riscutia]

Why does the function even return a value? As Benjamin said, it is just a
bunch of asserts with return 1 at the end.

I believe another way you can get rid of "statement with no effect" is to
cast return value to void, like (void)_PyUnicode_CHECK(unicode).

Thank you,
Vlad

On Tue, Oct 4, 2011 at 4:57 AM, Benjamin Peterson wrote:

> 2011/10/4 Victor Stinner :
> > Le 04/10/2011 01:34, benjamin.peterson a écrit :
> >>
> >> http://hg.python.org/cpython/rev/afb60b190f1c
> >> changeset:   72633:afb60b190f1c
> >> user:Benjamin Peterson
> >> date:Mon Oct 03 19:34:12 2011 -0400
> >> summary:
> >>   fix compiler warnings
> >>
> >> +++ b/Objects/unicodeobject.c
> >> @@ -369,6 +369,12 @@
> >>  }
> >>  return 1;
> >>  }
> >> +#else
> >> +static int
> >> +_PyUnicode_CheckConsistency(void *op)
> >> +{
> >> +return 1;
> >> +}
> >>  #endif
> >
> > Oh no, please don't do that. Calling _PyUnicode_CheckConsistency() is
> > reserved to debug builds. In release mode, we should not check string
> > consistency (it would slow down Python).
>
> It should be optimized out.
>
> >
> > Yes, there was a warning:
> >
> > Objects/unicodeobject.c:539:13: warning: statement with no effect
> >_PyUnicode_CHECK(unicode);
> >
> > I added these checks recently to ensure that strings are consistent just
> > before exiting (to help me to track down a bug).
> >
> > The right fix is just to replace _PyUnicode_CHECK(unicode) by
> > assert(_PyUnicode_CHECK(unicode)).
>
> But _PyUnicode_CheckConsistency is just a string of assertions. What
> sense does it make to check the return value?
>
>
> --
> Regards,
> Benjamin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Hg tips (was Re: [Python-checkins] cpython (merge default -> default): Merge heads.)

[Vlad Riscutia]

Great tips. Can we add them to the developer guide somewhere?

Thank you,
Vlad

On Thu, Sep 29, 2011 at 12:54 AM, Ezio Melotti wrote:

> Tip 1 -- merging heads:
>
> A while ago Éric suggested a nice tip to make merges easier and since I
> haven't seen many people using it and now I got a chance to use it again, I
> think it might be worth showing it once more:
>
> # so assume you just committed some changes:
> $ hg ci Doc/whatsnew/3.3.rst -m 'Update and reorganize the whatsnew entry
> for PEP 393.'
> # you push them, but someone else pushed something in the meanwhile, so the
> push fails
> $ hg push
> pushing to ssh://h...@hg.python.org/cpython
> searching for changes
> abort: push creates new remote heads on branch 'default'!
> (you should pull and merge or use push -f to force)
> # so you pull the other changes
> $ hg pull -u
> pulling from ssh://h...@hg.python.org/cpython
> searching for changes
> adding changesets
> adding manifests
> adding file changes
> added 4 changesets with 5 changes to 5 files (+1 heads)
> not updating, since new heads added
> (run 'hg heads' to see heads, 'hg merge' to merge)
> # and use "hg heads ." to see the two heads (yours and the one you pulled)
> in the current branch
> $ hg heads .
> changeset:   72521:e6a2b54c1d16
> tag: tip
> user:Victor Stinner 
> date:Thu Sep 29 04:02:13 2011 +0200
> summary: Fix hex_digit_to_int() prototype: expect Py_UCS4, not
> Py_UNICODE
>
> changeset:   72517:ba6ee5cc9ed6
> user:Ezio Melotti 
> date:Thu Sep 29 08:34:36 2011 +0300
> summary: Update and reorganize the whatsnew entry for PEP 393.
> # here comes the tip: before merging you switch to the other head (i.e. the
> one pushed by Victor),
> # if you don't switch, you'll be merging Victor changeset and in case of
> conflicts you will have to review
> # and modify his code (e.g. put a Misc/NEWS entry in the right section or
> something more complicated)
> $ hg up e6a2b54c1d16
> 6 files updated, 0 files merged, 0 files removed, 0 files unresolved
> # after the switch you will merge the changeset you just committed, so in
> case of conflicts
> # reviewing and merging is much easier because you know the changes already
> $ hg merge
> 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
> (branch merge, don't forget to commit)
> # here everything went fine and there were no conflicts, and in the diff I
> can see my last changeset
> $ hg di
> diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
> [...]
> # everything looks fine, so I can commit the merge and push
> $ hg ci -m 'Merge heads.'
> $ hg push
> pushing to ssh://h...@hg.python.org/cpython
> searching for changes
> remote: adding
> changesets
>
> remote: adding manifests
> remote: adding file changes
> remote: added 2 changesets with 1 changes to 1 files
> remote: buildbot: 2 changes sent successfully
> remote: notified python-check...@python.org of incoming changeset
> ba6ee5cc9ed6
> remote: notified python-check...@python.org of incoming changeset
> e7672fe3cd35
>
> This tip is not only useful while merging, but it's also useful for
> python-checkins reviews, because the "merge" mail has the same diff  of the
> previous mail rather than having 15 unrelated changesets from the last week
> because the committer didn't pull in a while.
>
>
> Tip 2 -- extended diffs:
>
> If you haven't already, enable git diffs, adding to your ~/.hgrc the
> following two lines:
>
>> [diff]
>> git = True
>>
> (this is already in the devguide, even if 'git = on' is used there. The
> mercurial website uses git = True too.)
> More info:
> http://hgtip.com/tips/beginner/2009-10-22-always-use-git-diffs/
>
>
> Tip 3 -- extensions:
>
> I personally like the 'color' extension, it makes the output of commands
> like 'hg diff' and 'hg stat' more readable (e.g. it shows removed lines in
> red and added ones in green).
> If you want to give it a try, add to your ~/.hgrc the following two lines:
>
>> [extensions]
>> color =
>>
>
> If you find operations like pulling, updating or cloning too slow, you
> might also want to look at the 'progress' extension, which displays a
> progress bar during these operations:
>
>> [extensions]
>> progress =
>>
>
>
> Tip 4 -- porting from 2.7 to 3.2:
>
> The devguide suggests:
>>
>> hg export a7df1a869e4a | hg import --no-commit -
>>
> but it's not always necessary to copy the changeset number manually.
> If you are porting your last commit you can just use 'hg export 2.7' (or
> any other branch name):
> * using the one-dir-per-branch setup:
>   wolf@hp:~/dev/py/2.7$ hg ci -m 'Fix some bug.'
>   wolf@hp:~/dev/py/2.7$ cd ../3.2
>   wolf@hp:~/dev/py/3.2$ hg pull -u ../2.7
>   wolf@hp:~/dev/py/3.2$ hg export 2.7 | hg import --no-commit -
> * using the single-dir setup:
>   wolf@hp:~/dev/python$ hg branch
>   2.7
>   wolf@hp:~/dev/python$ hg ci -m 'Fix some bug.'
>   wolf@hp:~/dev/python$ hg up 3.2  # here you might enjoy the progress
> extension
>   wolf@hp:~/dev/python$ h

Re: [Python-Dev] Ctypes and the stdlib (was Re: LZMA compression support in 3.3)

[Vlad Riscutia]

I also have some patches sitting on the tracker for some time:

http://bugs.python.org/issue12764
http://bugs.python.org/issue11835
http://bugs.python.org/issue12528 which also fixes
http://bugs.python.org/issue6069 and http://bugs.python.org/issue11920
http://bugs.python.org/issue6068 which also fixes
http://bugs.python.org/issue6493

Thank you,
Vlad

On Tue, Aug 30, 2011 at 6:09 AM, Vinay Sajip wrote:

> Meador Inge  gmail.com> writes:
>
>
> > 1. http://bugs.python.org/issue9041
>
> I raised a question about this patch (in the issue tracker).
>
> > 2. http://bugs.python.org/issue9651
> > 3. http://bugs.python.org/issue11241
>
> I presume, since Amaury has commit rights, that he could commit these.
>
> Regards,
>
> Vinay Sajip
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] FileSystemError or FilesystemError?

[Vlad Riscutia]

+1 for FileSystemError. I see myself misspelling it as FileSystemError if we
go with alternate spelling. I'll probably won't be the only one.

Thank you,
Vlad

On Wed, Aug 24, 2011 at 4:09 AM, Eli Bendersky  wrote:

>
> When reviewing the PEP 3151 implementation (*), Ezio commented that
>> "FileSystemError" looks a bit strange and that "FilesystemError" would
>> be a better spelling. What is your opinion?
>>
>> (*) http://bugs.python.org/issue12555
>>
>
> +1 for FileSystemError
>
> Eli
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL removal question

[Vlad Riscutia]

Removing GIL is interesting work and probably multiple people are willing to
contribute. Threading and synchronization is a deep topic and it might be
that if just one person toys around with removing GIL he might not see
performance improvement (not meaning to offend anyone who tried this,
honestly) but what about forking a branch for this work, with some good
benchmarks in place and have community contribute? Let's say first step
would be just replacing GIL with some fine grained locks with expected
performance degradation but afterwards we can try to incrementally improve
on this.

Thank you,
Vlad

On Wed, Aug 10, 2011 at 8:20 AM, Maciej Fijalkowski wrote:

> On Wed, Aug 10, 2011 at 1:43 PM, Guido van Rossum 
> wrote:
> > On Wed, Aug 10, 2011 at 7:32 AM, David Beazley  wrote:
> >>
> >> On Aug 10, 2011, at 6:15 AM, Nick Coghlan wrote:
> >>
> >>> On Wed, Aug 10, 2011 at 9:09 PM, David Beazley 
> wrote:
>  You're forgetting step 5.
> 
>  5. Put fine-grain locks around all reference counting operations (or
> rewrite all of Python's memory management and garbage collection from
> scratch).
> >>> ...
>  After implementing the aforementioned step 5, you will find that the
> performance of everything, including the threaded code, will be quite a bit
> worse.  Frankly, this is probably the most significant obstacle to have any
> kind of GIL-less Python with reasonable performance.
> >>>
> >>> PyPy would actually make a significantly better basis for this kind of
> >>> experimentation, since they *don't* use reference counting for their
> >>> memory management.
> >>>
> >>
> >> That's an experiment that would pretty interesting.  I think the real
> question would boil down to what *else* do they have to lock to make
> everything work.   Reference counting is a huge bottleneck for CPython to be
> sure, but it's definitely not the only issue that has to be addressed in
> making a free-threaded Python.
> >
> > They have a specific plan, based on Software Transactional Memory:
> >
> http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html
> >
> > Personally, I'm not holding my breath, because STM in other areas has
> > so far captured many imaginations without bringing practical results
> > (I keep hearing about it as this promising theory that needs more work
> > to implement, sort-of like String Theory in theoretical physics).
>
> Note that the PyPy's plan does *not* assume the end result will be
> comparable in the single-threaded case. The goal is to be able to
> compile two *different* pypy's, one fast single-threaded, one
> gil-less, but with a significant overhead. The trick is to get this
> working in a way that does not increase maintenance burden. It's also
> research, so among other things it might not work.
>
> Cheers,
> fijal
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Vlad Riscutia]

OK then. I don't have a *strong* opinion against it, just thought that most
people have one version of Python, maybe 2 versions as in 2.x and 3.x, so I
would understand python2.exe, python3.exe but yeah, it's not that big of a
deal either way.

Thank you,
Vlad

On Fri, Jul 22, 2011 at 6:41 AM, Brian Curtin wrote:

> On Thu, Jul 21, 2011 at 20:30, Vlad Riscutia wrote:
>
>> If versioned filenames are added in addition to python.exe, it still might
>> look confusing for most users: Why do I have python and python3.2
>> executables? What's the difference? I'd rather go with -v argument either
>> way, for people that *know* they want to call Python 3.2 instead of
>> Python 3.1...
>>
>> Thank you,
>> Vlad
>>
>
> Honestly, would it really be that confusing? Seeing python32.exe inside
> C:\Python32 shouldn't be a huge surprise, and ActiveState has been doing
> something like this for years (forever?).
>
> Versioned executables in addition to the standard python.exe is something
> I've wanted for a while, but that's outside of this PEP. This way you could
> have C:\Python27 and C:\Python32 on your path and explicitly open up the
> right one.
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Vlad Riscutia]

If versioned filenames are added in addition to python.exe, it still might
look confusing for most users: Why do I have python and python3.2
executables? What's the difference? I'd rather go with -v argument either
way, for people that *know* they want to call Python 3.2 instead of Python
3.1...

Thank you,
Vlad

On Thu, Jul 21, 2011 at 6:07 PM, Éric Araujo  wrote:

> Hi,
>
> Le 22/07/2011 03:03, Vlad Riscutia a écrit :
> > I'm kind of -1 on changing Python executable name. It would make sense
> for
> > different major versions, where there are known incompatibilities, so
> > python2-python3 would make sense but python31 python32 not that much...
> >
> > If my team is using Python and it gets pre-installed with other
> dev-tools,
> > do I need to let everyone know they must call python*31*? And if we
> upgrade,
> > make sure everyone knows they should now call python*32*? What if we have
> > scripts that call python? Make sure we update all of them whenever minor
> > version is changed?
>
> If I understand correctly, adding versioned filenames like python3.3.exe
> would happen in addition of python.exe, not in replacement.
>
> Regards
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python launcher command line usage (Was: 3.2.1 encoding surprise)

[Vlad Riscutia]

I'm kind of -1 on changing Python executable name. It would make sense for
different major versions, where there are known incompatibilities, so
python2-python3 would make sense but python31 python32 not that much...

If my team is using Python and it gets pre-installed with other dev-tools,
do I need to let everyone know they must call python*31*? And if we upgrade,
make sure everyone knows they should now call python*32*? What if we have
scripts that call python? Make sure we update all of them whenever minor
version is changed?

The way I look at it, most people have only one version of Python installed
at one time and it's just extra burden to make them remember major+minor
version number they have. If you actually install multiple versions, you do
that for a reason and, since you know what you're doing, you would rather
remember to pass correct -v argument to py than users who *just want to use
Python*.

Thank you,
Vlad

On Thu, Jul 21, 2011 at 12:42 PM, Terry Reedy  wrote:

> On 7/20/2011 7:55 PM, Mark Hammond wrote:
>
>> On 21/07/2011 4:38 AM, Terry Reedy wrote:
>>
>>  Many installers first make an organization directory and then an app
>>> directory within that. This annoys me sometimes when they only have one
>>> app to ever install, but is useful when there might really be multiple
>>> directories, as in our case. (Ditto for start menu entries.) This is
>>> what python should have done a decade ago.
>>>
>>
>  I disagree. If we followed that advice we would also be in "\Program
>> Files".
>>
>
> That is not what I suggested. I said let the use pick.
>
>
>  I have no problem with multiple Python versions installed
>> directly off the root, especially given most users probably have a very
>> small number of such installations. I think Python being a developer
>> tool rather than a user app is a reasonable justification for that (and
>> the justification used when the existing scheme was decided)
>>
>
> I put the multiple installations and several other directories into
> /programs. On my next machine, on order, I will use /devel.
>
>
>   > The two proposals
>>
>>> overlap but are not mutually exclusive. For future pythons, 'python33'
>>> is easier to remember and type than 'py -v 3.3' or whatever the proposed
>>> encantation is.
>>>
>>
>> 'py -3.3' - less chars to type than 'python33' and with no need to have
>> every Python directory on your PATH.
>>
>
> My proposal, as I clearly said, was for EXACTLY ONE directory to be added
> to PATH. In spite of Microsoft making is damned difficult for users to edit
> PATH, (and deleted programs not deleting their entries) I added
> 'C:/programs;'. I copied python32/python as py32 and python27/python as
> py27. Those are even fewer characters to type (4 versus 7). Now I can click
> a 'Command Prompt' icon and enter 'py32 -m test.regrtest' and it works
> without cd-ing to /programs/python32. Of course, I will have to re-copy with
> every install, which is why I would like something like this as part of
> installs.
>
>
>
>  IMO it is also simple enough that
>> people will remember it fairly easily.
>>
>
> py32 is even easier to remember.
>
>
>  Also, the launcher supports the ability to select either the 32 or 64bit
>> implementation - so maybe 'python33.exe' isn't really good enough and
>> should reflect the bittedness?
>>
>
> Like py32-6? If I install both Pythons on my new 64 bit machine, I will
> think about it, though I have no need for both now.
>
>
>  A python directory also gives a sensible (though optional) place to put
>>> other interpreters and even python-based apps. The launcher does not.
>>>
>>
>> What other interpreters? IMO it doesn't make sense to have IronPython,
>> jython etc be installed there. Ditto for apps - especially given most
>> apps tend to be tied to a subset of all possible Python versions.
>>
>
> If I install pypy, /programs is exactly where I would put it until I
> somehow discovered that to be a problem. Its startup could be copied as pp26
> or something.
>
>
> My idea may be not so good for general use, even though is now solves my
> problems, but please criticize what I said, allowing for obvious
> modifications like py32 instead of python32, and not a strawman that is
> wildly different.
>
> --
> Terry Jan Reedy
>
>
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

Anyone care to take a look at this?

Thank you,
Vlad

On Mon, Jul 11, 2011 at 8:59 AM, Vlad Riscutia wrote:

> Actually I already attached patch implementing everything to the issue (not
> too much time spent :)). I'm hoping someone can review.
>
> Thank you,
> Vlad
>
>
> On Mon, Jul 11, 2011 at 12:47 AM, "Martin v. Löwis" wrote:
>
>> Am 25.06.2011 18:33, schrieb Vlad Riscutia:
>> > I would like to hear some opinions on this.
>>
>> Sounds fine to me in principle. Warnings can be added to the
>> documentation at any time; please submit a patch to the tracker.
>> As for the API change - make sure you post your proposed API
>> to the list before spending time to implement it.
>>
>> Regards,
>> Martin
>>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

Actually I already attached patch implementing everything to the issue (not
too much time spent :)). I'm hoping someone can review.

Thank you,
Vlad

On Mon, Jul 11, 2011 at 12:47 AM, "Martin v. Löwis" wrote:

> Am 25.06.2011 18:33, schrieb Vlad Riscutia:
> > I would like to hear some opinions on this.
>
> Sounds fine to me in principle. Warnings can be added to the
> documentation at any time; please submit a patch to the tracker.
> As for the API change - make sure you post your proposed API
> to the list before spending time to implement it.
>
> Regards,
> Martin
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

Opened http://bugs.python.org/issue12528 to track this.

Thank you,
Vlad

On Sun, Jun 26, 2011 at 5:48 PM, Vlad Riscutia wrote:

> Well it's not really layout, because alignment is handled by pack option.
> It is how the field gets allocated. At this point I believe it will be more
> complex to come up with custom allocation option, precisely because it's up
> to each compiler to allocate the structure. Such flexibility will add a lot
> of complexity and it might not payoff. This is debatable, but I would go
> with a 3 option strategy at this point - native, GCC, MSVC and if we
> actually find interop issues with other compilers we can look into custom
> allocation.
>
> I will try to refactor the C code to more easily accommodate a mode option
> (while leaving behavior the same for now), then we can decide how the
> library interface should look like.
>
> Thank you,
> Vlad
>
> On Sat, Jun 25, 2011 at 4:37 PM, Greg Ewing 
> wrote:
>
>> Vlad Riscutia wrote:
>>
>>> Longer term though, I think it would be better to add a property on the
>>> Structure class for configurable allocation strategy, for example Native
>>> (default), GCC, MSVC
>>>
>>
>> It could also be good to have a mode which lets you specify
>> *exactly* how the bits are laid out, independent of any
>> particular compiler.
>>
>> --
>> Greg
>>
>> __**_
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
>> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
>> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

Well it's not really layout, because alignment is handled by pack option. It
is how the field gets allocated. At this point I believe it will be more
complex to come up with custom allocation option, precisely because it's up
to each compiler to allocate the structure. Such flexibility will add a lot
of complexity and it might not payoff. This is debatable, but I would go
with a 3 option strategy at this point - native, GCC, MSVC and if we
actually find interop issues with other compilers we can look into custom
allocation.

I will try to refactor the C code to more easily accommodate a mode option
(while leaving behavior the same for now), then we can decide how the
library interface should look like.

Thank you,
Vlad

On Sat, Jun 25, 2011 at 4:37 PM, Greg Ewing wrote:

> Vlad Riscutia wrote:
>
>> Longer term though, I think it would be better to add a property on the
>> Structure class for configurable allocation strategy, for example Native
>> (default), GCC, MSVC
>>
>
> It could also be good to have a mode which lets you specify
> *exactly* how the bits are laid out, independent of any
> particular compiler.
>
> --
> Greg
>
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

This is the cause of both bug reports and yes, it should improve interop
with GCC-compiled code on Windows. My point is that this is not a platform
thing, it's more of a compiler thing. Currently issues appear on Windows for
interop between MSVC Python and other GCC code but since bitfield allocation
is up to the compiler, it could appear on any other platform due to
different compilers being used.

On Sat, Jun 25, 2011 at 12:09 PM, Terry Reedy  wrote:

> On 6/25/2011 12:33 PM, Vlad Riscutia wrote:
>
>> I recently started looking at some ctypes issues. I dug a bit into
>> http://bugs.python.org/**issue6069 <http://bugs.python.org/issue6069> and
>> then I found
>> http://bugs.python.org/**issue11920 <http://bugs.python.org/issue11920>.
>> They both boil down to the fact that
>> bitfield allocation is up to the compiler, which is different in GCC and
>> MSVC. Currently we have hard-coded allocation strategy based on paltform
>> in cfields.c:
>>
>>   if (bitsize /* this is a bitfield request */
>>>
>>
>> &&  *pfield_size /* we have a bitfield open */
>>>  #ifdef MS_WIN32
>>>/* MSVC, GCC with -mms-bitfields */
>>>&&  dict->size * 8 == *pfield_size
>>>  #else
>>>/* GCC */
>>>&&  dict->size * 8<= *pfield_size
>>>  #endif
>>>&&  (*pbitofs + bitsize)<= *pfield_size) {
>>>/* continue bit field */
>>>fieldtype = CONT_BITFIELD;
>>>  #ifndef MS_WIN32
>>>} else if (bitsize /* this is a bitfield request */
>>>&&  *pfield_size /* we have a bitfield open */
>>>&&  dict->size * 8>= *pfield_size
>>>&&  (*pbitofs + bitsize)<= dict->size * 8) {
>>>/* expand bit field */
>>>fieldtype = EXPAND_BITFIELD;
>>>  #endif
>>>
>>
>> So when creating a bitfield structure, it's size can be different on
>> Linux vs Windows.
>>
>> class MyStructure(ctypes.**BigEndianStructure):
>> _pack_  = 1# aligned to 8 bits, not ctypes default of 32
>> _fields_= [
>>("Data0",   ctypes.c_uint32, 32),
>>("Data1",   ctypes.c_uint8, 3),
>>("Data2",   ctypes.c_uint16, 12),
>>   ]
>>
>> sizeof for above structure is 6 on GCC build and 7 on MSVC build. This
>> leads to some confusion and issues, because we can't always interop
>> correctly with code compiled with different compiler than the one Python
>> is compiled with on the platform.
>>
>
> Just curious, are you saying that this is the 'cause' of the two bug
> reports, or 'just' something you discovered while investigating them?
>
>
> > Short term solution is to add a warning in the documentation about this.
>
> For 2.7/3.2, yes.
>
>
> > Longer term though, I think it
>
>> would be better to add a property on the Structure class for
>> configurable allocation strategy, for example Native (default), GCC,
>> MSVC and when allocating the bitfield, use given strategy. Native would
>> behave similar to what happens now, but we would also allow GCC-style
>> allocation on Windows for example and the ability to extend this if we
>> ever run into similar issues with other compilers. This shouldn't be too
>> difficult to implement, will be backwards compatible and it would
>> improve interop. I would like to hear some opinions on this.
>>
>
> If this would allow the MSVC-compilied Python to better access dlls
> compiled with gcc (cygwin) on Windows, definitely -- in 3.3.
> If the new feature is (currently) only useful on Windows, doc should say
> so.
>
> --
> Terry Jan Reedy
>
> __**_
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] ctypes: Configurable bitfield allocation strategy

[Vlad Riscutia]

I recently started looking at some ctypes issues. I dug a bit into
http://bugs.python.org/issue6069 and then I found
http://bugs.python.org/issue11920. They both boil down to the fact that
bitfield allocation is up to the compiler, which is different in GCC and
MSVC. Currently we have hard-coded allocation strategy based on paltform in
cfields.c:

> if (bitsize /* this is a bitfield request */

>&& *pfield_size /* we have a bitfield open */
> #ifdef MS_WIN32
>/* MSVC, GCC with -mms-bitfields */
>&& dict->size * 8 == *pfield_size
> #else
>/* GCC */
>&& dict->size * 8 <= *pfield_size
> #endif
>&& (*pbitofs + bitsize) <= *pfield_size) {
>/* continue bit field */
>fieldtype = CONT_BITFIELD;
> #ifndef MS_WIN32
>} else if (bitsize /* this is a bitfield request */
>&& *pfield_size /* we have a bitfield open */
>&& dict->size * 8 >= *pfield_size
>&& (*pbitofs + bitsize) <= dict->size * 8) {
>/* expand bit field */
>fieldtype = EXPAND_BITFIELD;
> #endif

So when creating a bitfield structure, it's size can be different on Linux
vs Windows.

class MyStructure(ctypes.BigEndianStructure):
_pack_  = 1# aligned to 8 bits, not ctypes default of 32
_fields_= [
   ("Data0",   ctypes.c_uint32, 32),
   ("Data1",   ctypes.c_uint8, 3),
   ("Data2",   ctypes.c_uint16, 12),
  ]

sizeof for above structure is 6 on GCC build and 7 on MSVC build. This leads
to some confusion and issues, because we can't always interop correctly with
code compiled with different compiler than the one Python is compiled with
on the platform. Short term solution is to add a warning in the
documentation about this. Longer term though, I think it would be better to
add a property on the Structure class for configurable allocation strategy,
for example Native (default), GCC, MSVC and when allocating the bitfield,
use given strategy. Native would behave similar to what happens now, but we
would also allow GCC-style allocation on Windows for example and the ability
to extend this if we ever run into similar issues with other compilers. This
shouldn't be too difficult to implement, will be backwards compatible and it
would improve interop. I would like to hear some opinions on this.

Thank you,
Vlad
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com