[Cython] Docs repository

[Robert Bradshaw]

The cython documentation is now part of the main repository, see
https://github.com/cython/cython/tree/master/docs .
https://github.com/cython/cython-docs should no longer be used
(perhaps it should be removed entirely, are there any/many inbound
links?)

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] [cython-users] Cython .pxd introspection: listing defined constants

[Robert Bradshaw]

On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King wk...@drexel.edu wrote:
 First, here's a reminder of my end goal:

 On Wed, Feb 09, 2011 at 12:23:25PM -0500, W. Trevor King wrote:
 Now I want to expose those constants in Python, so I have `expose.pyx`:

   cimport mylib

   CONST_A = mylib.CONST_A
       CONST_B = mylib.CONST_B
   ...

 But the last part seems pretty silly.  I'd like to do something like

       cimport mylib
   import sys

   for name in dir(mylib):
       setattr(sys.modules[__name__], name, getattr(mylib, name))

 which compiles fine, but fails to import with...

 The discussion eventually led to:

 On Wed, Feb 09, 2011 at 12:22:41PM -0800, Robert Bradshaw wrote:
 cimport * will pollute expose.pyx's C-level namespace. You can use
 them just fine there. If you need to access these constants from
 Python, you do have to explicitly expose them.

 In an attempt to make it easier to explicitly expose them to other
 Python modules at compile time, I've been working through the Cython
 source, and found that when cythoning expose.pyx, all the items
 declared in mylib.pxd are stored in `Cython.Compiler.Symtab.Entry`s in
 mylib.pxd's `Cython.Compiler.Symtab.ModuleScope` instance.

 By tweaking
  Cython.Compiler.Node.CImportStatNode.generate_function_definitions
 to be
  def generate_function_definitions(self, env, code):
      modules = [m for m in env.cimported_modules if m.module_name == 
 self.module_name]
      if len(modules)  0:
          module = modules[0]
          for name,entry in module.entries.iteritems():
              code.putln('/* %s.%s */' % (self.module_name, name))
 I can add a comment to extern.c listing all the stuff I'd declared in
 `cdef extern` blocks in mylib.pxd.  That's nice, but not very useful
 (it does show, however, that Cython is not forgetting about the
 definitions as I previously thought).

 What I'm missing is a way to bind the ModuleScope namespace to a name
 in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib,
 name)` will work in expose.pyx.  If it would be easier to pre-compute
 the result of these commands and hard-code them in `extern.c` at
 compile time, that's fine.

 For example, my comment-generating version of
 `generate_function_definitions` is not far from being able to generate
 the results of `dir(mylib)`, it would just need to wrap the results in
 a list and place references to that list in place of the stock `dir()`
 execution code when the dir argument resolved to the cimported module.
 Yuck ;).

Cython modules have two namespaces, the Python level one and the C
level one. This separation is necessary, as the C level one may
contain objects that are unrepresentable in the Python-level layer,
and get statically bound (e.g. one would want to mark them as
read-only at the very least). Many users also like the fact that the
C-level implementation details of their Cython modules do not get
leaked out to their Python namespaces (though use of __all__ could
provide, but not enforce, similar behavior).

You have also hit into the thorny issue that .pxd files are used for
many things. They may be pure C library declarations with no Python
module backing, they may be declarations of (externally implemented)
Python modules (such as numpy.pxd), or they may be declarations for
Cython-implemented modules.

In terms of your specific question, I don't think hijacking the
builtin dir and getattr to blur this line.

 It seems like it would be easier to generate some kind of wrapper
 class (PxdModule?) for mylib when it is cimported (at compile time),
 and then further interactions would take care of themselves (at run
 time).

 Does anyone know how I would go about doing this?

Would such an object be created anew for every module that cimports
the declaration file?

I have toyed with the idea of subclassing the module object itself for
better support of C-level attributes from the Python (and Cython)
namespaces.

Here's another idea, what if extern blocks could contain cpdef
declarations, which would automatically generate a Python-level
wrappers for the declared members (if possible, otherwise an error)?

 On an administrative level, since this would seem to require altering
 the Cython source code, should I move this discussion to cython-devel,
 or does splitting the thread across two lists make it too hard to
 follow?

Yes, it would probably be best to move this thread over there. A
succinct summary of what issue you are trying to solve would probably
be helpful now anyways.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Control flow graph

[Robert Bradshaw]

On Tue, Feb 15, 2011 at 2:24 AM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 15.02.2011 10:59:
 2011/2/15 Stefan Behnelstefan...@behnel.de:
 Robert Bradshaw, 15.02.2011 08:21:

 On Mon, Feb 14, 2011 at 9:49 PM, Vitja Makarov wrote:

 2011/2/15 Robert Bradshaw:

 On Sun, Feb 13, 2011 at 11:40 PM, Vitja Makarov wrote:

 Hi!

 In order to implement reaching definitions algorithm.
 I'm now working on control-flow (or data-flow) graph.

 Here is funny picture made with graphviz ;)

 http://piccy.info/view3/1099337/ca29d7054d09bd0503cefa25f5f49420/1200/

 Cool. Any plans on handling exceptions?

 Sure, but I don't have much time for this :(

 Linear block inside try...except body should be split by assignments
 and each subblock should point to exception handling entry point.

 Would every possible failing sub-expression have to point to the
 exception handling point(s)?


 Not sure here as now graph node is list of NameNode ref and AssignmentNode.
 I'm not sure but it seems that every sub-expression could raise
 exception now, is there any flag?

 Well, in most cases (especially the interesting ones), this will be the
 function exit point, so it'll be easy. And in some cases, we may be able to
 infer that a specific exception that an expression (e.g. arithmetics or a
 'raise' statement) can raise will not get caught by a given except clause
 (although that's certainly a tricky optimisation).

 But in general, I think any subexpression that potentially raises an
 exception must point to the next exception handling point.


 Right, and each exception handling point should have reference to the
 next one or function exit point.


 I suppose it depends on whether you'll be handling more than assignment
 tracking.

 We *may* get away with a statement-level graph in that case, but I somehow
 doubt it already. For example, list comprehensions leak their variable in
 Py2 code, so it's important to know if they are executed or not, and they
 may appear in any kind of expression.


 This should be special case for scoped expression node.

 Now I build graph from scratch it include only name node references
 and assignments, and positions marks to draw gv.

 May be node tree should be translated into graph and then local
 variables graph could be created from it.

 On the other hand CreateAbstractGraph transformation could be used to
 create specialized graph.

 Note that building that graph is only a smaller part of the work. It needs
 to be queriable efficiently. These graphs can easily get huge, so if the
 graph needs to get traversed for each little piece of information, that'll
 seriously slow things down.

On this note, one thought that I had is that one could traverse the
graph while holding a single state object which nodes could read from
and write to. This would alleviate the need to need to query the graph
and simultaneously store the (large) number of potential states at any
point in the code path.

 The current (limited) support for control flow analysis initially crashed
 for a beautiful code example that had lots of if-blocks in a row, because
 it was using recursive traversal. I refactored it back then, but he have to
 make sure the new implementation stays scalable.

 It's worth reading a bit of literature here. AFAIR, I posted a couple of
 sources to the list a while back.

 Stefan
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev

___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] switching mailing lists

[Robert Bradshaw]

On Fri, Feb 11, 2011 at 4:57 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 10.02.2011 20:30:
 On Thu, Feb 10, 2011 at 1:39 AM, Stefan Behnel wrote:
 Dag Sverre Seljebotn, 10.02.2011 10:08:
 How about librelist.com? Here's a blog post with some more background:

 http://zedshaw.com/blog/2009-12-03.html

 I like their philosophy and I like their archives. I also like that you can
 actually *get* their archives.

 Yes, this is nice.

 They do seem to have a couple of lists there already, including some Python
 or Ruby related lists. That doesn't guarantee librelist.com is there to
 stay forever, but I wouldn't mind giving them a try.

 BTW, we should first set up the list, then re-subscribe gmane, then
 announce the new list to have people subscribe (or bulk subscribe them, in
 case that works).

 Google groups works well for my workflow, but I know you don't like
 it. I could go with librelist. The interface seems nice and clean and
 it's fast enough. Most of the lists it hosts are empty or dead... Do
 you know what the moderation options are like? Hopefully they'll be
 around for a while, as this is the 2nd (3rd?) time we've had to move
 lists...

 One caveat I found: you can't name a list cython-...@librelist.com,
 because they don't allow hyphens in list names (reserved as command
 separators). They only allow dots as special characters, which is somewhat
 unusual IMHO. Then the list would become cython@librelist.com, or
 cythondev@... or simply cython@ The latter would be fine with me,
 given that we have a separate cython-users mailing list anyway.

I think we should definitely have dev or devel in the name. Very
strange that they don't allow a hyphen: not a blocker, but as we have
not commitment yet it's worth keeping our eyes open for an equivalent
service that does.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] switching mailing lists

[Robert Bradshaw]

On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith n...@pobox.com wrote:
 On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
 rober...@math.washington.edu wrote:
 I think we should definitely have dev or devel in the name. Very
 strange that they don't allow a hyphen: not a blocker, but as we have
 not commitment yet it's worth keeping our eyes open for an equivalent
 service that does.

 Of course you could also just find mailman hosting somewhere -- I host
 some lists myself, and it works well and people are familiar with it.
 It's also easy to import archives[1] and the default archive pages
 have links to download the raw archive. Perhaps the scipy.org folks
 would be willing to help? They already host a number of lists, not
 just for scipy[2].

 [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
 [2] http://mail.scipy.org/mailman/listinfo

I like that idea a lot better than going with a random, un-related
list hosting service. (I had thought about python.org too, would that
be an appropriate fit?) SciPy is certainly very related, but I don't
want to pigeon-hole ourselves as being only for scientific computation
(even if this is our biggest userbase).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] switching mailing lists

[Robert Bradshaw]

On Fri, Feb 11, 2011 at 10:58 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 11.02.2011 19:50:
 On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith wrote:
 On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
 rober...@math.washington.edu  wrote:
 I think we should definitely have dev or devel in the name. Very
 strange that they don't allow a hyphen: not a blocker, but as we have
 not commitment yet it's worth keeping our eyes open for an equivalent
 service that does.

 Of course you could also just find mailman hosting somewhere -- I host
 some lists myself, and it works well and people are familiar with it.
 It's also easy to import archives[1] and the default archive pages
 have links to download the raw archive. Perhaps the scipy.org folks
 would be willing to help? They already host a number of lists, not
 just for scipy[2].

 [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
 [2] http://mail.scipy.org/mailman/listinfo

 I like that idea a lot better than going with a random, un-related
 list hosting service. (I had thought about python.org too, would that
 be an appropriate fit?) SciPy is certainly very related, but I don't
 want to pigeon-hole ourselves as being only for scientific computation
 (even if this is our biggest userbase).

 :) You read my mind here.

 However, my first thought about python.org was that python-dev and
 cython-dev are pretty close, especially given that python is often
 referred to as cpython. This may trigger some confusion.

We could go with cython-devel. The fact that the first letter is
different is more comforting to me than a different suffix in these
days of address auto-completion. Should I go ahead and ask?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] Cython mailing list

[Robert Bradshaw]

Python mailman admins,

The Cython project (http://cython.org) is in need of a new mailing
list, as our current hosting provider is discontinuing services. We
were thinking it would make sense to host it at mail.python.org. Would
you be willing to provide us with cython-de...@python.org?

Thanks,
Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] switching mailing lists

[Robert Bradshaw]

On Fri, Feb 11, 2011 at 11:06 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 11.02.2011 20:02:
 On Fri, Feb 11, 2011 at 10:58 AM, Stefan Behnel wrote:
 Robert Bradshaw, 11.02.2011 19:50:
 On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith wrote:
 On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
 rober...@math.washington.edu    wrote:
 I think we should definitely have dev or devel in the name. Very
 strange that they don't allow a hyphen: not a blocker, but as we have
 not commitment yet it's worth keeping our eyes open for an equivalent
 service that does.

 Of course you could also just find mailman hosting somewhere -- I host
 some lists myself, and it works well and people are familiar with it.
 It's also easy to import archives[1] and the default archive pages
 have links to download the raw archive. Perhaps the scipy.org folks
 would be willing to help? They already host a number of lists, not
 just for scipy[2].

 [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
 [2] http://mail.scipy.org/mailman/listinfo

 I like that idea a lot better than going with a random, un-related
 list hosting service. (I had thought about python.org too, would that
 be an appropriate fit?) SciPy is certainly very related, but I don't
 want to pigeon-hole ourselves as being only for scientific computation
 (even if this is our biggest userbase).

 :) You read my mind here.

 However, my first thought about python.org was that python-dev and
 cython-dev are pretty close, especially given that python is often
 referred to as cpython. This may trigger some confusion.

 We could go with cython-devel. The fact that the first letter is
 different is more comforting to me than a different suffix in these
 days of address auto-completion. Should I go ahead and ask?

 +1

We now have cython-de...@python.org

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] switching mailing lists

[Robert Bradshaw]

On Thu, Feb 10, 2011 at 1:39 AM, Stefan Behnel stefan...@behnel.de wrote:
 Dag Sverre Seljebotn, 10.02.2011 10:08:
 (I think top-posting is OK in this one instance...)

 You're excused. ;)


 How about librelist.com? Here's a blog post with some more background:

 http://zedshaw.com/blog/2009-12-03.html

 I like their philosophy and I like their archives. I also like that you can
 actually *get* their archives.

Yes, this is nice.

 Maybe we can even manage to drop our
 existing archive there, but it seems like we'd have to ask if/how that
 would work.

 They do seem to have a couple of lists there already, including some Python
 or Ruby related lists. That doesn't guarantee librelist.com is there to
 stay forever, but I wouldn't mind giving them a try.

 BTW, we should first set up the list, then re-subscribe gmane, then
 announce the new list to have people subscribe (or bulk subscribe them, in
 case that works).

Google groups works well for my workflow, but I know you don't like
it. I could go with librelist. The interface seems nice and clean and
it's fast enough. Most of the lists it hosts are empty or dead... Do
you know what the moderation options are like? Hopefully they'll be
around for a while, as this is the 2nd (3rd?) time we've had to move
lists...

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Problem with generate_stararg_init_code()

[Robert Bradshaw]

On Tue, Feb 8, 2011 at 2:21 AM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 08.02.2011 10:16:
 Trying to merge latest changes in argument parsing code I found that
 it still uses direct returns

 https://github.com/cython/cython/blob/master/Cython/Compiler/Nodes.py#L2624

 Like this:
          if self.starstar_arg:
              self.starstar_arg.entry.xdecref_cleanup = 0
              code.putln('%s = PyDict_New(); if (unlikely(!%s)) return %s;' % 
 (
                      self.starstar_arg.entry.cname,
                      self.starstar_arg.entry.cname,
                      self.error_value()))
              code.put_gotref(self.starstar_arg.entry.cname)
 Or this:
              if self.starstar_arg:
                  code.putln()
                  code.putln(if (unlikely(!%s)) { % 
 self.star_arg.entry.cname)
                  code.put_decref_clear(self.starstar_arg.entry.cname,
 py_object_type)
                  code.putln('return %s;' % self.error_value())
                  code.putln('}')
              else:
                  code.putln(if (unlikely(!%s)) return %s; % (
                          self.star_arg.entry.cname, self.error_value()))

 That's not good because current scope and refnanny context is already
 created and should be freed.

 These aren't really critical bugs as they only deal with memory problems.
 Unless you want to rework them now, I think this is something that we
 should clean up as part of the DefNode/CFuncDefNode refactoring during the
 workshop.

They are certainly bugs, so please file a ticket or add a note in the
code at least.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Switching from Py_UNICODE to Py_UCS4

[Robert Bradshaw]

On Sun, Feb 6, 2011 at 12:45 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 04.02.2011 19:50:
 On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel wrote:
 I am a bit concerned about the performance overhead of the Py_UCS4 to
 Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
 maybe that's both uncommon and negligible.

 I think so. If users deal with Py_UNICODE explicitly, they'll likely type
 their respective variables anyway, so that there won't be an intermediate
 step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at 
 all.

 Coming back to this once more: if the PEP gets implemented, we will only
 know at C compile time (Py=3.3 or not) if the result of indexing
 (including for-loop iteration) is Py_UCS4 or Py_UNICODE. For Cython's type
 inference, Py_UCS4 is therefore the more correct guess. So my proposal
 stands to always infer Py_UCS4 instead of Py_UNICODE for indexing, even if
 we ignore surrogate pairs in narrow Python builds.

 I will implement this for now, so that we can see what it gives.

Yes, that makes sense.

 Also, this would be inconsistant with
 python-level slicing, indexing, and range, right?

 Yes, it does not match well with slicing and indexing. That's the problem
 with narrow builds in both CPython and Cython. Only the PEP can fix that by
 basically dropping the restrictions of a narrow build.

 Lets let indexing do what indexing does.

 Ok. So you'd continue to get whatever CPython returns for indexing, i.e.
 Py_UNICODE in Py=3.2 and Py_UCS4 in Python versions that implement the
 PEP. That includes separate code points for surrogate pairs on narrow builds.

Yep, exactly. Note that indexing taking into account surrogate pairs
can be O(n) rather than O(1) as well.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage problem after fixing #654

[Robert Bradshaw]

On Sat, Jan 29, 2011 at 2:24 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 On Fri, Jan 28, 2011 at 9:14 PM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 28.01.2011 22:49:
 On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel wrote:
 And given that Cython cannot know that the pointer is actually not a
 pointer, it generates the expected code here which only the C compiler can
 detect as invalid.
 This needs fixing in Sage in one way or another.

 Cython didn't used to support tyepdef'd arrays, so that's why it was
 declared as a pointer, but yes, that should be fixed. I'll do that in
 our copy, and it will go out with the 0.14.1 fixes (even though we
 don't need it yet). There's still the case of, say,

      mpfr_init2(self._args[f(i)], self.domain.prec())

 where f is a cdef int -  int function. Do we handle that?

 Sure. The problem isn't (any longer) that we *forget* to put things into a
 temp, the problem is that we try to put things into a temp now that we
 cannot store away.

 I've been dropping some cases in my latest commits where the temp copying
 mechanism was triggered needlessly. It's basically a matter of properly
 implementing is_simple() for an ExprNode in order to decide when a non-temp
 result is side-effect free.

 I'm still not sure if a coerce_to_owned() method would be worthwhile. As
 long as SimpleCallNode is the only place where this is actually needed,
 it's fine to leave the code there. Can anyone think of other places where
 the order of C value evaluation matters? Tuples and other container
 literals may qualify, but given that they use only Python values, I don't
 see it being a problem there.

 I'm not sure we handle the optimization of, e.g, a in (x, y, z)
 There might be others as well.

 Fixing Sage is turning out to be non-trivial...there are many places
 that assume mpz_t is (compatible with) a void*. We also have the
 problem that Cython doesn't let you take the address of an array, as
 it is a non-lvalue, but that's legal in C (and used in Sage).

There's another problem, assigning things to a temporary C++ class or
struct can have side effects (as well as be inefficient).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] [cython-users] Re: Cython 0.14.1 has been released

[Robert Bradshaw]

On Fri, Feb 4, 2011 at 7:00 AM, Stefan Behnel stefan...@behnel.de wrote:
 robertwb, 04.02.2011 11:51:

 Cython 0.14.1 has just been released. This release is primarily a bug
 fix release building on top of 0.14.

 Download: http://cython.org/release/Cython-0.14.1rc3.tar.gz

 You may prefer getting it from PyPI or from this URL:

 http://cython.org/release/Cython-0.14.1.tar.gz

Oops, I posted the rc3 link. I've done a 302 redirect to the correct
one for the time being.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Switching from Py_UNICODE to Py_UCS4

[Robert Bradshaw]

On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 29.01.2011 10:01:
 On Fri, Jan 28, 2011 at 11:37 PM, Stefan Behnel wrote:
 there is a recent discussion on python-dev about a new memory layout for
 the unicode type in CPython 3.3(?), proposed by Martin von Löwis (so it's
 serious ;)

 http://comments.gmane.org/gmane.comp.python.devel/120784

 That's an interesting PEP, I like it.

 Yep, after some discussion, I started liking it too. Even if it means I'll
 have to touch a lot of code in Cython again. ;)


 If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit
 unsigned int), which I had completely lost from sight. It's public and
 undocumented and has been there basically forever, but it's a much nicer
 type to support than Py_UNICODE, which changes size based on build time
 options. Py_UCS4 is capable of representing any Unicode code point on any
 platform.

 So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4
 internally (without breaking user code which can continue to use either of
 the two explicitly). This means that loops over unicode objects will infer
 Py_UCS4 as loop variable, as would indexing. It would basically become the
 native C type that 1 character unicode strings would coerce to and from.
 Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value
 is too large in the given CPython runtime, as would write access to unicode
 objects (in case anyone really does that) outside of the platform specific
 Py_UNICODE value range. Writing to unicode buffers will be dangerous and
 tricky anyway if the above PEP gets accepted.

 I am a bit concerned about the performance overhead of the Py_UCS4 to
 Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
 maybe that's both uncommon and negligible.

 I think so. If users deal with Py_UNICODE explicitly, they'll likely type
 their respective variables anyway, so that there won't be an intermediate
 step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at all.


 One open question that I see is whether we should handle surrogate pairs
 automatically. They are basically a split of large Unicode code point
 values (65535) into two code points in specific ranges that are safe to
 detect. So we could allow a 2 'character' surrogate pair in a unicode
 string to coerce to one Py_UCS4 character and coerce that back into a
 surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that
 this would only work for single characters, not for looping or indexing
 (without the PEP, that is). So it's somewhat inconsistent. It would work
 well for literals, though. Also, we'd have to support it for 'in' tests, as
 a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the
 character is in the string.

 No, I don't think we should handle surrogate pairs automatically, at
 least without making it optional--this could be a significant
 performance impact with little benefit for most users. Using these
 higher characters is rare, but using them on a non USS4 build is
 probably even rarer.

 Well, basically they are the only way to use 'wide' Unicode characters on
 16bit Unicode builds.

 I think a unicode string of length 2 should be able to coerce into a
 Py_UCS4 value at runtime instead of raising the current exception because
 it's too long.

Sure, that's fine by me.

 For the opposite direction, integer to unicode string, you
 already get a string of length 2 on narrow builds, that's how
 unichr()/chr() work in Python 2/3. So, in a way, it's actually more
 consistent with how narrow builds work today.

OK.

 The only reason this isn't
 currently working in Cython is that Py_UNICODE is too small on narrow
 builds to represent the larger Unicode code points. If we switched to
 Py_UCS4, the problem would go away in narrow builds now and code could be
 written today that would easily continue to work efficiently in a post-PEP
 CPython as it wouldn't rely on the deprecated (and then inefficient)
 Py_UNICODE type anymore.

 What about supporting surrogate pairs in 'in' tests only on narrow
 platforms? I mean, we could simply duplicate the search code for that,
 depending on how large the code point value really is at runtime. That code
 will become a lot more involved anyway when the PEP gets implemented.

Sure. This shouldn't have non-negligible performance overhead for the
simple case, and would be consistent with coercing to a 2-character
Unicode as above then doing the Python in operator.

 Also, this would be inconsistant with
 python-level slicing, indexing, and range, right?

 Yes, it does not match well with slicing and indexing. That's the problem
 with narrow builds in both CPython and Cython. Only the PEP can fix that by
 basically dropping the restrictions of a narrow build.

Lets let indexing do what indexing does.

- Robert
___
Cython-dev mailing list
Cython-dev

Re: [Cython] Cython workshop

[Robert Bradshaw]

On Sun, Jan 23, 2011 at 2:50 AM, Stefan Behnel stefan...@behnel.de wrote:
 Dag Sverre Seljebotn, 23.01.2011 10:12:
 On 01/23/2011 09:53 AM, Robert Bradshaw wrote:
 On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote:

 Dag Sverre Seljebotn, 20.01.2011 12:26:

 Starting new thread in case somebody did not see the hijacking.

 I've created this wiki page:

 http://wiki.cython.org/workshop1

 If you're interested in coming to a Cython workshop, please fill in your
 details in the table to help decide when and where a workshop should be
 held.

 Current status so far: 4 core developers have signed in, 5 people in total,
 all from Europe. Robert said he'd like to join in, too.

 Have there been any further off-list replies so far?

 I've spoken to two people, both of whom won't be able to do much the
 first half of this year.

 Still, I think close to now is a very good time to do it, because 0.14.x
 is nearing stability and thus leaving the focus but needs a serious
 docsprint, 0.15 is in the pipeline and worth putting some work into and 1.0
 is getting in sight and worth discussing. Certainly enough to fill four days.


 In case we actually want to advertise and run this before the end of April
 (or even March, and Vitja's visa will likely take a while to get), it would
 be good if the remaining core developers and other interested parties could
 speak up soon, say, within a week's time, so that we can decide on date and
 location.

 The first half of March doesn't work for me, and April would be
 tricky, but I might be able to do March 30-April 3. More ideal would
 be May-June-July, but that conflicts directly with Stefans
 availability--are there any weekends in there that would work or are
 those months right out?

 Tricky for me. The first weekend in June might work, but I'd have to check
 that with my employer. End of June may work better.


 I think 4 days is about the right amount of time.

 Within the months I listed there's good chances of things working for
 me; although April-June is better than July. March 30 - April 3 should work.

 Assuming there's never a date that won't be tricky for any of us, it seems
 that the first weekend in April could make it.

Lets officially plan on that.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] Fwd: sage.math sysadmin

[Robert Bradshaw]

This is relevant to us, as we use the sage.math infrastructure. Thanks
Joanna, and William, for supporting and letting us use your equipment.

- Robert

-- Forwarded message --
From: Joanna jga...@speakeasy.org
Date: Sat, Jan 29, 2011 at 4:25 PM
Subject: sage.math sysadmin
To: sage.math users sagemath-us...@googlegroups.com


Hello. I am William's new sysadmin for the sage.math cluster.
I have a couple projects to start out with on the cluster (woohoo,
expanded offsite backups), but I'll also try to pick up requests
from this list as they happen. You can also just email me if
you notice something that needs attention and might not interest
everyone else.

Thanks!

-- Joanna

--
You received this message because you are subscribed to the Google
Groups sage.math users group.
To post to this group, send email to sagemath-us...@googlegroups.com
To unsubscribe from this group, send email to
sagemath-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sagemath-users
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

On Fri, Jan 28, 2011 at 9:39 PM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 29.01.2011 01:55:
 On Fri, Jan 28, 2011 at 3:38 PM, Dominic Sacré wrote:
 On Fri, Jan 28, 2011 at 11:40 AM, Robert Bradshaw wrote:
 New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz

 Is it just me, or do the release candidates break isinstance() when
 testing against a tuple of multiple types? When I do isinstance(x,
 (Foo, Bar)) a Foo object will be recognized, but a Bar object will
 not. The same code used to work fine in older versions, including
 0.14.

 Thanks for the report. I did fix one isinstance bug, perhaps I
 introduced another one (though I thought I tested this...). I'll make
 sure this works before release.

 My bad. There was code in Optimize.py, line 2003:

             if type_check_function not in tests:
                 tests.append(type_check_function)
                 test_nodes.append(...)

 Basically, it executes each type check function only once, but regardless
 of the type it is testing against. Works well for builtin types, doesn't
 work for extension types.

 https://github.com/cython/cython/commit/c14533e4a00e789df8d800fa9f4cc099faabb67e

 Hmm, I'm not sure how to merge commits over git branches with hg-git...

I've cherry-picked it in.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Switching from Py_UNICODE to Py_UCS4

[Robert Bradshaw]

On Fri, Jan 28, 2011 at 11:37 PM, Stefan Behnel stefan...@behnel.de wrote:
 Hi,

 there is a recent discussion on python-dev about a new memory layout for
 the unicode type in CPython 3.3(?), proposed by Martin von Löwis (so it's
 serious ;)

 http://comments.gmane.org/gmane.comp.python.devel/120784

That's an interesting PEP, I like it.

 If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit
 unsigned int), which I had completely lost from sight. It's public and
 undocumented and has been there basically forever, but it's a much nicer
 type to support than Py_UNICODE, which changes size based on build time
 options. Py_UCS4 is capable of representing any Unicode code point on any
 platform.

 So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4
 internally (without breaking user code which can continue to use either of
 the two explicitly). This means that loops over unicode objects will infer
 Py_UCS4 as loop variable, as would indexing. It would basically become the
 native C type that 1 character unicode strings would coerce to and from.
 Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value
 is too large in the given CPython runtime, as would write access to unicode
 objects (in case anyone really does that) outside of the platform specific
 Py_UNICODE value range. Writing to unicode buffers will be dangerous and
 tricky anyway if the above PEP gets accepted.

I am a bit concerned about the performance overhead of the Py_UCS4 to
Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
maybe that's both uncommon and negligible.

 One open question that I see is whether we should handle surrogate pairs
 automatically. They are basically a split of large Unicode code point
 values (65535) into two code points in specific ranges that are safe to
 detect. So we could allow a 2 'character' surrogate pair in a unicode
 string to coerce to one Py_UCS4 character and coerce that back into a
 surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that
 this would only work for single characters, not for looping or indexing
 (without the PEP, that is). So it's somewhat inconsistent. It would work
 well for literals, though. Also, we'd have to support it for 'in' tests, as
 a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the
 character is in the string.

 Comments?

No, I don't think we should handle surrogate pairs automatically, at
least without making it optional--this could be a significant
performance impact with little benefit for most users. Using these
higher characters is rare, but using them on a non USS4 build is
probably even rarer. Also, this would be inconsistant with
python-level slicing, indexing, and range, right?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage problem after fixing #654

[Robert Bradshaw]

On Fri, Jan 28, 2011 at 9:14 PM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 28.01.2011 22:49:
 On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel wrote:
 And given that Cython cannot know that the pointer is actually not a
 pointer, it generates the expected code here which only the C compiler can
 detect as invalid.
 This needs fixing in Sage in one way or another.

 Cython didn't used to support tyepdef'd arrays, so that's why it was
 declared as a pointer, but yes, that should be fixed. I'll do that in
 our copy, and it will go out with the 0.14.1 fixes (even though we
 don't need it yet). There's still the case of, say,

      mpfr_init2(self._args[f(i)], self.domain.prec())

 where f is a cdef int -  int function. Do we handle that?

 Sure. The problem isn't (any longer) that we *forget* to put things into a
 temp, the problem is that we try to put things into a temp now that we
 cannot store away.

 I've been dropping some cases in my latest commits where the temp copying
 mechanism was triggered needlessly. It's basically a matter of properly
 implementing is_simple() for an ExprNode in order to decide when a non-temp
 result is side-effect free.

 I'm still not sure if a coerce_to_owned() method would be worthwhile. As
 long as SimpleCallNode is the only place where this is actually needed,
 it's fine to leave the code there. Can anyone think of other places where
 the order of C value evaluation matters? Tuples and other container
 literals may qualify, but given that they use only Python values, I don't
 see it being a problem there.

I'm not sure we handle the optimization of, e.g, a in (x, y, z)
There might be others as well.

Fixing Sage is turning out to be non-trivial...there are many places
that assume mpz_t is (compatible with) a void*. We also have the
problem that Cython doesn't let you take the address of an array, as
it is a non-lvalue, but that's legal in C (and used in Sage).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

On Sun, Jan 23, 2011 at 3:16 PM, Arfrever Frehtes Taifersar Arahesis
arfrever@gmail.com wrote:
 Test failures with Python 2.7:

 ==
 ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll)
 --
 Traceback (most recent call last):
  File 
 /var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/Debugger/Tests/TestLibCython.py,
  line 177, in setUp
    python_version_number = [int(a.strip()) for a in 
 python_version.strip('()').split(',')[:3]]
 ValueError: invalid literal for int() with base 10: 'sys.version_info(major=2'

 ==
 ERROR: runTest (__main__.EndToEndTest)
 End-to-end basic_cythonize.srctree
 --
 Traceback (most recent call last):
  File runtests.py, line 797, in setUp
    os.path.join('tests', 'build', self.treefile), self.workdir)
  File 
 /var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py,
  line 176, in unpack_source_tree
    f = open(tree_file)
 IOError: [Errno 2] No such file or directory: 
 'tests/build/basic_cythonize.srctree'

I think all of these failed because of a unfixed chdir in the failed
debugging test. Please try again with the latest rc.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

On Tue, Jan 18, 2011 at 12:56 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 A release candidate is up at
 http://cython.org/release/Cython-0.14.1rc1.tar.gz . Hopefully there
 shouldn't be many issues, there's just a pile of bugfixes on top of
 0.14. I also started http://wiki.cython.org/ReleaseNotes-0.14.1

New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] debuger test failure

[Robert Bradshaw]

On Fri, Jan 28, 2011 at 3:59 AM, mark florisson
markflorisso...@gmail.com wrote:
 On 28 January 2011 08:40, Robert Bradshaw rober...@math.washington.edu 
 wrote:
 On Tue, Jan 25, 2011 at 1:46 PM, Vitja Makarov vitja.maka...@gmail.com 
 wrote:
 It's okay now. But with few warnings.

 vitja@vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp
 --no-annotate libc
 Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
 [GCC 4.4.5]

 Running tests against Cython 0.14.1rc2

 test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning:
 codefile.pyx:25:6: 'eggs' redeclared
 Function __pyx_pf_8codefile_5outer_inner not defined.
 ..
 --
 Ran 22 tests in 8.263s

 I would like to get rid of these warnings if possible...

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


 I see you duplicated the checks for the gdb and python versions in a
 global function. Are you still working on removing the duplication?
 (I'm talking about the code in
 Cython/Debugger/Tests/TestLibCython:GdbDebuggerTestCase.setUp).

The global function caches its value, so it's only doing the actual
check once, but we do want to guard both bodies (on the assumption
that later tests could be written).

BTW, thanks for writing these tests for the debugger. It's unfortunate
that 7.2 is still so new (or maybe, that so many systems are so
behind) but I know what a pain it is to write tests for something
interactive like this, so kudos to you.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] unbound variables

[Robert Bradshaw]

On Fri, Jan 28, 2011 at 12:20 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2011/1/28 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Jan 27, 2011 at 10:46 PM, Vitja Makarov vitja.maka...@gmail.com 
 wrote:
 2011/1/28 Vitja Makarov vitja.maka...@gmail.com:
 2011/1/28 Stefan Behnel stefan...@behnel.de:
 Vitja Makarov, 28.01.2011 07:19:
 2011/1/27 Stefan Behnel:
 Vitja Makarov, 27.01.2011 11:19:
 https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995

 
       def analyse_types(self, env):
           if not self.label_num:
               error(self.pos, 'yield' not supported here)
 

 This error message should be replaced with assertion on self.label_num
 or internal error.

 Yes, if handled by the transform already.


 I tried to handle IfStatNode terminator here:

 https://github.com/vitek/cython/commits/uninitialized



 About tests I the easiest way is to add compiler directive -Werror

 We already have an errors are fatal option, but I like this one.


 This one means treat warnings as errors.


 And add it in cython header comment

 # cython: werror=True

 Or rather warnings_are_errors=True.

 Cython/Options.py:
     'warn': None,
     'warn.undeclared': False,

 Or better warn.errors

 But I'm wondering how to get access to
   env.directives

 inside Erros.warning


 So maybe it would be better to have -Werror command line switch?

 Directives can be enabled from the command line, so I'd rather it be a
 directive. +1 to warnings_are_errors  over the more cryptic (for those
 not familiar with gcc) werror.


 Command line options are stored in Cython.Options,
 Compiler directives in environment is it accessible from Cython.Errors?

That is a good point. No, Cython.Errors is global wheras directives
(may) have smaller scope. As it's not anything that influences the
behavior of the program itself, I'm less adamant that there be a way
to express it in the source code file.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage problem after fixing #654

[Robert Bradshaw]

On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel stefan...@behnel.de wrote:
 Stefan Behnel, 27.01.2011 09:40:
 Robert Bradshaw, 27.01.2011 07:39:
 On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote:
 there is this code in Sage (wrapper_rr.pyx/.pxd):

     cdef class Wrapper_rr(Wrapper):
         cdef int _n_args
         cdef mpfr_t* _args
         ...

     # offending line:
     mpfr_init2(self._args[i], self.domain.prec())

 The signature of mpfr_init2() is

       ctypedef __mpfr_struct* mpfr_t
       void mpfr_init2 (mpfr_t x, mp_prec_t prec)
  [...]
     sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in
 assignment

 Any idea what might be going wrong here?

 mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have
 stack-allocated, pass-by-reference semantics.) We only need to pull
 non-simple arguments into temp variables.

 Ah, that would explain it. But this also means that there may be cases
 where we can't help breaking existing code with such a change.

 Ok, this particular code really can't be worked around by Cython. From the
 POV of the compiler, the second function argument (a Python function call
 result) may potentially have side effects on the first one, so both must be
 coerced to a temp in the right order to be correct.

  2732     /* sage/ext/interpreters/wrapper_rr.pyx:56
  2733  *         if self._args == NULL: raise MemoryError
  2734  *         for i in range(count):
  2735  *             mpfr_init2(self._args[i], self.domain.prec()) # 
  2736  *         val = args['constants']
  2737  *         self._n_constants = len(val)
  2738  */

  2739     __pyx_t_8 = (((struct
 __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr
 *)__pyx_v_self)-_args[__pyx_v_i]);

  2740     __pyx_t_2 = PyObject_GetAttr(((PyObject *)((struct
 __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr
 *)__pyx_v_self)-domain), __pyx_n_s__prec); if /*...*/

  2741     __Pyx_GOTREF(__pyx_t_2);
  2742     __pyx_t_3 = PyObject_Call(__pyx_t_2, ((PyObject
 *)__pyx_empty_tuple), NULL); if /*...*/

  2743     __Pyx_GOTREF(__pyx_t_3);
  2744     __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0;
  2745     __pyx_t_9 = __Pyx_PyInt_from_py_mp_prec_t(__pyx_t_3); if /*...*/
  2746     __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0;
  2747     mpfr_init2(__pyx_t_8, __pyx_t_9);


 And given that Cython cannot know that the pointer is actually not a
 pointer, it generates the expected code here which only the C compiler can
 detect as invalid.
 This needs fixing in Sage in one way or another.

Cython didn't used to support tyepdef'd arrays, so that's why it was
declared as a pointer, but yes, that should be fixed. I'll do that in
our copy, and it will go out with the 0.14.1 fixes (even though we
don't need it yet). There's still the case of, say,

mpfr_init2(self._args[f(i)], self.domain.prec())

where f is a cdef int - int function. Do we handle that?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage problem after fixing #654

[Robert Bradshaw]

On Thu, Jan 27, 2011 at 12:19 PM, Stefan Behnel stefan...@behnel.de wrote:
 Stefan Behnel, 27.01.2011 09:40:
 Robert Bradshaw, 27.01.2011 07:39:
 On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote:
 ticket #654 describes a problem with the order in which function call
 arguments are evaluated. I fixed it and it broke Sage.

 Thanks for fixing this, I never got around to it last night. I'm
 wondering how many other similar situations may be lurking around.

 [...]this also means that there may be cases
 where we can't help breaking existing code with such a change.

 My attempts to fix this properly seem to uncover brittle code in several
 other places. Given that the particular problem that #654 describes is less
 likely to break code than my current attempts to fix it, I would suggest
 releasing 0.14.1 without my latest changes, i.e. from Mark's last commit
 fe17af96655e7ab0310a.

+1, I was actually thinking that the -devel branch is starting to
diverge to far for this release.

Lets move everything that's not happening off of the 0.14.1 milestone.
I'll take a look at the couple of test failure reports later today,
and push out another (last?) rc.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] unbound variables

[Robert Bradshaw]

On Thu, Jan 27, 2011 at 10:46 PM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2011/1/28 Vitja Makarov vitja.maka...@gmail.com:
 2011/1/28 Stefan Behnel stefan...@behnel.de:
 Vitja Makarov, 28.01.2011 07:19:
 2011/1/27 Stefan Behnel:
 Vitja Makarov, 27.01.2011 11:19:
 https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995

 
       def analyse_types(self, env):
           if not self.label_num:
               error(self.pos, 'yield' not supported here)
 

 This error message should be replaced with assertion on self.label_num
 or internal error.

 Yes, if handled by the transform already.


 I tried to handle IfStatNode terminator here:

 https://github.com/vitek/cython/commits/uninitialized



 About tests I the easiest way is to add compiler directive -Werror

 We already have an errors are fatal option, but I like this one.


 This one means treat warnings as errors.


 And add it in cython header comment

 # cython: werror=True

 Or rather warnings_are_errors=True.

 Cython/Options.py:
     'warn': None,
     'warn.undeclared': False,

 Or better warn.errors

 But I'm wondering how to get access to
   env.directives

 inside Erros.warning


 So maybe it would be better to have -Werror command line switch?

Directives can be enabled from the command line, so I'd rather it be a
directive. +1 to warnings_are_errors  over the more cryptic (for those
not familiar with gcc) werror.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] debuger test failure

[Robert Bradshaw]

On Tue, Jan 25, 2011 at 1:46 PM, Vitja Makarov vitja.maka...@gmail.com wrote:
 It's okay now. But with few warnings.

 vitja@vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp
 --no-annotate libc
 Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
 [GCC 4.4.5]

 Running tests against Cython 0.14.1rc2

 test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning:
 codefile.pyx:25:6: 'eggs' redeclared
 Function __pyx_pf_8codefile_5outer_inner not defined.
 ..
 --
 Ran 22 tests in 8.263s

I would like to get rid of these warnings if possible...

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] unbound variables

[Robert Bradshaw]

On Tue, Jan 25, 2011 at 11:49 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2011/1/25 Stefan Behnel stefan...@behnel.de:
 Vitja Makarov, 25.01.2011 10:01:
 2011/1/25 Stefan Behnel:
      def x():
          do_some_stuff()

          return # disable rest of code, e.g. during debugging

          unreachable_code()

 Cython shouldn't bother generating dead code here (even if the C compiler
 would drop it anyway).

 That should be rather easy to do: remove all the nodes in StatList
 after: break, continue, return, raise, something else?

 Careful, this may involve recursive propagation through helper nodes. The
 tree isn't always as simple as that.

 I think an attribute is_terminator on Nodes might do the job. It's set to
 False by default and to True on all nodes you mentioned above, and is
 inherited by StatListNode if its last node is a terminator (while dropping
 its remaining child nodes at the same time) and by all helper nodes that
 contain StatListNodes. This could be done in analyse_types() (or maybe
 earlier?).


 May be it could be placed in StatNodeList.analyse_flow_control()?

This is crufty code that should probably be removed, I wouldn't count
on using it too much.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage problem after fixing #654

[Robert Bradshaw]

On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel stefan...@behnel.de wrote:
 Hi,

 ticket #654 describes a problem with the order in which function call
 arguments are evaluated. I fixed it and it broke Sage.

Thanks for fixing this, I never got around to it last night. I'm
wondering how many other similar situations may be lurking around. Of
course we could store every sub-expression in temporary variables, but
that would be less than ideal.

 The problem was that in cases where some arguments are simple and others
 are not, the non-simple arguments are stuffed into a temp before hand, thus
 being evaluated before the other arguments. This can break side-effects of
 simple arguments, which include C function calls.

 My fix was to put all arguments into temps if any of them are in temps
 anyway. However, there is this code in Sage (wrapper_rr.pyx/.pxd):

   cdef class Wrapper_rr(Wrapper):
       cdef int _n_args
       cdef mpfr_t* _args
       ...

   # offending line:
   mpfr_init2(self._args[i], self.domain.prec())

 The signature of mpfr_init2() is

     ctypedef __mpfr_struct* mpfr_t
     void mpfr_init2 (mpfr_t x, mp_prec_t prec)

 Cython generates this code:

     mpfr_t __pyx_t_7;
     ...
     __pyx_t_7 = (((struct ...)__pyx_v_self)-_args[__pyx_v_i]);
     ...
     mpfr_init2(__pyx_t_7, __pyx_t_8);

 Looks reasonable at first sight. However, gcc complains about the temp
 assignment:

   sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in
 assignment

 Any idea what might be going wrong here?

mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have
stack-allocated, pass-by-reference semantics.) We only need to pull
non-simple arguments into temp variables.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage problem after fixing #654

[Robert Bradshaw]

On Wed, Jan 26, 2011 at 10:35 PM, Stefan Behnel stefan...@behnel.de wrote:
 Stefan Behnel, 27.01.2011 07:29:
 ticket #654 describes a problem with the order in which function call
 arguments are evaluated. I fixed it and it broke Sage.

 The problem was that in cases where some arguments are simple and others
 are not, the non-simple arguments are stuffed into a temp before hand, thus
 being evaluated before the other arguments. This can break side-effects of
 simple arguments, which include C function calls.

 My fix was to put all arguments into temps if any of them are in temps
 anyway.

 BTW, I wonder if a better fix would be to declare C function calls
 non-simple in general.

If C function calls are simple, that's a bug. Simple expressions
should be side-effect free by definition.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] unbound variables

[Robert Bradshaw]

On Mon, Jan 24, 2011 at 11:33 PM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 25.01.2011 08:15:
 I want to write simple code to find unbound variables.

I'm assuming you mean unassigned (as opposed to unbound in, e.g., a closure)?

 There is some code that does (part of) that already, in a transform IIRC.
 It's used to detect if we need to initialise a local variable to None or
 can safely set it to NULL.


 As it's hard to detect them in common case it will mark them as:
   - bound (defenitley bound)
 def foo():
      a = 1
      print a

 Bound variables should be initialized to NULL

   - unbound (defenitley unbound)
 def foo():
      print a
      a = 1

 Unbound variables should be initialized to Py_None.
 And user may be warned (should depend on command line flags)

   - don't know (not sure here)
 def foo(x):
      if x:
          a = 1
      [else: # optional
          a = 2]
      print a

 Algo is too stupid it don't know much about branches, so it's not sure here.

 Well, it *can't* know what will happen at runtime. That's not being stupid
 at all, it's just the correct way to do it.

If the else clause is there, it should be smart enough. Loops are a
bit trickier.

 This ones will be initialized to Py_None.

 To be correct, they'd have to get initialised to NULL and a check needs to
 get generated on read access as long as we don't know for sure if it has
 been initialised or not. CPython raises an exception on unbound locals, so
 Cython should do the same and should do it efficiently. When we get to the
 point that we safely know which variables are being initialised and for
 which only the runtime behaviour can tell us if they are or not, I think
 it's fine to add the little performance penalty of checking for NULL in
 exactly the unsafe cases.

Note also that assignment, and possibly cleanup, needs to be made
NULL-safe as well.

 Also I would like to check for unused variables

We already do this (to avoid compiler warnings).

 and unreachable code
 (this should be removed).

 Some unreachable code is getting dropped during constant folding, usually
 stuff like if False: ..., but I agree that there's always more that can
 be done. Think of this:

     def x():
         do_some_stuff()

         return # disable rest of code, e.g. during debugging

         unreachable_code()

 Cython shouldn't bother generating dead code here (even if the C compiler
 would drop it anyway).

Personally, I think it's better to focus on stuff that the C compiler
can't do for us, but if it doesn't complicate things too much than
it's nice to have.

 Knowing where a function returns and if it has trailing code with a default
 None return would also be good to know. It would be a step towards
 inferring the return type of functions automatically.

Yep, that would be nice. Currently we don't have much (anything?) in
the way of control flow analysis.

It probably wouldn't be too hard to walk the tree to discover this
kind of information, recording on each NameNode as one goes along what
its possible states are.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] unbound variables

[Robert Bradshaw]

On Tue, Jan 25, 2011 at 1:23 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 25.01.2011 10:00:
 On Mon, Jan 24, 2011 at 11:33 PM, Stefan Behnel wrote:
 Vitja Makarov, 25.01.2011 08:15:
 I want to write simple code to find unbound variables.

 I'm assuming you mean unassigned (as opposed to unbound in, e.g., a closure)?
[...]
 It probably wouldn't be too hard to walk the tree to discover this
 kind of information, recording on each NameNode as one goes along what
 its possible states are.

 A general NameNode dependency walk of the tree could give us *loads* of
 information, also for better type inference. Knowing that a Python variable
 is definitely not None from a given point on, or that it has a specific
 type at one point and a different type at another would be really cool.

 Basically, we could build a dependency tree for each NameNode (at its
 specific point in the tree, not just through its symtab Entry) that
 references its assignment RHSs, but including a representation of relevant
 branches in the code.

Yes, that's exactly what I was thinking. Each NameNode would have an
associated status (which could contain arbitrary information and have
merge methods) and we would walk the tree updating the status as we
went, merging on the branches.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython workshop

[Robert Bradshaw]

On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel stefan...@behnel.de wrote:
 Dag Sverre Seljebotn, 20.01.2011 12:26:
 Starting new thread in case somebody did not see the hijacking.

 I've created this wiki page:

 http://wiki.cython.org/workshop1

 If you're interested in coming to a Cython workshop, please fill in your
 details in the table to help decide when and where a workshop should be
 held.

 Current status so far: 4 core developers have signed in, 5 people in total,
 all from Europe. Robert said he'd like to join in, too.

 Have there been any further off-list replies so far?

I've spoken to two people, both of whom won't be able to do much the
first half of this year.

 In case we actually want to advertise and run this before the end of April
 (or even March, and Vitja's visa will likely take a while to get), it would
 be good if the remaining core developers and other interested parties could
 speak up soon, say, within a week's time, so that we can decide on date and
 location.

The first half of March doesn't work for me, and April would be
tricky, but I might be able to do March 30-April 3. More ideal would
be May-June-July, but that conflicts directly with Stefans
availability--are there any weekends in there that would work or are
those months right out?

I think 4 days is about the right amount of time.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

On Tue, Jan 18, 2011 at 12:56 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 A release candidate is up at
 http://cython.org/release/Cython-0.14.1rc1.tar.gz . Hopefully there
 shouldn't be many issues, there's just a pile of bugfixes on top of
 0.14. I also started http://wiki.cython.org/ReleaseNotes-0.14.1

Another candidate up at http://cython.org/release/Cython-0.14.1rc2.tar.gz

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython workshop

[Robert Bradshaw]

On Thu, Jan 20, 2011 at 7:09 AM, Dag Sverre Seljebotn
da...@student.matnat.uio.no wrote:
 On 01/20/2011 03:48 PM, Stefan Behnel wrote:

 Dag Sverre Seljebotn, 20.01.2011 15:36:

 On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote:

 On 01/20/2011 03:26 PM, Stefan Behnel wrote:

 Dag Sverre Seljebotn, 20.01.2011 12:26:

 I've created this wiki page:

 http://wiki.cython.org/workshop1

 I restricted access to that page to authenticated users as we are
 collecting personal information there.

Thanks for setting this up.

 I think that is counterproductive, because we want to develop that page
 into the poster for the workshop that anyone can visit to see if
 they're interested in coming.

 Sounds like a different page to me.

 For this workshop there's a very gray area between announcing and planning.
 E.g., here's an email I just sent to numpy-discussion.

+1. There will be more formal planning and announcement once we get a
sense for people's interest and who may be able to come.

 Anyone who's
 interested enough to read it can always register and it is just a minor
 annoyance, so I guess I'm fine with it.

 Like Mark I don't see what the lots of information is supposed to be, and
 the thought didn't strike me at all until you mentioned it. Then again,
 academics want their name and location spread around as much as possible
 anyway :-)

That's how I feel too. In any case, as long as no one posts other
people's information, there's no privacy invasion issue. People should
feel free to put contact me in those boxes if they're sensitive
about that kind of stuff.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

A release candidate is up at
http://cython.org/release/Cython-0.14.1rc1.tar.gz . Hopefully there
shouldn't be many issues, there's just a pile of bugfixes on top of
0.14. I also started http://wiki.cython.org/ReleaseNotes-0.14.1

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

On Tue, Jan 18, 2011 at 10:07 AM, mark florisson
markflorisso...@gmail.com wrote:
 On an unrelated side note, should we modify the testrunner to continue when
 importing a test module fails and have it print a traceback instead?

Yes, that would make the most sense.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1 release candidate

[Robert Bradshaw]

On Tue, Jan 18, 2011 at 10:09 AM, Arfrever Frehtes Taifersar Arahesis
arfrever@gmail.com wrote:
 2011-01-18 18:54:03 mark florisson napisał(a):
 Yes, you need to run those tests with a python build for which debug
 symbols are available.

 Should I turn this into a warning instead of an error?

 This test should be automatically skipped when debug symbols are unavailable.

+1

Mark, you should be able to push now.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] [OT] Python like lanugages [was Re: After C++, what with Python?]

[Robert Bradshaw]

On Tue, Jan 18, 2011 at 1:46 AM, William Stein wst...@gmail.com wrote:
 On Tue, Jan 18, 2011 at 1:13 AM, Stefan Behnel stefan...@behnel.de wrote:
 Terry Reedy, 17.01.2011 22:38:
 A few other details confuse me, but enough for now.

 It's helpful to be reminded from time to time that the documentation shows
 the usual features of an underfinanced OpenSource project. Part of it was
 inherited from the original Pyrex documentation and suffers from bit rot,
 and some other parts were copied over from the Wiki or from talk notes and
 continue to be badly integrated with the rest.

 I recently got some new nontrivial NSF funding for a Cython workshop.
  A docathon would be a great activity at such a workshop.

 When some Cython developers are ready to all get flown somewhere cool
 to work together on Cython for a few days, all expenses paid, let me
 know.  I'm serious.

+1, docs would be a good topic. That and, once generators are in, a
push for 1.0.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython workshop

[Robert Bradshaw]

On Tue, Jan 18, 2011 at 2:31 AM, Stefan Behnel stefan...@behnel.de wrote:
 William Stein, 18.01.2011 10:46:
 I recently got some new nontrivial NSF funding for a Cython workshop.
    A docathon would be a great activity at such a workshop.

 When some Cython developers are ready to all get flown somewhere cool
 to work together on Cython for a few days, all expenses paid, let me
 know.  I'm serious.

 Hmm, in case the funding allows non-US territory - anyone interested in
 coming to Munich? Maybe not the coolest place in the world but not too bad
 either. And the (tiny) local PUG is based in the local University, so I
 should be able to get some infrastructure up for a weekend and likely a
 couple of days around that. We could advertise a couple of public talks to
 increase their interest.

Certainly organizing it where someone is local simplifies the
logistics, thanks for volunteering.

 Bonus: last I heard, it's a lot easier to get into Germany than into the US
 part of North America. :)

Most certainly true for an EU citizen such as yourself, and several
other countries. My last trip through German customs
was...bearable--I'd be more than willing to do it again to maximize
participation, and Munich looks like a nice place. The other question
is one of expense--how many people would we have to ship that
direction vs. this.

Anyone interested in coming, please reply (on or off list) with your
preferences and what you think you could contribute.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Stable ABI mode?

[Robert Bradshaw]

On Sun, Jan 16, 2011 at 2:07 AM, Stefan Behnel stefan...@behnel.de wrote:
 Hi,

 Python 3.2 comes with a stable ABI for extension modules.

 http://www.python.org/dev/peps/pep-0384/

I hadn't seen that, interesting. I can see how this would be useful
for Windows users.

 It would be nice if Cython provided an option to restrict the C-API usage
 to what the ABI considers fixed and stable. That would disable excesive
 optimisations like list.pop(), but also disallow several other assumptions
 that we use all over the place in the generated C code.

 I'd figure it would be a somewhat involved change, though...

It would be nice to have the option, but I'm not sure how restrictive
it would be. For example, would we even be able to construct
tracebacks for exception propagation? It's probably worth making an
attempt though to see how hard it would be (and, if necessary, perhaps
we could push to extend the ABI).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] string literal parsing problem

[Robert Bradshaw]

On Sat, Jan 15, 2011 at 10:55 AM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 15.01.2011 19:29:
 2011/1/15 Stefan Behnel:
 Stefan Behnel, 15.01.2011 19:13:
 Vitja Makarov, 15.01.2011 18:46:
 What will it print for '\u1234'?

 Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
 [GCC 4.4.5] on linux2
 Type help, copyright, credits or license for more information.
 '\u'
 '\\u'
 '\u1234'
 '\\u1234'


 I think that '\u' should be translated into '\\u' for python2

 That's what it does, yes. This works because we actually parse unprefixed
 strings in parallel as byte strings and unicode strings.

 However, now that I tried it, I actually get the same result in Py3,
 although it should have parsed the string correctly. Not sure if we
 discussed this problem before, but it looks like a bug to me.

 Thinking about this some more, it's inconsistent either way.

 1) If the literal string semantics should be fixed at compile time, you
 shouldn't get a unicode string in Python 3 in the first place.

 2) If the literal should become a byte string in Py2 and a unicode string
 in Py3, then the unicode string should be what you you'd get if you ran
 your code in Py3, i.e. the unescaped unicode literal.

 Given that 1) is out of discussion, 2) should be fixed, IMHO.

 Can't we rely on -[23] cython switch?

 In -2 mode strings are always byte string and -3 always unicode?

 With -3, unprefixed strings *are* unicode strings.

 As for -2, there isn't currently any change in behaviour when you use that
 switch, and I feel reluctant to change that. For one, I doubt that anyone
 would seriously use it. The problem that unicode escapes in unprefixed
 strings behave differently in Python 2 and Python 3 is unlikely to create
 problems in real world code, i.e. outside of CPython's regression test suite.


          self.assertEqual(audioop.lin2alaw(data[0], 1), '\xd5\xc5\xf5')

 That's a different problem. You will notice that this code has been fixed
 to use the 'b' prefix in the Py3 test suite.

 This is a problem that cannot be solved automatically. For Python 2 code,
 the compiler cannot know if the user intended an unprefixed literal to be a
 (binary) byte string or a unicode (text) string. Only a human brain can
 disambiguate the code here. Remember that Python 2 will also try to decode
 the above binary bytes literal if it happens to be concatenated with a
 unicode string for some reason. String handling is structurally hard to get
 right in Python 2, we have to live with that (and hope that Py2 will die
 out soon).

I wouldn't count on it.

 I think it's a great feature of Cython that it fails fast and thus tells
 you that your code is ambiguous and requires changes to work in Python 3.
 It perfectly found the problems in the above code, for one.

Whether it's the -2 flag, or something else, we should at least have a
mode that handles things exactly as they would be handled in Python 2.
Otherwise people won't be able to just compile their existing code
without worrying about subtle issues like this. Of course, a compile
time error is much more acceptable than different runtime behavior.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] string literal parsing problem

[Robert Bradshaw]

On Sat, Jan 15, 2011 at 1:02 PM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 15.01.2011 21:26:
 Whether it's the -2 flag, or something else, we should at least have a
 mode that handles things exactly as they would be handled in Python 2.
 Otherwise people won't be able to just compile their existing code
 without worrying about subtle issues like this.

 Hmm. I wouldn't mind having a mode that compiles Python 2 code and fails
 fast on compilation under Python 3 with a C #error - if someone has
 enough interest in this to implement it. I certainly don't.

That's an interesting idea.

 While I agree
 that these issues are subtle, they certainly aren't common enough to really
 worry about them. Broken code is best worked around by fixing the code.
 Even the Python 2.x line hasn't always bowed to the holy cow of backwards
 compatibility.

 Anyway, given that -3 doesn't prevent code from working in Python 2, the
 -2 flag doesn't seem like a good match. It should be something clear as
 in --python2-only.

I think the primary motivation of the -2 flag is so that, eventually,
we can make -3 the default without providing a recourse for people who
don't want to change their code right away.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1?

[Robert Bradshaw]

On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel stefan...@behnel.de wrote:
 mark florisson, 13.01.2011 12:05:
 In any case, there are virtually no changes when the --gdb
 flag is inactive. These last additions would be very welcome [...]

 Ok, here's a general statement. I personally consider the debugging support
 new, experimental and not a critical feature for compiler or language. You
 are the current maintainer of that part anyway, so I'm fine with merging
 the changes for 0.14.1, even unseen, as long as it doesn't break anything
 that's more important.

I'm actually looking at that code right now.

 And, no, that's not a green card for back doors. ;-)

:)

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1?

[Robert Bradshaw]

On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel stefan...@behnel.de wrote:
 mark florisson, 13.01.2011 12:05:
 In any case, there are virtually no changes when the --gdb
 flag is inactive. These last additions would be very welcome [...]

 Ok, here's a general statement. I personally consider the debugging support
 new, experimental and not a critical feature for compiler or language. You
 are the current maintainer of that part anyway, so I'm fine with merging
 the changes for 0.14.1, even unseen, as long as it doesn't break anything
 that's more important.

 I'm actually looking at that code right now.

I've merged this. Note that this brings up
http://trac.cython.org/cython_trac/ticket/645 . I'd say we're ready
for an rc.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1?

[Robert Bradshaw]

On Fri, Jan 14, 2011 at 3:08 AM, mark florisson
markflorisso...@gmail.com wrote:


 On 14 January 2011 12:06, Stefan Behnel stefan...@behnel.de wrote:

 Robert Bradshaw, 14.01.2011 10:05:
  On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote:
  On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote:
  mark florisson, 13.01.2011 12:05:
  In any case, there are virtually no changes when the --gdb
  flag is inactive. These last additions would be very welcome [...]
 
  Ok, here's a general statement. I personally consider the debugging
  support
  new, experimental and not a critical feature for compiler or language.
  You
  are the current maintainer of that part anyway, so I'm fine with
  merging
  the changes for 0.14.1, even unseen, as long as it doesn't break
  anything
  that's more important.
 
  I'm actually looking at that code right now.
 
  I've merged this.

 The build complains because it cannot find a file called

 Cython/Debugger/do_repeat.pyx

 I don't have it, how does it complain?

It was added to line 103 from setup.py, but not to the repo?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1?

[Robert Bradshaw]

On Fri, Jan 14, 2011 at 3:11 AM, mark florisson
markflorisso...@gmail.com wrote:


 On 14 January 2011 12:08, mark florisson markflorisso...@gmail.com wrote:


 On 14 January 2011 12:06, Stefan Behnel stefan...@behnel.de wrote:

 Robert Bradshaw, 14.01.2011 10:05:
  On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote:
  On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote:
  mark florisson, 13.01.2011 12:05:
  In any case, there are virtually no changes when the --gdb
  flag is inactive. These last additions would be very welcome [...]
 
  Ok, here's a general statement. I personally consider the debugging
  support
  new, experimental and not a critical feature for compiler or
  language. You
  are the current maintainer of that part anyway, so I'm fine with
  merging
  the changes for 0.14.1, even unseen, as long as it doesn't break
  anything
  that's more important.
 
  I'm actually looking at that code right now.
 
  I've merged this.

 The build complains because it cannot find a file called

 Cython/Debugger/do_repeat.pyx

 I don't have it, how does it complain?


 Is that a missing commit?

 Stefan
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev

 Apparently something left that should have been removed, apologies. You can
 remove it from the setup.py in the list on line 104.

Done.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1?

[Robert Bradshaw]

On Fri, Jan 14, 2011 at 3:27 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 14.01.2011 12:12:
 On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote:
 On 14 January 2011 12:08, mark florisson wrote:
 On 14 January 2011 12:06, Stefan Behnel wrote:
 The build complains because it cannot find a file called

 Cython/Debugger/do_repeat.pyx

 Apparently something left that should have been removed, apologies. You can
 remove it from the setup.py in the list on line 104.

 Done.

 Ok, that looks better.

 Now that the debugger support is an official feature, should we have a
 Hudson job that builds Cython and all tests with gdb support enabled?

Yes, that would make sense. We'd have to build our own newer gdb of
course, and a debug version of Python.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1?

[Robert Bradshaw]

On Fri, Jan 14, 2011 at 3:45 AM, mark florisson
markflorisso...@gmail.com wrote:


 On 14 January 2011 12:31, Robert Bradshaw rober...@math.washington.edu
 wrote:

 On Fri, Jan 14, 2011 at 3:27 AM, Stefan Behnel stefan...@behnel.de
 wrote:
  Robert Bradshaw, 14.01.2011 12:12:
  On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote:
  On 14 January 2011 12:08, mark florisson wrote:
  On 14 January 2011 12:06, Stefan Behnel wrote:
  The build complains because it cannot find a file called
 
  Cython/Debugger/do_repeat.pyx
 
  Apparently something left that should have been removed, apologies.
  You can
  remove it from the setup.py in the list on line 104.
 
  Done.
 
  Ok, that looks better.
 
  Now that the debugger support is an official feature, should we have a
  Hudson job that builds Cython and all tests with gdb support enabled?

 Yes, that would make sense. We'd have to build our own newer gdb of
 course, and a debug version of Python.

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev

 That would be great. In that case, is it possible that my branch will make
 it to Hudson? Otherwise Hudson will only be useful after merging into
 mainline, which is probably just a few hours or days before the release. In
 any case it would also help prevent for instance the issue we had now where
 my branch did introduce a problem just before the release (in the future I
 will run the full test suite instead of skipping some of them with those
 options to runtests.py). If that's a hassle I wouldn't mind mainline branch
 access either, I promise I'll be good :)

I've read enough of your code that I think that'd be fine. I think it
makes sense to have a separate branch for the debugger stuff so things
could be caught before merging.

 The important thing is Python with debug symbols, it does not necessarily
 have to be a debug build. The thing is that at least many Linux distros ship
 debug symbols with the debug build (usually Python is compiled with -g, but
 then stripped). This is for instance the case in Ubuntu (and probably
 Debian). However, in Fedora for instance even the debug build is shipped
 without symbols, and they need to be installed separately.

We build our own Pythons, so could include debug symbols in them.
Other than that, is the only requirement to have a working gdb = 7.2
in the path somewhere?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] raw string problem

[Robert Bradshaw]

On Wed, Jan 12, 2011 at 10:29 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 Hi!

 It seems that cython parses raw strings as usual strings:

 Try this:
 print r'\'

This is certainly a bug. http://trac.cython.org/cython_trac/ticket/641

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] raw string problem

[Robert Bradshaw]

On Wed, Jan 12, 2011 at 11:03 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2011/1/12 Robert Bradshaw rober...@math.washington.edu:
 On Wed, Jan 12, 2011 at 10:29 AM, Vitja Makarov vitja.maka...@gmail.com 
 wrote:
 Hi!

 It seems that cython parses raw strings as usual strings:

 Try this:
 print r'\'

 This is certainly a bug. http://trac.cython.org/cython_trac/ticket/641


 This should be easy to fix...
 Why do you think that this could be a major change?
 Does some code use this bug?

It is very possible that there is code out there using this bug, and
if so it would be a silent, backwards incompatible change.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] raw string problem

[Robert Bradshaw]

On Wed, Jan 12, 2011 at 12:51 PM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 12.01.2011 19:29:
 It seems that cython parses raw strings as usual strings:

 Try this:
 print r'\'

 That's slightly exaggerated. The only broken cases are '\' and '\'',
 everything else works AFAICT.

That is comforting, and given that it's a tiny corner case I think
it's worth and safe fixing in 0.14.1.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Closure scope object initialization

[Robert Bradshaw]

On Mon, Jan 10, 2011 at 3:28 AM, mark florisson
markflorisso...@gmail.com wrote:
 Hello,
 There is a slight problem with the scope object in closures because it is
 not initialized to NULL in closure frames, it is instead initialized later
 by assigning a function argument to it. For the debugger this is a problem
 when execution of the debuggee is halted before this assignment happens,
 because there is no way to tell whether closed-over variables are valid or
 not. For e.g. the listing of local variables this is not a big issue, as
 this is entirely safe. But for code execution this means the debugger will
 have the debuggee try to put an invalid pointer in a dict in which the code
 will be executed, with a near 100% chance of segfaulting the debuggee.
 So the question is, could we simply initialize the scope objects to NULL?

I think that'd be a fine thing to do. Are there any code paths that
don't initialize this before its actually used (in which case the
initial assignment may get optimized away)?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Closure scope object initialization

[Robert Bradshaw]

On Mon, Jan 10, 2011 at 11:05 AM, mark florisson
markflorisso...@gmail.com wrote:


 On 10 January 2011 19:31, Robert Bradshaw rober...@math.washington.edu
 wrote:

 On Mon, Jan 10, 2011 at 3:28 AM, mark florisson
 markflorisso...@gmail.com wrote:
  Hello,
  There is a slight problem with the scope object in closures because it
  is
  not initialized to NULL in closure frames, it is instead initialized
  later
  by assigning a function argument to it. For the debugger this is a
  problem
  when execution of the debuggee is halted before this assignment happens,
  because there is no way to tell whether closed-over variables are valid
  or
  not. For e.g. the listing of local variables this is not a big issue, as
  this is entirely safe. But for code execution this means the debugger
  will
  have the debuggee try to put an invalid pointer in a dict in which the
  code
  will be executed, with a near 100% chance of segfaulting the debuggee.
  So the question is, could we simply initialize the scope objects to
  NULL?

 I think that'd be a fine thing to do. Are there any code paths that
 don't initialize this before its actually used (in which case the
 initial assignment may get optimized away)?


 No the scope object is always initialized (it has to be as it's only present
 when inner scopes access free variables from outer scopes, or when the outer
 scope needs to store user-variables in scope objects), but the problem is
 that execution may be halted by the debugger before it is initialized. When
 automatic variables are set to NULL and the debugger breaks on the function,
 accesses to such variables will return NULL even before the actual
 assignment and declaration happen, instead of something random. In any
 case, I think automatic variables that are initialized to NULL but not used
 may get optimized anyway.
 So then we come to the next point, who wants to implement this? :) I had a
 brief initial attempt by assigning 'NULL' to the 'init' attribute of the
 scope AST entry, which was unfruitful. I could take a deeper look into this
 issue, if I should, please tell me (and please don't hesitate to give some
 pointers :).

Since you're dealing with the state of objects before argument
parsing, it may be more fruitful to initialize it directly as part of
the header generation code than via the AST (whose first statement
assumes the argument parsing has been atomically done previously).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Fri, Jan 7, 2011 at 4:27 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2010/12/31 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Dec 30, 2010 at 10:05 AM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 30.12.2010 18:53:
 Why did you reverted decorators stuff?

 https://github.com/vitek/cython/commit/e0d366d9409680849e6f429992ac9724e2ad1016

 Because I didn't get it finished, it broke the Sage build and cython-devel
 should stay cleanly working before the release.

 +1

 I added a link to the build
 failure log to the trac ticket but didn't even manage to come up with a
 suitable test case before I had to stop working on it.

 What I implemented so far works in most cases but there are issues with
 functions at the module scope. I didn't check the history but I think I
 recall from the code comments that Robert worked in this area (method
 binding) a while ago, and the comments hint at an unfinished refactoring
 that would help with this change.

 I'm not working on anything in the area at the moment though (or,
 contrary to my expectations, much of anything Cython-wise... having
 too much fun hanging out with the family :).

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


 I've added GenerartorDefNode and GeneratorBodyDefNode now all the
 generators stuff is handled there.

Yay! I'm pretty busy this weekend, but have been following your branch
from a distance and so reviewing this shouldn't take too long. I'm
thinking we should get a 0.14.1 bug fix out ASAP (e.g. once those
Windows and type inference bugs are in) and then 0.15 will follow with
generators.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 Bugreport

[Robert Bradshaw]

On Thu, Jan 6, 2011 at 10:54 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 On Thu, Jan 6, 2011 at 10:48 AM, Vitja Makarov vitja.maka...@gmail.com 
 wrote:
 2011/1/6 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel stefan...@behnel.de wrote:
 Daniel Norberg, 06.01.2011 12:20:
 I'm getting a curious error in Cython 0.14 when trying to compile this:

    def bar(foo):
        qux = foo
        quux = foo[qux.baz]

 The error message:

       $ cython bar.py

       Error compiling Cython file:
       
       ...
       def bar(foo):
               qux = foo
               quux = foo[qux.baz]
                     ^
       

       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type 
 'unspecified' has no attribute 'baz'

 Cython 0.13 compiles this just fine. I also tried the latest revision of 
 cython-devel (b816b03ff502) and it fails.

 I can reproduce this. From a quick test, it seems like the type inference
 machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before
 'qux'. Anyone interested in taking a closer look?

 That shouldn't be happening, as it should know the inferred type of
 quux depends on the inferred type of qux. I'll take a look.

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


 Strange thing this code compiles:

 def bar(foo):
    qux = foo
    xquux = foo[qux.baz]

 but this doesn't:

 def bar(foo):
    qux = foo
    aquux = foo[qux.baz]

 Since it doesn't detect the dependency, half the time it'll get lucky
 (probably as a function of the variable names).

The problem was that the indexing operator inference was changed to
depend on the index type as well as the base type, but its
type_dependencies method wasn't updated to reflect this. Thanks for
catching that, fix at
https://github.com/cython/cython/commit/43a3dbea06e00bf6eb6592017964b79e35bae2d6

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 Bugreport

[Robert Bradshaw]

On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel stefan...@behnel.de wrote:
 Daniel Norberg, 06.01.2011 12:20:
 I'm getting a curious error in Cython 0.14 when trying to compile this:

    def bar(foo):
        qux = foo
        quux = foo[qux.baz]

 The error message:

       $ cython bar.py

       Error compiling Cython file:
       
       ...
       def bar(foo):
               qux = foo
               quux = foo[qux.baz]
                     ^
       

       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type 
 'unspecified' has no attribute 'baz'

 Cython 0.13 compiles this just fine. I also tried the latest revision of 
 cython-devel (b816b03ff502) and it fails.

 I can reproduce this. From a quick test, it seems like the type inference
 machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before
 'qux'. Anyone interested in taking a closer look?

That shouldn't be happening, as it should know the inferred type of
quux depends on the inferred type of qux. I'll take a look.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 Bugreport

[Robert Bradshaw]

On Thu, Jan 6, 2011 at 10:48 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2011/1/6 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel stefan...@behnel.de wrote:
 Daniel Norberg, 06.01.2011 12:20:
 I'm getting a curious error in Cython 0.14 when trying to compile this:

    def bar(foo):
        qux = foo
        quux = foo[qux.baz]

 The error message:

       $ cython bar.py

       Error compiling Cython file:
       
       ...
       def bar(foo):
               qux = foo
               quux = foo[qux.baz]
                     ^
       

       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type 
 'unspecified' has no attribute 'baz'

 Cython 0.13 compiles this just fine. I also tried the latest revision of 
 cython-devel (b816b03ff502) and it fails.

 I can reproduce this. From a quick test, it seems like the type inference
 machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before
 'qux'. Anyone interested in taking a closer look?

 That shouldn't be happening, as it should know the inferred type of
 quux depends on the inferred type of qux. I'll take a look.

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


 Strange thing this code compiles:

 def bar(foo):
    qux = foo
    xquux = foo[qux.baz]

 but this doesn't:

 def bar(foo):
    qux = foo
    aquux = foo[qux.baz]

Since it doesn't detect the dependency, half the time it'll get lucky
(probably as a function of the variable names).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Fixing #602 - type inference for byte string literals

[Robert Bradshaw]

On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin dalc...@gmail.com wrote:
 On 3 January 2011 04:41, Stefan Behnel stefan...@behnel.de wrote:
 Hi,

 I've been working on a fix for ticket #602, negative indexing for inferred
 char*.

 http://trac.cython.org/cython_trac/ticket/602

 Currently, when you write this:

     s = b'abc'

 s is inferred as char*. This has several drawbacks. For one, we loose the
 length information, so len(s) becomes O(n) instead of O(1). Negative
 indexing fails completely because it will use pointer arithmetic, thus
 leaving the allocated memory area of the string. Also, code like the
 following is extremely inefficient because it requires multiple conversions
 from a char* of unknown length to a Python bytes object:

     s = b'abc'
     a = s1 + s
     b = s2 + s

 I came to the conclusion that the right fix is to stop letting byte string
 literals start off as char*. This immediately fixes these issues and
 improves Python compatibility while still allowing automatic coercion, but
 it also comes with its own drawbacks.

 In nogil blocks, you will have to explicitly declare a variable as char*
 when assigning a byte string literal to it, otherwise you'd get a compile
 time error for a Python object assignment. I think this is a minor issue as
 most users would declare their variables anyway when using nogil blocks.
 Given that there isn't much you can do with a Python string inside of a
 nogil block, we could also honour nogil blocks during type inference and
 automatically infer char* for literals here. I don't think it would hurt
 anyone to do that.

 The second drawback is that it impacts type inference for char loops.
 Previously, you could write

     s = b'abc'
     for c in s:
         print c

 and Cython would infer 'char' for c and print integer byte values. When s
 is inferred as 'bytes', c will be inferred as 'Python object' because
 Python 2 returns 1-byte strings and Python 3 returns integers on iteration.
 Thus the loop will run entirely in Python code and return different things
 in Py2 and Py3.

 I do not expect that this is a major issue either. Iteration over literals
 should be rare, after all, and if the byte string is constructed in any
 way, the type either becomes a bytes object through Python operations (like
 concatenation) or is explicitly provided, e.g. as a return type of a
 function call. But it is a clear behavioural change for the type inference
 in an area where Cython's (and Python's) semantics are tricky anyway.

 Personally, I think that the advantages outweigh the disadvantages here.
 Most common use cases won't notice the change because coercion will not be
 impacted, and most existing code (IMHO) either uses explicit typing or
 expects a Python bytes object anyway. So my preferred change would be to
 make byte string literals 'bytes' by default, except in nogil blocks.

 Opinions?


 +1


+1 I might say it should even be required in nogil blocks for consistency.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Thu, Dec 30, 2010 at 10:05 AM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 30.12.2010 18:53:
 Why did you reverted decorators stuff?

 https://github.com/vitek/cython/commit/e0d366d9409680849e6f429992ac9724e2ad1016

 Because I didn't get it finished, it broke the Sage build and cython-devel
 should stay cleanly working before the release.

+1

 I added a link to the build
 failure log to the trac ticket but didn't even manage to come up with a
 suitable test case before I had to stop working on it.

 What I implemented so far works in most cases but there are issues with
 functions at the module scope. I didn't check the history but I think I
 recall from the code comments that Robert worked in this area (method
 binding) a while ago, and the comments hint at an unfinished refactoring
 that would help with this change.

I'm not working on anything in the area at the moment though (or,
contrary to my expectations, much of anything Cython-wise... having
too much fun hanging out with the family :).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Hg to git

[Robert Bradshaw]

On Mon, Dec 27, 2010 at 2:31 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 21.12.2010 18:32:
 On Tue, Dec 21, 2010 at 3:10 AM, Stefan Behnelstefan...@behnel.de  wrote:
 Robert Bradshaw, 21.12.2010 07:12:
 Anyone object to making http://hg.cython.org/ read-only, and making
 https://github.com/cython/cython/ (and its forks) the live devel
 branch?

 I think it's a good time to do this now that 0.14 is out.

 Yep. Done.

 Hudson also builds from the git repo now.

Excellent, thanks. Ironically, I was just about to do this myself this morning.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14.1

[Robert Bradshaw]

On Mon, Dec 27, 2010 at 12:07 AM, Stefan Behnel stefan...@behnel.de wrote:
 Arfrever Frehtes Taifersar Arahesis, 26.12.2010 12:58:
 2010-12-26 09:46:48 Stefan Behnel napisał(a):
 regarding the next release, I think we should fix the ref-counting bug that
 Martijn Meijers found (ticket #633) and then release 0.14.1 ASAP from the
 current git tip, which already has a couple of other bug fixes.

My thoughts exactly.

 It would be nice if compatibility with Numpy 1.5 was fixed (ticket #630).

 IMHO, that's not the same level of criticality as a serious crash bug, so
 unless someone provides a ready-to-review-and-merge patch for #630, I'd
 prefer getting 0.14.1 out without waiting for a fix.

I agree.

There's a fair number of fixes in the tree already, so I was actually
thinking about doing a 0.14.1 release soon anyways. Of course
something big like this necessitates a quicker release, but how about
we see what fixes we can get in this next release and release first
thing in January?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Broken .pyxdep files

[Robert Bradshaw]

On Wed, Dec 22, 2010 at 12:48 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 On Tue, Dec 21, 2010 at 11:49 PM, Alexey Borzenkov sna...@gmail.com wrote:
 On Wed, Dec 22, 2010 at 1:19 AM, Alexey Borzenkov sna...@gmail.com wrote:
 I've recently started using Cython in one of my projects because I
 needed pyximport and automatic building of extensions and found that
 adding .pyxdep files makes Cython raise NameError because _test_files
 is not defined.
 [...]
 Also, there's another problem with pyximport that makes it unsafe to
 use it without build_in_temp=False. For example, if I have
 package1/module.pyx and package2/module.pyx then only one of those
 will build/import correctly,

 Btw, I just realized that Cython is finally using git and github, so
 if anyone is interested here are my patches:

 https://github.com/snaury/cython/commit/b43f68280919e98767bd4d163cb1eaf88c4c857e
 https://github.com/snaury/cython/commit/6f73b4280958f5d3f5fa77c3a4ed19b277784140

 Thanks. pyximport is really cool, but no one's been maintaining it,
 and it looks like we haven't been testing it either. (Code is either
 new, tested, or broken...) At first glance the patches look fine, I'll
 try to take a deeper look over the holidays (if no one beats me to
 it...)

Merged. Thanks!

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Hg to git

[Robert Bradshaw]

On Thu, Dec 23, 2010 at 5:16 AM, Arfrever Frehtes Taifersar Arahesis
arfrever@gmail.com wrote:
 2010-12-21 07:12:40 Robert Bradshaw napisał(a):
 Anyone object to making http://hg.cython.org/ read-only, and making
 https://github.com/cython/cython/ (and its forks) the live devel
 branch?

 http://hg.cython.org/cython-devel/rev/b816b03ff502 contains 
 http://hg.cython.org/cython-devel/raw-rev/b816b03ff502 URL,
 which allows to easily download a patch. Does 
 https://github.com/cython/cython/commit/e3c9a78686b7a7f0d36da8e6189b1e1fb4037c73
 provide similar functionality?

Apparently it works like this:

http://support.github.com/discussions/feature-requests/429-download-patch-files

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Broken .pyxdep files

[Robert Bradshaw]

On Tue, Dec 21, 2010 at 11:49 PM, Alexey Borzenkov sna...@gmail.com wrote:
 On Wed, Dec 22, 2010 at 1:19 AM, Alexey Borzenkov sna...@gmail.com wrote:
 I've recently started using Cython in one of my projects because I
 needed pyximport and automatic building of extensions and found that
 adding .pyxdep files makes Cython raise NameError because _test_files
 is not defined.
 [...]
 Also, there's another problem with pyximport that makes it unsafe to
 use it without build_in_temp=False. For example, if I have
 package1/module.pyx and package2/module.pyx then only one of those
 will build/import correctly,

 Btw, I just realized that Cython is finally using git and github, so
 if anyone is interested here are my patches:

 https://github.com/snaury/cython/commit/b43f68280919e98767bd4d163cb1eaf88c4c857e
 https://github.com/snaury/cython/commit/6f73b4280958f5d3f5fa77c3a4ed19b277784140

Thanks. pyximport is really cool, but no one's been maintaining it,
and it looks like we haven't been testing it either. (Code is either
new, tested, or broken...) At first glance the patches look fine, I'll
try to take a deeper look over the holidays (if no one beats me to
it...)

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14: KeyError during compilation of Yasm

[Robert Bradshaw]

On Tue, Dec 21, 2010 at 1:00 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 20.12.2010 20:35:
 On Sun, Dec 19, 2010 at 11:20 AM, Stefan Behnel wrote:
 The __new__ method must be called __cinit__.

 Without closer investigation, I suspect that the relevant change in Cython
 0.14 is that it knows about __new__ being a staticmethod. However, for
 cdef classes is silently translates __new__ into __cinit__ internally.
 It seems like both don't work together.

 Using __new__ instead of __cinit__ has been deprecated for years now,
 how about turning it into a clean error (which would fix this bug).

 +1, that's what I thought, too.

Done: 
https://github.com/cython/cython/commit/e3c9a78686b7a7f0d36da8e6189b1e1fb4037c73
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Hg to git

[Robert Bradshaw]

On Tue, Dec 21, 2010 at 3:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 21.12.2010 07:12:
 Anyone object to making http://hg.cython.org/ read-only, and making
 https://github.com/cython/cython/ (and its forks) the live devel
 branch?

 I think it's a good time to do this now that 0.14 is out.

Yep. Done.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14: KeyError during compilation of Yasm

[Robert Bradshaw]

On Sun, Dec 19, 2010 at 11:20 AM, Stefan Behnel stefan...@behnel.de wrote:
 Arfrever Frehtes Taifersar Arahesis, 19.12.2010 15:39:
 $ wget http://www.tortall.net/projects/yasm/releases/yasm-1.1.0.tar.gz
 ...
 $ tar -xzf yasm-1.1.0.tar.gz
 $ cd yasm-1.1.0
 $ ./configure --enable-python --enable-python-bindings
 ...
 $ make
 ...
 /usr/bin/python -c from Cython.Compiler.Main import main; 
 main(command_line=1) \
          -o yasm_python.c yasm.pyx
 Traceback (most recent call last):
 [...]
    File /usr/lib64/python2.7/site-packages/Cython/Compiler/Nodes.py, line 
 3249, in analyse_declarations
      self.body.analyse_declarations(scope)
    File /usr/lib64/python2.7/site-packages/Cython/Compiler/Nodes.py, line 
 346, in analyse_declarations
      stat.analyse_declarations(env)
    File /usr/lib64/python2.7/site-packages/Cython/Compiler/Nodes.py, line 
 1999, in analyse_declarations
      self.analyse_signature(env)
    File /usr/lib64/python2.7/site-packages/Cython/Compiler/Nodes.py, line 
 2097, in analyse_signature
      arg.hdr_type = sig.fixed_arg_type(i)
    File /usr/lib64/python2.7/site-packages/Cython/Compiler/TypeSlots.py, 
 line 100, in fixed_arg_type
      return self.format_map[self.fixed_arg_format[i]]
 KeyError: 'T'

 This problem doesn't occur with Cython 0.13.
 Is it a bug in Cython 0.14 or in Yasm?

 Both, I'd say. The crash is in line 100 of yasm.pyx, where it says:

 cdef class __assoc_data_callback:
     cdef yasm_assoc_data_callback *cb
     def __new__(self, destroy, print_):          # === HERE
         self.cb = yasm_assoc_data_callback *  \
                   malloc(sizeof(yasm_assoc_data_callback))
         self.cb.destroy = void (*) (void *)PyCObject_AsVoidPtr(destroy)
         #self.cb.print_ = void (*) (void *, FILE *, int) \
                            PyCObject_AsVoidPtr(print_)


 The __new__ method must be called __cinit__.

 Without closer investigation, I suspect that the relevant change in Cython
 0.14 is that it knows about __new__ being a staticmethod. However, for
 cdef classes is silently translates __new__ into __cinit__ internally.
 It seems like both don't work together.

Using __new__ instead of __cinit__ has been deprecated for years now,
how about turning it into a clean error (which would fix this bug).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Compiler crash with pyrex_gdb=True

[Robert Bradshaw]

On Sat, Dec 18, 2010 at 10:56 AM, mark florisson
markflorisso...@gmail.com wrote:


 On 18 December 2010 18:19, Vineet Jain vinjv...@gmail.com wrote:

 The error can be produced by directly using cython as well. See below:
 Python version:

 Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
 [GCC 4.4.5] on linux2
 Type help, copyright, credits or license for more information.

 v...@host61:/host/trading/pytrade$ python-dbg
 Python 2.6.6 (r266:84292, Sep 15 2010, 17:09:03)
 [GCC 4.4.5] on linux2
 Type help, copyright, credits or license for more information.

 cython --gdb container.pyx
 Error compiling Cython file:
 
 ...
 def test(param):
     def test2():
    ^
 
 container.pyx:2:4: Compiler crash in DebugTransform
 ModuleNode.body = StatListNode(container.pyx:1:0)
 StatListNode.stats[0] = DefNode(container.pyx:1:0,
     modifiers = [...]/0,
     name = u'test',
     needs_closure = True,
     num_required_args = 1,
     reqd_kw_flags_cname = '0')
 DefNode.body = StatListNode(container.pyx:2:4)
 StatListNode.stats[0] = DefNode(container.pyx:2:4,
     modifiers = [...]/0,
     name = u'test2',
     needs_outer_scope = True,
     reqd_kw_flags_cname = '0')
 Compiler crash traceback from this point on:
   File Visitor.py, line 173, in
 Cython.Compiler.Visitor.TreeVisitor._visitchild
 (/root/Cython-0.14/Cython/Compiler/Visitor.c:3444)
   File
 /usr/local/lib/python2.6/dist-packages/Cython/Compiler/ParseTreeTransforms.py,
 line 1646, in visit_FuncDefNode
     self.serialize_local_variables(node.local_scope.entries)
   File
 /usr/local/lib/python2.6/dist-packages/Cython/Compiler/ParseTreeTransforms.py,
 line 1703, in serialize_local_variables
     qualified_name=entry.qualified_name,
 AttributeError: 'Entry' object has no attribute 'qualified_name'
 [126744 refs]


 On Sat, Dec 18, 2010 at 11:38 AM, Vineet Jain vinjv...@gmail.com wrote:

 Fails to compile when I add
         pyrex_gdb=True
 to my setup.py file.

 def test(param):
     def test2():
         x = param.x
 rror compiling Cython file:
 
 ...
 def test(param):
     def test2():
    ^
 
 container.pyx:2:4: Compiler crash in DebugTransform


 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


 I fixed the issue and pushed to https://github.com/markflorisson88/cython ,
 but I was actually not able to reproduce the crash. For me it generated some
 extra XML that would simply be ignored by the debugger (which would mean you
 wouldn't be able to set a Cython breakpoint for the inner function). This
 was even the case with a fresh 0.14 non-development branch copy.

I was never able to reproduce the crash either, but +1 to closure
support. Is your branch ready to pull from? (The fix looks pretty
clean.)

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] provide 'builtins' builtin module also for -2 mode

[Robert Bradshaw]

On Thu, Dec 16, 2010 at 2:11 PM, Stefan Behnel stefan...@behnel.de wrote:
 [CC-ing cython-dev]

 Lisandro Dalcin, 16.12.2010 23:03:
 On 16 December 2010 18:53, Stefan Behnel wrote:
 Lisandro Dalcin, 16.12.2010 22:33:

 Just a few comments about this changeset:

 changeset:   4244:e0508ac59b35
 user:        Stefan Behnel
 date:        Thu Dec 16 20:11:29 2010 +0100
 summary:     provide 'builtins' builtin module in -3 mode


 IIUC, under -3 we can use both __builtin__ and builtins, but under
 -2 only __builtin__. Why not to also provide builtins under -2
 mode? We are already supporting builtins in some parts, for example
 at ModuleNode.py, method generate_type_import_call() ...

 Well, as long as it actually means that users can import any of the two in
 their code and have it work at runtime in both Py2 and Py3?

 Yes, of course...

 But that would
 require that Cython changes the user imported module name depending on the C
 compile time environment. I'm not sure that's a good idea.

 Well, the code at generate_type_import_call() work along these lines,
 right? I mean, it translates __builtin__ and builtins to the right
 name in the C runtime.

 But that's only when it actually runs in Py3, which is still in eventual
 migration state for many users. As long as you keep your code in a Py2
 environment and only import __builtin__, Cython won't do any magic behind
 your back.


 The problem I see is that users may have a builtins module lying around
 somewhere in their Py2 code base that Cython would misinterpret and wouldn't
 be able to import. builtins isn't that a bad name for a module, e.g. in a
 plugin environment...

 Yes, you have a good point. However the import would need to be
 absolute for Cython to misinterpret it, right? I would argue that such
 code smells...

 It didn't smell all that much before Py3 renamed the module.

 Anyway, I'm +0 on changing this. Let's hear what the others think.

I'm +0 as well.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] Hg to git

[Robert Bradshaw]

Anyone object to making http://hg.cython.org/ read-only, and making
https://github.com/cython/cython/ (and its forks) the live devel
branch?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] bug report, operator overloading for cdef classes

[Robert Bradshaw]

On Sun, Dec 19, 2010 at 1:31 AM, Kornél JAHN corn...@gmail.com wrote:
 Hello,
 I have been experiencing strange behavior regarding overloaded operators for
 cdef classes, using Cython 0.14. I think it can be regarded as a bug. See
 the example code below:
 # File: opoverload.pyx
 class Test1:
     def __rmul__(self, other):
         print Calling Test1.__rmul__() with, type(self), type(other)
     def __mul__(self, other):
         print Calling Test1.__mul__() with, type(self), type(other)
 cdef class Test2:
     def __rmul__(self, other):
         print Calling Test2.__rmul__() with, type(self), type(other)
     def __mul__(self, other):
         print Calling Test2.__mul__() with, type(self), type(other)
 Python console:
 from opoverload import *
 Test1() * 2.0
 Calling Test1.__mul__() with type 'instance' type 'float'
 2.0 * Test1()
 Calling Test1.__rmul__() with type 'instance' type 'float'
 Test2() * 2.0
 Calling Test2.__mul__() with type 'opoverload.Test2' type 'float'
 2.0 * Test2()
 Calling Test2.__mul__() with type 'float' type 'opoverload.Test2'
 Test2.__rmul__() should be called and type(self) should not become float!

Have you read http://docs.cython.org/src/userguide/special_methods.html ?

 Can you please suggest a temporary workaround?

Manually check the types in your arithmetic functions. (Yes, I know
that's not as nice as it could be, it's just a reflection of the
underlying C API that we haven't glossed over.)

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] bug report, operator overloading for cdef classes

[Robert Bradshaw]

On Sun, Dec 19, 2010 at 1:47 AM, Kornél JAHN corn...@gmail.com wrote:
 Thank you for your quick reply, you are right, I ought to have read it.
 Sorry for bothering.

No problem, you're not the first to be bitten by that. It's an
unfortunate wart in the language, but fortunately one of the few.

 On Sun, Dec 19, 2010 at 10:40 AM, Robert Bradshaw
 rober...@math.washington.edu wrote:

 On Sun, Dec 19, 2010 at 1:31 AM, Kornél JAHN corn...@gmail.com wrote:
  Hello,
  I have been experiencing strange behavior regarding overloaded operators
  for
  cdef classes, using Cython 0.14. I think it can be regarded as a bug.
  See
  the example code below:
  # File: opoverload.pyx
  class Test1:
      def __rmul__(self, other):
          print Calling Test1.__rmul__() with, type(self), type(other)
      def __mul__(self, other):
          print Calling Test1.__mul__() with, type(self), type(other)
  cdef class Test2:
      def __rmul__(self, other):
          print Calling Test2.__rmul__() with, type(self), type(other)
      def __mul__(self, other):
          print Calling Test2.__mul__() with, type(self), type(other)
  Python console:
  from opoverload import *
  Test1() * 2.0
  Calling Test1.__mul__() with type 'instance' type 'float'
  2.0 * Test1()
  Calling Test1.__rmul__() with type 'instance' type 'float'
  Test2() * 2.0
  Calling Test2.__mul__() with type 'opoverload.Test2' type 'float'
  2.0 * Test2()
  Calling Test2.__mul__() with type 'float' type 'opoverload.Test2'
  Test2.__rmul__() should be called and type(self) should not become
  float!

 Have you read http://docs.cython.org/src/userguide/special_methods.html ?

  Can you please suggest a temporary workaround?

 Manually check the types in your arithmetic functions. (Yes, I know
 that's not as nice as it could be, it's just a reflection of the
 underlying C API that we haven't glossed over.)

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev



 --
 Kornél JAHN
 engineering physicist, PhD student
 Budapest University of Technology and Economics, Hungary

 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython debugging wiki

[Robert Bradshaw]

Yes, I'd say so. Perhaps we should replace the entire page with a link
to the docs, in case it's being linked to elsewhere.

On Wed, Dec 15, 2010 at 12:50 PM, mark florisson
markflorisso...@gmail.com wrote:
 With the addition of cygdb
 (http://docs.cython.org/src/userguide/debugging.html),  should we now remove
 the debugging wiki (http://wiki.cython.org/DebuggingTechniques)? It seems
 kind of obsolete now.
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] failing test: py_unicode_type

[Robert Bradshaw]

On Tue, Dec 14, 2010 at 12:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 Lisandro Dalcin, 14.12.2010 00:46:
 No clue about what's going on here...

 $ python runtests.py py_unicode_type
 Python 2.6.4 (r264:75706, Jun  4 2010, 18:20:16)
 [GCC 4.4.4 20100503 (Red Hat 4.4.4-2)]

 Running tests against Cython 0.14.rc0

 py_unicode_type.c:1:2: error: #error Do not use this file, it is the
 result of a failed Cython compilation.

 That was one of the three broken tests I meantioned earlier. Seems to be
 fixed now.

Yep, I fixed that. It was a missing None check that was getting triggered.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] ANN: Cython 0.14 released

[Robert Bradshaw]

The Cython project is happy to announce the release of Cython 0.14.

This release fixes several bugs in the previous 0.13 release, improved
debugging and build support, and heavily improves Python compatibility
in terms of supported features. It can be downloaded from
http://cython.org/ or http://pypi.python.org/pypi/Cython/0.14

Release notes: http://wiki.cython.org/ReleaseNotes-0.14


== New Features ==

 * Python classes can now be nested and receive a proper closure at
definition time.

 * Redefinition is supported for Python functions, even within the same scope.

 * Lambda expressions are supported in class bodies and at the module level.

 * Metaclasses are supported for Python classes, both in Python 2 and
Python 3 syntax. The Python 3 syntax (using a keyword argument in the
type declaration) is preferred and optimised at compile time.

 * final extension classes prevent inheritance in Python space. This
feature is available through the new cython.final decorator. In the
future, these classes may receive further optimisations.

 * internal extension classes do not show up in the module
dictionary. This feature is available through the new
cython.internal decorator.

 * Extension type inheritance from builtin types, such as cdef class
MyUnicode(unicode), now works without further external type
redeclarations (which are also strongly discouraged now and continue
to issue a warning).

 * GDB support. http://docs.cython.org/src/userguide/debugging.html

 * A new build system with support for inline distutils directives,
correct dependency tracking, and parallel compilation.
http://wiki.cython.org/enhancements/distutils_preprocessing

 * Support for dynamic compilation at runtime via the new
cython.inline function and cython.compile decorator.
http://wiki.cython.org/enhancements/inline



== General improvements and bug fixes ==

 * In parallel assignments, the right side was evaluated in reverse
order in 0.13. This could result in errors if it had side effects
(e.g. function calls).

 * In some cases, methods of builtin types would raise a SystemError
instead of an AttributeError when called on None.

 * Constant tuples are now cached over the lifetime of an extension
module, just like CPython does. Constant argument tuples of Python
function calls are also cached.

 * Closures have tightened to include exactly the names used in the
inner functions and classes. Previously, they held the complete locals
of the defining function.

 * nogil blocks are supported when compiling pure Python code by
writing with cython.nogil.

 * The builtin next() function in Python 2.6 and later is now
implemented internally and therefore available in all Python versions.
This makes it the preferred and portable way of manually advancing an
iterator.

 * In addition to the previously supported inlined generator
expressions in 0.13, sorted(genexpr) can now be used as well. Typing
issues were fixed in sum(genexpr) that could lead to invalid C code
being generated. Other known issues with inlined generator expressions
were also fixed that make upgrading to 0.14 a strong recommendation
for code that uses them. Note that general generators and generator
expressions continue to be not supported.

 * Iterating over arbitrary pointer types is now supported, as is an
optimized version of the in operator, e.g. x in ptr[a:b].

 * Inplace arithmetic operators now respect the cdivision directive
and are supported for complex types.



== Incompatible changes ==

 * Typing a variable as type complex previously gave it the Python
object type. It now uses the appropriate C/C++ {{{double complex}}}
type. A side-effect is that assignments and typed function parameters
now accept anything that Python can coerce to a complex, including
integers and floats, and not only complex instances.

 * Large integer literals pass through the compiler in a safer way. To
prevent truncation in C code, non 32-bit literals are turned into
Python objects if not used in a C context. This context can either be
given by a clear C literal suffix such as UL or LL (or L in
Python 3 code), or it can be an assignment to a typed variable or a
typed function argument, in which case it is up to the user to take
care of a sufficiently large value space of the target.

 * Python functions are declared in the order they appear in the file,
rather than all being created at module creation time. This is
consistent with Python and needed to support, for example, conditional
or repeated declarations of functions. In the face of circular imports
this may cause code to break, so a new
{{{--disable-function-redefinition}}} flag was added to revert to the
old behavior. This flag will be removed in a future release, so should
only be used as a stopgap until old code can be fixed.



== Contributors ==

Many people contributed to this release, including:

 * Haoyu Bai
 * Stefan Behnel
 * Robert Bradshaw
 * Ondrej Certik
 * Lisandro Dalcin
 * Mark Florisson
 * Eric Huss
 * Vitja

Re: [Cython] Cython 0.14 release candidate

[Robert Bradshaw]

On Sat, Dec 11, 2010 at 9:45 PM, Dag Sverre Seljebotn
da...@student.matnat.uio.no wrote:
 On 12/12/2010 05:29 AM, Robert Bradshaw wrote:
 On Sat, Dec 11, 2010 at 7:07 PM, Robert Bradshaw
 rober...@math.washington.edu  wrote:

 On Sat, Dec 11, 2010 at 5:52 PM, Robert Kernrobert.k...@gmail.com  wrote:

 On 2010-12-11 17:09 , Robert Bradshaw wrote:

 On Sat, Dec 11, 2010 at 1:44 PM, Arfrever Frehtes Taifersar Arahesis
 arfrever@gmail.com    wrote:

 2010-12-11 12:01:11 Robert Bradshaw napisał(a):

 Release candidate up at
 http://cython.org/release/Cython-0.14.rc0.tar.gz . Unless something
 bad is discovered, this will be the release sometime early next week.

 Some tests fail.

 Test failures with Python 2.6 (the same tests fail with Python 2.7):

 [...]

 Not sue what's up with all these numpy errors, but many (not all) of
 them look like a change in formatting of error messages. The basic
 functionality still seems to be passing--does anyone know if this is a
 regression?

 I think so. I think these are real regressions:

    ValueError: Does not understand character buffer dtype format string 
 (':')

   From regular Python, I've tried a few of the dtype() calls that you 
 make, and
 they seem to work fine. I don't see a problem in the generated code, 
 though.

 They all work for me too, but maybe that's because I'm using an older
 version of NumPy (the one that ships with Sage). Has anyone tried
 Numpy 1.5 with Cython 0.13?

 OK, I just install Numpy 1.5.1, and all the tests pass for me with
 both 0.13 and 0.14.rc0 (Python 2.6, OS X). Anyone else able to
 reproduce these errors? Arfrever, can you try Cython 0.13 in your
 setup?


 The thing that has happened here is that the new NumPy supports the PEP
 3118 protocol, and so the code in numpy.pxd is not used, but instead the
 other code. Probably new tests should be written, perhaps some changes
 must be done to buffer format string parsing, and so on. But it is
 unlikely to be a regression in Cython, nothing changed here w.r.t
 Cython  0.13.x.

I've created http://trac.cython.org/cython_trac/ticket/630 , hopefully
someone will look into this.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 release candidate

[Robert Bradshaw]

On Sun, Dec 12, 2010 at 8:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 [remainings of a completely unrelated thread deleted]

 Vitja Makarov, 12.12.2010 14:51:
 Cython-tests are now broken...

 Yep, the new class scope for the conjugate changes broke three tests.
 Unless it's an easy fix, I'd vote for reverting this post-RC change.

Oops. I'll fix that.

At this point everything pushed to cython-devel is post-release,
unless something really serious comes up.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 release candidate

[Robert Bradshaw]

On Mon, Dec 13, 2010 at 9:32 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2010/12/13 Robert Bradshaw rober...@math.washington.edu:
 On Sun, Dec 12, 2010 at 8:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 [remainings of a completely unrelated thread deleted]

 Vitja Makarov, 12.12.2010 14:51:
 Cython-tests are now broken...

 Yep, the new class scope for the conjugate changes broke three tests.
 Unless it's an easy fix, I'd vote for reverting this post-RC change.

 Oops. I'll fix that.

 At this point everything pushed to cython-devel is post-release,
 unless something really serious comes up.

 - Robert

 What's about trailing spaces? ;)

OK, fair enough. Done. That's not a big code change though. I haven't
yet pulled into the generators branch, but I think git has a nice
ignore-trailing-whitespace-conflicts merge option.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] Sage broken

[Robert Bradshaw]

It appears that something broke circular cimports in Cython, and now
Sage won't start up (as of
https://sage.math.washington.edu:8091/hudson/job/sage-tests/388/ )
Unfortunately, the tests reported 0 failures for not starting up
(fixed now), so it looked like all was well.

It looks like the offending patch is
http://hg.cython.org/cython-devel/rev/6c79b26286ce (or somewhere close
by). Any ideas? I'm looking into this now.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage broken

[Robert Bradshaw]

On Mon, Dec 13, 2010 at 9:41 PM, Robert Bradshaw
rober...@math.washington.edu wrote:
 It appears that something broke circular cimports in Cython, and now
 Sage won't start up (as of
 https://sage.math.washington.edu:8091/hudson/job/sage-tests/388/ )
 Unfortunately, the tests reported 0 failures for not starting up
 (fixed now), so it looked like all was well.

 It looks like the offending patch is
 http://hg.cython.org/cython-devel/rev/6c79b26286ce (or somewhere close
 by). Any ideas? I'm looking into this now.

Specifically, it's somewhere between
http://hg.cython.org/cython-devel/rev/06a35b532a49 and
http://hg.cython.org/cython-devel/rev/c48f814a8481

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Sage broken

[Robert Bradshaw]

On Mon, Dec 13, 2010 at 9:45 PM, Robert Bradshaw
rober...@math.washington.edu wrote:
 On Mon, Dec 13, 2010 at 9:41 PM, Robert Bradshaw
 rober...@math.washington.edu wrote:
 It appears that something broke circular cimports in Cython, and now
 Sage won't start up (as of
 https://sage.math.washington.edu:8091/hudson/job/sage-tests/388/ )
 Unfortunately, the tests reported 0 failures for not starting up
 (fixed now), so it looked like all was well.

 It looks like the offending patch is
 http://hg.cython.org/cython-devel/rev/6c79b26286ce (or somewhere close
 by). Any ideas? I'm looking into this now.

 Specifically, it's somewhere between
 http://hg.cython.org/cython-devel/rev/06a35b532a49 and
 http://hg.cython.org/cython-devel/rev/c48f814a8481

FYI, I've figured out what the problem was. With the new function
re-declaration code, def functions are created and added to the module
as it is executed, rather than initialized as part of the module
creation from the method_def struct. While this is arguably the more
correct behavior, it broke Sage which was depending on the fact that
functions were defined before type imports, and manifested itself
because the import was circular.

I've added a --disable-function-redefinition option for legacy code
like this (which should in the long run get cleaned up, but I don't
think we should break outright break it now). Of course, if set, this
disables module-level lambdas, etc., but none of those are in use for
old code. I'm not sure exactly what to do about this in the long term,
but I'm wondering (but not sure) if we should have import, cimport,
cdef class, etc. all happen as if they were executed in sequence too,
especially in the face of circular imports.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 release candidate

[Robert Bradshaw]

Release candidate up at
http://cython.org/release/Cython-0.14.rc0.tar.gz . Unless something
bad is discovered, this will be the release sometime early next week.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14 release candidate

[Robert Bradshaw]

On Sat, Dec 11, 2010 at 1:44 PM, Arfrever Frehtes Taifersar Arahesis
arfrever@gmail.com wrote:
 2010-12-11 12:01:11 Robert Bradshaw napisał(a):
 Release candidate up at
 http://cython.org/release/Cython-0.14.rc0.tar.gz . Unless something
 bad is discovered, this will be the release sometime early next week.

 Some tests fail.

:(

 Test failures with Python 2.6 (the same tests fail with Python 2.7):

What OS and NumPy version?

 ==
 FAIL: Doctest: numpy_test
 --
 Traceback (most recent call last):
  File /usr/lib64/python2.6/doctest.py, line 2163, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
 AssertionError: Failed doctest test for numpy_test
  File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 25, in numpy_test

 --
 File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 114, in numpy_test
 Failed example:
    test_c_contig(f_arr)
 Expected:
    Traceback (most recent call last):
       ...
    ValueError: ndarray is not C contiguous
 Got:
    Traceback (most recent call last):
      File /usr/lib64/python2.6/doctest.py, line 1253, in __run
        compileflags, 1) in test.globs
      File doctest numpy_test[29], line 1, in module
        test_c_contig(f_arr)
      File numpy_test.pyx, line 265, in numpy_test.test_c_contig 
 (numpy_test.c:2619)
    ValueError: ndarray is not C-contiguous
 --
 File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 122, in numpy_test
 Failed example:
    test_c_contig(c_arr[::2,::2])
 Expected:
    Traceback (most recent call last):
       ...
    ValueError: ndarray is not C contiguous
 Got:
    Traceback (most recent call last):
      File /usr/lib64/python2.6/doctest.py, line 1253, in __run
        compileflags, 1) in test.globs
      File doctest numpy_test[31], line 1, in module
        test_c_contig(c_arr[::2,::2])
      File numpy_test.pyx, line 265, in numpy_test.test_c_contig 
 (numpy_test.c:2619)
    ValueError: ndarray is not C-contiguous
 --
 File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 161, in numpy_test
 Failed example:
    test_dtype('%si' % other_endian, inc1_int)
 Expected:
    Traceback (most recent call last):
       ...
    ValueError: Non-native byte order not supported
 Got:
    Traceback (most recent call last):
      File /usr/lib64/python2.6/doctest.py, line 1253, in __run
        compileflags, 1) in test.globs
      File doctest numpy_test[60], line 1, in module
        test_dtype('%si' % other_endian, inc1_int)
      File numpy_test.pyx, line 339, in numpy_test.test_dtype 
 (numpy_test.c:5311)
      File numpy_test.pyx, line 280, in numpy_test.inc1_int 
 (numpy_test.c:3129)
    ValueError: Big-endian buffer not supported on little-endian compiler
 --
 File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 168, in numpy_test
 Failed example:
    test_recordarray()
 Exception raised:
    Traceback (most recent call last):
      File /usr/lib64/python2.6/doctest.py, line 1253, in __run
        compileflags, 1) in test.globs
      File doctest numpy_test[61], line 1, in module
        test_recordarray()
      File numpy_test.pyx, line 347, in numpy_test.test_recordarray 
 (numpy_test.c:5474)
    ValueError: Does not understand character buffer dtype format string (':')
 --
 File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 170, in numpy_test
 Failed example:
    print(test_nested_dtypes(np.zeros((3,), dtype=np.dtype([            ('a', 
 np.dtype('i,i')),            ('b', np.dtype('i,i'))        ]
 Exception raised:
    Traceback (most recent call last):
      File /usr/lib64/python2.6/doctest.py, line 1253, in __run
        compileflags, 1) in test.globs
      File doctest numpy_test[62], line 1, in module
        print(test_nested_dtypes(np.zeros((3,), dtype=np.dtype([            
 ('a', np.dtype('i,i')),            ('b', np.dtype('i,i'))        ]
      File numpy_test.pyx, line 374, in numpy_test.test_nested_dtypes 
 (numpy_test.c:5792)
    ValueError: Does not understand character buffer dtype format string (':')
 --
 File 
 /var/tmp/portage/dev-python/cython-0.14_rc0/work/Cython-0.14.rc0/BUILD/run/c/numpy_test.so,
  line 174, in numpy_test
 Failed

Re: [Cython] Cython 0.14 release candidate

[Robert Bradshaw]

On Sat, Dec 11, 2010 at 9:45 PM, Dag Sverre Seljebotn
da...@student.matnat.uio.no wrote:
 On 12/12/2010 05:29 AM, Robert Bradshaw wrote:
 On Sat, Dec 11, 2010 at 7:07 PM, Robert Bradshaw
 rober...@math.washington.edu  wrote:

 On Sat, Dec 11, 2010 at 5:52 PM, Robert Kernrobert.k...@gmail.com  wrote:

 On 2010-12-11 17:09 , Robert Bradshaw wrote:

 On Sat, Dec 11, 2010 at 1:44 PM, Arfrever Frehtes Taifersar Arahesis
 arfrever@gmail.com    wrote:

 2010-12-11 12:01:11 Robert Bradshaw napisał(a):

 Release candidate up at
 http://cython.org/release/Cython-0.14.rc0.tar.gz . Unless something
 bad is discovered, this will be the release sometime early next week.

 Some tests fail.

 Test failures with Python 2.6 (the same tests fail with Python 2.7):

 [...]

 Not sue what's up with all these numpy errors, but many (not all) of
 them look like a change in formatting of error messages. The basic
 functionality still seems to be passing--does anyone know if this is a
 regression?

 I think so. I think these are real regressions:

    ValueError: Does not understand character buffer dtype format string 
 (':')

   From regular Python, I've tried a few of the dtype() calls that you 
 make, and
 they seem to work fine. I don't see a problem in the generated code, 
 though.

 They all work for me too, but maybe that's because I'm using an older
 version of NumPy (the one that ships with Sage). Has anyone tried
 Numpy 1.5 with Cython 0.13?

 OK, I just install Numpy 1.5.1, and all the tests pass for me with
 both 0.13 and 0.14.rc0 (Python 2.6, OS X). Anyone else able to
 reproduce these errors? Arfrever, can you try Cython 0.13 in your
 setup?


 The thing that has happened here is that the new NumPy supports the PEP
 3118 protocol, and so the code in numpy.pxd is not used, but instead the
 other code. Probably new tests should be written, perhaps some changes
 must be done to buffer format string parsing, and so on. But it is
 unlikely to be a regression in Cython, nothing changed here w.r.t
 Cython  0.13.x.

That would make sense, thanks. I didn't think much had changed with
numpy support on our side, but didn't know for sure.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Fri, Dec 10, 2010 at 12:22 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2010/12/10 Stefan Behnel stefan...@behnel.de:
 Stefan Behnel, 09.12.2010 10:46:
 Vitja Makarov, 08.12.2010 22:45:
 Please review this patch. It's not yet finished, and mostly doesn't work.
 But you can take a look at patch and generated code.

 I'll take a close look ASAP.

    - Temps are saved/restored/allocated inside YieldExprNode using
 helper ClosureTempAllocator

 Ok, that currently lives in ParseTreeTransforms, which is totally the wrong
 place for anything that generates code. It also mixes functionality that
 normally lives in Symtab.py with functionality that lives in Code.py.


 Yes, I know. That was placed there to give it a try (actually I'm not
 sure what is the best place for it). ClosureTempAllocator now is
 attribute of defnode and all yieldnodes, that's not clean.

 ExprNode.StopIterationNode isn't expression should it be moved to Nodes.py?

 create_abstract_generator() is rather dirty

 I don't know how to bind C functions to CClass.

 I think that generator body could have PyObject *body(PyObject *self,
 PyObject *value) signature
 With value=Null meaning exception.

 Could you give me (scoder, or better all Cython devs) commit rights to
 your repo so that I can start refactoring the code and fix problems I find?


 I've added you, but I don't know how to add all devs together.

I don't think there's a way to do that, just add whoever you want help
participating on this branch. My username is robertwb, and though I
haven't rally had time to read any code yet, I'm very interested and
would like to help out if I can.

 I could also add Hudson jobs for it. Robert, would it be ok to add them for
 all Python versions?

I think that would be excessive. Lets just do 2.4, 2.7, and a 3.x. If
there's a strange corner case that only manifests in, say, 2.5 then
we'll deal with it later.

 Yeah, please that would be nice.

    - PyxGenerator methods are defined via declare_var

 Should we call that CyGenerator ? :)

 I actually meant to call it CyGenerator at the Python level, but I now
 see that it's actually a purely internal thing. That makes __Pyx_Generator
 a better choice, given that __pyx_ is the globally reserved prefix in
 Cython code.


 Should rename it back ;)

 Eventually, we may(!) want to reuse the superclass across modules, but I
 currently have no idea how that could be made to work. So it's just another
 thing that we don't need to worry about now.


 Okay.

 How can I declare function entry with METH_VARGS signature?
 Using TypeSlots I can add METH_VARGS|METH_KWARGS entries but not
 single METH_VARGS

 That's preferd for throw method.

Just let it be METH_VARGS|METH_KWARGS for now. That doesn't mean it
can actually take arbitrary keyword arguments, just that the arguments
that are needed can be passed in as keywords.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Fri, Dec 10, 2010 at 11:15 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2010/12/10 Robert Bradshaw rober...@math.washington.edu:
 On Fri, Dec 10, 2010 at 12:22 AM, Vitja Makarov vitja.maka...@gmail.com 
 wrote:
 2010/12/10 Stefan Behnel stefan...@behnel.de:
 Stefan Behnel, 09.12.2010 10:46:
 Vitja Makarov, 08.12.2010 22:45:
 Please review this patch. It's not yet finished, and mostly doesn't work.
 But you can take a look at patch and generated code.

 I'll take a close look ASAP.

    - Temps are saved/restored/allocated inside YieldExprNode using
 helper ClosureTempAllocator

 Ok, that currently lives in ParseTreeTransforms, which is totally the wrong
 place for anything that generates code. It also mixes functionality that
 normally lives in Symtab.py with functionality that lives in Code.py.


 Yes, I know. That was placed there to give it a try (actually I'm not
 sure what is the best place for it). ClosureTempAllocator now is
 attribute of defnode and all yieldnodes, that's not clean.

 ExprNode.StopIterationNode isn't expression should it be moved to Nodes.py?

 create_abstract_generator() is rather dirty

 I don't know how to bind C functions to CClass.

 I think that generator body could have PyObject *body(PyObject *self,
 PyObject *value) signature
 With value=Null meaning exception.

 Could you give me (scoder, or better all Cython devs) commit rights to
 your repo so that I can start refactoring the code and fix problems I find?


 I've added you, but I don't know how to add all devs together.

 I don't think there's a way to do that, just add whoever you want help
 participating on this branch. My username is robertwb, and though I
 haven't rally had time to read any code yet, I'm very interested and
 would like to help out if I can.


 I've added you too.

Thanks.

 I could also add Hudson jobs for it. Robert, would it be ok to add them for
 all Python versions?

 I think that would be excessive. Lets just do 2.4,

(I meant 2.3 here...)

 2.7, and a 3.x. If
 there's a strange corner case that only manifests in, say, 2.5 then
 we'll deal with it later.


 Can you please add pyregr for 2.7 also?

Sure, that makes sense as well.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Fri, Dec 10, 2010 at 2:38 PM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 10.12.2010 22:01:
 I've pushed commit that moves exc_save_vars into temps

 Yes, I think that makes good sense and should simplify things a lot.

 (now it's declared with type=py_object_type and manage_refs=False)

 Sounds right.

Can't their be multiple nestings of exc_save_vars (if try statements
are nested)?

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

[Cython] Release candidate/feature freeze

[Robert Bradshaw]

I'm planning on posting a release candidate tonight, at which point I
would no longer be pulling from the devel branch. If you know of any
features/bugfixes that should go in, let me know.

If all goes well, we'll release early next week. We also need to
finish fleshing out the release notes and documentation (myself
included).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Thu, Dec 9, 2010 at 1:46 AM, Stefan Behnel stefan...@behnel.de wrote:
 Vitja Makarov, 08.12.2010 22:45:
 Please review this patch. It's not yet finished, and mostly doesn't work.
 But you can take a look at patch and generated code.

 *Way* cool, thanks! I'll take a close look ASAP. Please feel free to send
 in a new patch when you have it. If you use hg, you can use hg bundle to
 collect commit series. Don't know about git.

 BTW, why not set up a github branch for this?

That's exactly what I was thinking. It'll make it easier to review as
well, and for others to contribute without disturbing the main line.

   - Temps are saved/restored/allocated inside YieldExprNode using
 helper ClosureTempAllocator
   - PyxGenerator methods are defined via declare_var

 Should we call that CyGenerator ? :)


   - Send,  __next__(), __iter__() are implemented while close(), throw() 
 aren't
   - YieldExprNode doesn't handle exceptions

 Ok, sure, that's why close() and throw() don't work yet. I think that's
 fine for now. The first three are the most interesting ones. However,
 adding exception support shouldn't be hard at all, AFAICT, but they may
 benefit from a cdef function for the generator body in order to properly
 pass in the exception propagation trigger.


   - I don't know how to make refnanny work, should refnanny context be
 stored in closure?

 Good call. That's a tricky question.

 What about generating a GIVEREF for every Python temp you store in the
 closure, a GOTREF for everything you take back out, and finish/restart the
 refnanny context around the yield?

That's what I would do as well.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Thu, Dec 9, 2010 at 2:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 09.12.2010 11:00:
 On Thu, Dec 9, 2010 at 1:46 AM, Stefan Behnel wrote:
 Vitja Makarov, 08.12.2010 22:45:
 Please review this patch. It's not yet finished, and mostly doesn't work.
 But you can take a look at patch and generated code.

 *Way* cool, thanks! I'll take a close look ASAP. Please feel free to send
 in a new patch when you have it. If you use hg, you can use hg bundle to
 collect commit series. Don't know about git.

 BTW, why not set up a github branch for this?

 That's exactly what I was thinking. It'll make it easier to review as
 well, and for others to contribute without disturbing the main line.

 BTW, when this goes in, the next Cython version to release is basically
 1.0, right? There are some minor Python compatibility issues left, but
 that'll be the last major feature, the way I count it.

This will be the last major feature before 1.0, but I don't know that
the first release this is in should be the 1.0 release.

I also think we should set the compatibility bar a bit higher for 1.0
as well--I want a concrete list of all the differences between Python
X.y and Cython (for un-annotated code) and to be able to say that, if
you don't count on the behavior in this list (e.g. identity vs
equality for floating point literals, maybe stack frames) then you
should be able to compile your code and have it just work, but
hopefully faster. At the very least, until we at least attempt to make
such a list, we won't know where we stand (the regression tests
presumably being somewhat of a starting point, though I'd like to get
all of them passing (unless they deal with implementation
details...)).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] make cython stop on the first error

[Robert Bradshaw]

On Sun, Dec 5, 2010 at 5:42 PM, Ondrej Certik ond...@certik.cz wrote:
 On Sat, Dec 4, 2010 at 11:49 PM, Robert Bradshaw
 rober...@math.washington.edu wrote:
 On Sat, Dec 4, 2010 at 9:25 PM, Dag Sverre Seljebotn
 da...@student.matnat.uio.no wrote:
 On 12/05/2010 01:27 AM, Ondrej Certik wrote:
 On Sat, Dec 4, 2010 at 3:38 PM, Vitja Makarovvitja.maka...@gmail.com  
 wrote:

 2010/12/4 Ondrej Certikond...@certik.cz:

 Hi,

 I was missing the option to tell Cython to stop on the first error. So
 I have implemented it:

 https://github.com/cython/cython/pull/3

 Ondrej


 Hi!

 Is that a good idea to call sys.exit() in Errors.py maybe there is
 another way to stop compilation?

 Yes, that is something that I was not sure when I coded it, as I am
 not too familiar with Cython internals.


 The usecases are more important than the internals: Imagine for a moment
 what this will to do pyximport, or build systems able to launch Cython
 in-process (SCons, waf).

 I've made it raise the same error it does between phases to signal the
 compiler should abort.

 After this abort change, here is how it looks like:


 ond...@crow:~/repos/hermes/hermes1d/hermes1d(fix)$ make
 [  4%] Built target cython_utils
 [  8%] Built target numpy_utils
 [ 93%] Built target hermes1d-debug
 [ 95%] Cythonizing h1d_wrapper.pyx

 Error compiling Cython file:
 
 ...
        prj_type = hermes1d.H1D_H1_ortho_global
    else:
        raise ValueError(Unknown projection type)
    global _A
    _A = f
    hermes1d.assemble_projection_matrix_rhs(mesh.thisptr, A.thisptr,
                                               ^
 

 /home/ondrej/repos/hermes/hermes1d/hermes1d/h1d_wrapper/h1d_wrapper.pyx:369:48:
 Cannot convert 'Space *' to Python object

 Error compiling Cython file:
 
 ...
        prj_type = hermes1d.H1D_H1_ortho_global
    else:
        raise ValueError(Unknown projection type)
    global _A
    _A = f
    hermes1d.assemble_projection_matrix_rhs(mesh.thisptr, A.thisptr,
                                               ^
 

 /home/ondrej/repos/hermes/hermes1d/hermes1d/h1d_wrapper/h1d_wrapper.pyx:369:48:
 Compiler crash in AnalyseExpressionsTransform

 ModuleNode.body = StatListNode(h1d_wrapper.pyx:6:0)
 StatListNode.stats[19] = StatListNode(h1d_wrapper.pyx:358:0)
 StatListNode.stats[0] = DefNode(h1d_wrapper.pyx:358:0,
    modifiers = [...]/0,
    name = u'assemble_projection_matrix_rhs',
    num_required_args = 4,
    reqd_kw_flags_cname = '0')
 File 'Nodes.py', line 351, in analyse_expressions:
 StatListNode(h1d_wrapper.pyx:360:4)
 File 'Nodes.py', line 3346, in analyse_expressions:
 ExprStatNode(h1d_wrapper.pyx:369:43)
 File 'ExprNodes.py', line 322, in analyse_expressions:
 SimpleCallNode(h1d_wrapper.pyx:369:43,
    analysed = True,
    use_managed_ref = True)
 File 'ExprNodes.py', line 2865, in analyse_types:
 SimpleCallNode(h1d_wrapper.pyx:369:43,
    analysed = True,
    use_managed_ref = True)
 File 'ExprNodes.py', line 3947, in analyse_types:
 TupleNode(h1d_wrapper.pyx:369:43,
    is_sequence_constructor = 1,
    use_managed_ref = True)
 File 'ExprNodes.py', line 3742, in analyse_types:
 TupleNode(h1d_wrapper.pyx:369:43,
    is_sequence_constructor = 1,
    use_managed_ref = True)
 File 'ExprNodes.py', line 614, in coerce_to_pyobject:
 AttributeNode(h1d_wrapper.pyx:369:48,
    attribute = u'thisptr',
    is_attribute = 1,
    member = u'thisptr',
    needs_none_check = True,
    op = '-',
    use_managed_ref = True)
 File 'ExprNodes.py', line 3317, in coerce_to:
 AttributeNode(h1d_wrapper.pyx:369:48,
    attribute = u'thisptr',
    is_attribute = 1,
    member = u'thisptr',
    needs_none_check = True,
    op = '-',
    use_managed_ref = True)
 File 'ExprNodes.py', line 580, in coerce_to:
 AttributeNode(h1d_wrapper.pyx:369:48,
    attribute = u'thisptr',
    is_attribute = 1,
    member = u'thisptr',
    needs_none_check = True,
    op = '-',
    use_managed_ref = True)
 File 'ExprNodes.py', line 7076, in __init__:
 CoerceToPyTypeNode(h1d_wrapper.pyx:369:48,
    is_temp = 1,
    use_managed_ref = True)

 Compiler crash traceback from this point on:
  File /home/ondrej/usr/lib/python/Cython/Compiler/ExprNodes.py,
 line 7076, in __init__
    Cannot convert '%s' to Python object % arg.type)
  File /home/ondrej/usr/lib/python/Cython/Compiler/Errors.py, line
 151, in error
    report_error(err)
  File /home/ondrej/usr/lib/python/Cython/Compiler/Errors.py, line
 143, in report_error
    raise InternalError, abort
 InternalError: Internal compiler error: abort
 Traceback (most recent call last):
  File /home/ondrej/usr/bin/cython, line 8, in module
    main(command_line = 1)
  File /home/ondrej/usr/lib/python/Cython/Compiler/Main.py, line 773, in main
    result = compile(sources, options)
  File /home

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Thu, Dec 9, 2010 at 2:50 AM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2010/12/9 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Dec 9, 2010 at 2:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 09.12.2010 11:00:
 On Thu, Dec 9, 2010 at 1:46 AM, Stefan Behnel wrote:
 Vitja Makarov, 08.12.2010 22:45:
 Please review this patch. It's not yet finished, and mostly doesn't work.
 But you can take a look at patch and generated code.

 *Way* cool, thanks! I'll take a close look ASAP. Please feel free to send
 in a new patch when you have it. If you use hg, you can use hg bundle to
 collect commit series. Don't know about git.

 BTW, why not set up a github branch for this?

 That's exactly what I was thinking. It'll make it easier to review as
 well, and for others to contribute without disturbing the main line.

 BTW, when this goes in, the next Cython version to release is basically
 1.0, right? There are some minor Python compatibility issues left, but
 that'll be the last major feature, the way I count it.

 This will be the last major feature before 1.0, but I don't know that
 the first release this is in should be the 1.0 release.

 I also think we should set the compatibility bar a bit higher for 1.0
 as well--I want a concrete list of all the differences between Python
 X.y and Cython (for un-annotated code) and to be able to say that, if
 you don't count on the behavior in this list (e.g. identity vs
 equality for floating point literals, maybe stack frames) then you
 should be able to compile your code and have it just work, but
 hopefully faster. At the very least, until we at least attempt to make
 such a list, we won't know where we stand (the regression tests
 presumably being somewhat of a starting point, though I'd like to get
 all of them passing (unless they deal with implementation
 details...)).

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


   - Comprehensions are now handled by OldYieldExprNode

... another one of those bad names. ;)

Any reason you couldn't reuse it for both?

 Didn't yet implemented ;) GenexpNode should be replaced with DefFunc
 one that's easy but I want to fix refnanny first. Then fix refcount
 leaks.

   - I don't know how to make refnanny work, should refnanny context be
 stored in closure?

 Good call. That's a tricky question.

 What about generating a GIVEREF for every Python temp you store in the
 closure, a GOTREF for everything you take back out, and finish/restart the
 refnanny context around the yield?

 Yes, that would help. But the main problem is refnanny context
 setupcontext defines local variable __pyx_refnanny, we should have one
 more macro refnanny_reinitalize_context()

Yes, we'll need to split this up into two macros. This has actually
bitten be before, when I was writing the profiler. Did you know that ;
all by itself is considered a statement, hence one can't declare
variables after it in some strict compilers?

 BTW, why not set up a github branch for this?

 That's exactly what I was thinking. It'll make it easier to review as
 well, and for others to contribute without disturbing the main line.

 Should I fork cython from github?

Yes, please do.

 Is github insync with hg repo?

I've been manually keeping the two in sync--just pushed now.

 What is main repo now?

It's a DVCS :). Some people have been developing on the git side of
things, and some on the hg side of things, and I've been manually
syncing the two now and then. Obviously this isn't a long term
solution, and I think setting up an automated synchronization both
ways would cause more confusion than benefit, so as soon as the
buildbot starts looking at github I'll make the hg ones read-only.
(Probably worth waiting until after the release to do this.)

 ps: I think we should hurry with 1.0 release

Yes, I'd like to hurry too, and with all the stuff you're doing it
should be soon :). I just think it's important to be able to say
compiles all Python code rather than compiles most Python code and
have a (hopefully small) concrete lits of well-delineated caveats.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14

[Robert Bradshaw]

A beta is up at http://cython.org/release/Cython-0.14.beta1.tar.gz .
I'm planning on rolling a release candidate in the next day or two,
and if there are no issues, that will become the release.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Cython 0.14

[Robert Bradshaw]

On Thu, Dec 9, 2010 at 4:27 AM, Robert Bradshaw
rober...@math.washington.edu wrote:
 A beta is up at http://cython.org/release/Cython-0.14.beta1.tar.gz .
 I'm planning on rolling a release candidate in the next day or two,
 and if there are no issues, that will become the release.

Make that http://cython.org/release/Cython-0.14.beta2.tar.gz
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] Complex to float coercion

[Robert Bradshaw]

On Thu, Dec 9, 2010 at 5:58 AM, Dag Sverre Seljebotn
da...@student.matnat.uio.no wrote:
 In the ongoing fwrapification of SciPy [1], I'm getting regression test
 breakage because of this:

 def func(float x):
     return x

   func(1 + 0j)
     Traceback (most recent call last):
     ...
     TypeError: can't convert complex to float


 I'm wondering: Would people be OK to Cython so that when the imaginary
 part is *exactly* 0, it is OK to convert to native float? Or is failing
 hard the preferred behaviour?

 (For all I know, SciPy may prefer to change their behaviour; so I'm just
 doing a poll for now.)

 I guess the way to do it would be to try float conversion first, and if
 that fails, try complex conversion and see if the imaginary part ends up
 as 0. That is, it would mean that __complex__ would be called on passed
 in objects that do not support __float__ in the example above.

I'm -1 on making this change. If the author of the user class defined
a __complex__ but not a __float__, we should respect that. Not to
mention the added verbosity to the object - double path.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

On Thu, Dec 9, 2010 at 12:45 PM, Vitja Makarov vitja.maka...@gmail.com wrote:
 2010/12/9 Vitja Makarov vitja.maka...@gmail.com:
 2010/12/9 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Dec 9, 2010 at 2:50 AM, Vitja Makarov vitja.maka...@gmail.com 
 wrote:
 2010/12/9 Robert Bradshaw rober...@math.washington.edu:
 On Thu, Dec 9, 2010 at 2:10 AM, Stefan Behnel stefan...@behnel.de wrote:
 Robert Bradshaw, 09.12.2010 11:00:
 On Thu, Dec 9, 2010 at 1:46 AM, Stefan Behnel wrote:
 Vitja Makarov, 08.12.2010 22:45:
 Please review this patch. It's not yet finished, and mostly doesn't 
 work.
 But you can take a look at patch and generated code.

 *Way* cool, thanks! I'll take a close look ASAP. Please feel free to 
 send
 in a new patch when you have it. If you use hg, you can use hg 
 bundle to
 collect commit series. Don't know about git.

 BTW, why not set up a github branch for this?

 That's exactly what I was thinking. It'll make it easier to review as
 well, and for others to contribute without disturbing the main line.

 BTW, when this goes in, the next Cython version to release is basically
 1.0, right? There are some minor Python compatibility issues left, but
 that'll be the last major feature, the way I count it.

 This will be the last major feature before 1.0, but I don't know that
 the first release this is in should be the 1.0 release.

 I also think we should set the compatibility bar a bit higher for 1.0
 as well--I want a concrete list of all the differences between Python
 X.y and Cython (for un-annotated code) and to be able to say that, if
 you don't count on the behavior in this list (e.g. identity vs
 equality for floating point literals, maybe stack frames) then you
 should be able to compile your code and have it just work, but
 hopefully faster. At the very least, until we at least attempt to make
 such a list, we won't know where we stand (the regression tests
 presumably being somewhat of a starting point, though I'd like to get
 all of them passing (unless they deal with implementation
 details...)).

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


   - Comprehensions are now handled by OldYieldExprNode

... another one of those bad names. ;)

Any reason you couldn't reuse it for both?

 Didn't yet implemented ;) GenexpNode should be replaced with DefFunc
 one that's easy but I want to fix refnanny first. Then fix refcount
 leaks.

   - I don't know how to make refnanny work, should refnanny context be
 stored in closure?

 Good call. That's a tricky question.

 What about generating a GIVEREF for every Python temp you store in the
 closure, a GOTREF for everything you take back out, and finish/restart the
 refnanny context around the yield?

 Yes, that would help. But the main problem is refnanny context
 setupcontext defines local variable __pyx_refnanny, we should have one
 more macro refnanny_reinitalize_context()

 Yes, we'll need to split this up into two macros. This has actually
 bitten be before, when I was writing the profiler. Did you know that ;
 all by itself is considered a statement, hence one can't declare
 variables after it in some strict compilers?

 BTW, why not set up a github branch for this?

 That's exactly what I was thinking. It'll make it easier to review as
 well, and for others to contribute without disturbing the main line.

 Should I fork cython from github?

 Yes, please do.

 Is github insync with hg repo?

 I've been manually keeping the two in sync--just pushed now.

 What is main repo now?

 It's a DVCS :). Some people have been developing on the git side of
 things, and some on the hg side of things, and I've been manually
 syncing the two now and then. Obviously this isn't a long term
 solution, and I think setting up an automated synchronization both
 ways would cause more confusion than benefit, so as soon as the
 buildbot starts looking at github I'll make the hg ones read-only.
 (Probably worth waiting until after the release to do this.)

 ps: I think we should hurry with 1.0 release

 Yes, I'd like to hurry too, and with all the stuff you're doing it
 should be soon :). I just think it's important to be able to say
 compiles all Python code rather than compiles most Python code and
 have a (hopefully small) concrete lits of well-delineated caveats.

 - Robert
 ___
 Cython-dev mailing list
 Cython-dev@codespeak.net
 http://codespeak.net/mailman/listinfo/cython-dev


 I've created github repo https://github.com/vitek/cython/

 And changed __Pyx_Generator to __CyGenerator...

 --
 vitja.


 I've fixed refnanny. Now regr are okay:

 --
 Ran 4058 tests in 1520.368s

 FAILED (failures=18, errors=611, skipped=4)

Cool.
___
Cython-dev mailing list
Cython-dev@codespeak.net
http

Re: [Cython] Cython 0.13.1

[Robert Bradshaw]

On Sun, Dec 5, 2010 at 7:55 AM, Stefan Behnel stefan...@behnel.de wrote:
 mark florisson, 05.12.2010 15:11:
 On 5 December 2010 14:15, Stefan Behnel wrote:
 I haven't tested it yet, mainly because I don't have the latest gdb version
 installed. When I tried building lxml with that branch, I ran into a couple
 of compiler crashes that Mark resolved, but I can't tell if there aren't
 any left. I also didn't review the code in any way.

 AFAIC, I'm not sure I want this right in 0.14, but that's more because I
 can't really comment on it. I certainly agree that it needs more testing.
 [...]
 But I agree that it should be tested more widely before merging it. The
 documentation (currently only on hudson but I presume it will make it
 to docs.cython.org with the new release) already contains a link to my
 git branch so users that are interested could still use it.

 Then I would suggest we keep the docs as they are now to make your gdb
 support visible, but with a big note that this hasn't been merged yet and
 keep your branch alive for a bit longer after the release. I wouldn't mind
 merging it for 0.14.1 when it seems mature enough, and I hope that release
 isn't too far off. 0.14 is a release with many, many new features, so it's
 quite likely that people find bugs quickly once it's out in the wild.

OTOH, when the flag is off, the changes made are quite noninvasive, so
I would be fine merging this as an experimental feature to get more
widespread testing and for what utility it already has, even if it's
not fully polished yet (not that I have any bugs to point at...).

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] make cython stop on the first error

[Robert Bradshaw]

On Sun, Dec 5, 2010 at 9:47 PM, Stefan Behnel stefan...@behnel.de wrote:
 Ondrej Certik, 06.12.2010 02:42:
 Maybe we should create a StopOnFirstError
 exception, and then catch it and print exiting the compilation due to
 --fatal-errors command line switch?

 +1, although I'd name it AbortCompilation or something like that, maybe
 SilentlyAbortCompilation.

We should trigger the exact same kind of abort that happens between
(some) pipeline phases if there are any errors. This will trigger the
necessary cleanup (e.g. a castrated C file), etc. I thought that's
what I did, but I guess I only tested running cython manually (which
didn't give a nasty traceback).

While I'm thinking about this, I'd rather the flag be named
--fast-fail, as errors are always fatal, it's just a question of
timing.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev

Re: [Cython] General generator expressions

[Robert Bradshaw]

First off, great to hear you're working on this. I've wanted to do it
for a while (well, essentially ever since I got closures done), but
don't know when I'll find the time.


On Mon, Dec 6, 2010 at 12:56 AM, Vitja Makarov vitja.maka...@gmail.com wrote:

 def my_generator_body(closure, value, is_exception):

 Shouldn't that be a cdef function?

 Guess no. It has very special signature and should be used only in
 generator closure.

 It can't be cdef as cdef doesn't parse (args, kwargs) and generator
 does this on first run.

Eventually the argument parsing code should be abstracted out so that
we don't have this restriction.

       # switch case
       switch (closure.resume_lable):
           case 0:
                  goto parse_arguments;
           case 1:
                  goto resume_from_yield1;
       parse_arguments:
       # function body here

 Ok, sure, as in the CEP.


 About temps:
 now temps are allocated in closure using declare_var() and it works fine.

 I think that could seriously slow down things inside of the generator,
 though. It means that it needs to jump through the closure indirection for
 every little value that it keeps around temporarily, including lots of
 intermediate C results. Also, I would expect that a lot more temps are used
 inside of the generator body than what actually needs to be kept alive
 during yields, so a single yield inside of a lengthy generator body could
 have a tremendous impact on the overall runtime and closure size.

 If you only store away the live temp values before a yield, you don't even
 need to do any reference counting for them. It's just a quick bunch of C
 assignments on yield and resume. That's about the smallest impact you can 
 get.

 BTW, if you change the object struct of the generator closure anyway, maybe
 a dynamically sized PyVarObject would be a way to incorporate the temps?



 I don't understand why do you think about performance problems?

 It seems to me that if temps are stored in tuple (or another kind of
 object) you have to manully save/restore them, and this code will
 really slow down generator.

-1 to storing them all in a tuple. That wouldn't handle C values very
well, for one thing. I wouldn't use arrays though--either declare each
temp individually or create a custom struct that has the correct slot
for each temp. Copy to locals on enter, copy out on
yield/exception/return.

 On the other hand when all temps are declared in closure you just have
 to say closure-pyx_temp_v1

 Do you think that closure- will be much slower then stack operations?

Yes, the performance difference can be dramatic. This is one of the
reasons good numpy support is tricky to get right--the compiler can
optimize stuff on the stack like crazy, but the same can't be said of
indirect references. I'm fine with sticking all temps in the closure
for now if that gets us up and running faster, and optimizing things
later.

 I'm not sure about manage_ref flag I guess that temps with
 manage_ref=False shouldn't be allocated on closure (am I right?)

 manage_ref=False (for a Python object value) just means that the
 reference is either borrowed or will be stolen at some point. I'm not sure
 that stealing the reference may not happen *after* a yield, but I guess
 that would produce other problems if the reference isn't kept alive in
 another way. Nothing to worry about now, carefully crafted tests will
 eventually catch that.

 In any case, all live temps (Python refs, borrowed refs and C values) may
 be needed after the yield, so they must *all* end up in the closure.

And of course we need to make all live reference are properly
deallocated when the closure is GC'd as well.

- Robert
___
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev