On Mon, Jun 23, 2008 at 9:40 AM, Brian Granger <[EMAIL PROTECTED]> wrote:
> Here is my two cents on this... > > Something like OpenMP has already been tried in Python, although it > really hasn't made it out into the wild. Here is the link: > > http://www.cs.utk.edu/~luszczek/pubs/plw-200605.pdf<http://www.cs.utk.edu/%7Eluszczek/pubs/plw-200605.pdf> I have been in touch with the author. The implemenation was never complete. PLW project never actually implemented full parallel annotations. They were proposed but not implemented. > > > Using markup or directives has never really become a very popular way > of handling parallelism. I am not sure why, but OpenMP has never > really caught on like MPI even though MPI is more painful for many > things. It is becoming more widespread now. Every single system/chip/compiler vendor I know of is pushing this. The advent of multicores is a big reason for this since parallelism is now moving into the mainstream from HPC. One other reason why OpenMP hasnt caught on is that compilers have taken many years to mature. GCC for example had no support for OpenMP till very recently. For OpenMP, its still usually better to use vendor compilers than GCC. In the Python world, MPI has been popular. Because, as it happens, Python offers you NO WAY to handle parallelism any other way except a process based way. > I think it would be much better to use the language (Python) itself to > handle these things as the language is already extremely powerful. > But, I think the actual parallel constructs should be kept out of > Cython proper. In general I think it is much better to develop > libraries, not new language syntax. Obviously, Cython itself is a > distinct language, but I think considerable effort must be made to > prevent feature creep. I am not against new features, but it is easy > to get carried away with these things. It is much more important to > make the existing features and syntax more robust. This is why most > successful languages develop in a rather conservative manner. I would > suggest a good way of evaluating new ideas is to ask to question "what > would Guido say?" Of course, Cython is not Python, but we all hope > the two don't stray far from each other. What will Guido say is the wrong thing to ask in this case. This is not a question of language syntax. What we are discussing is not new syntax sugar over some existing facility. What we are discussing is a system facility. Tell me : is Guido against removing GIL? He is not. He only says that removing the GIL is a practical problem in that it not-so-easy to modify the CPython source base. Is Guido against threads offering real concurrency? He is not or he would not like Jython. Guido only says : a) GIL is a practical problem b) For many (but not all) applications process based parallelism is sufficient. Thus it does not give HIM and the rest of the cpython developers enough incentive to put work into removing GIL when he has more pressing issues. He is however happy if someone else does it. Let me stress once again. WE ARE NOT DISCSSING SYNTAX. We are discussing a system facility. And as it happens, this system facility cannot be handled any other way except either a) writing a compiler that can produce parallel C code b) writing a modified CPython. c) writing a JIT that can produce parallel code at runtime. similar to a. As we dont have the option b or c, we have to go through the route a. If you simply dont care about the system facility being provided (loop based parallelism), thats a different issue. > Lots of projects have begun using Cython lately, which is really > great. This makes it more important than ever to focus on the core > language rather than wild new features. Since I am not a Cython developer, I will of course not comment on this. I will however point out that I am also writing a compiler called unPython which focuses a lot on numpy and I am 100% sure that I am going to add a parallel loop. Even if the syntax I add turns out ugly, thats fine with me as long as people get to write parallel code. Even a slightly ugly Python syntax is a lot better than writing manually C code. Thus I propose that I am going ahead with my implementation, for now based on prange. This way I can also report on how easy or hard it is to implement. Cython developers can have discussions about whether they want to add or not. Since unPython is very alpha, and isnt used very much, so I have liberty of changing syntax later if by consensus some other syntax is found to be better. So in summary, I am going ahead right now and will come back and report in a month or so. thanks, rahul
_______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
