[Caml-list] help with ocaml makefiles
Each time I need more debug information for the ocaml runtime itself (which hopefully is not often) I recompile Ocaml adding various -g and -O0 here and there in some makefiles. But I've noticed there are some special targets and rules, specifically in byterun/Makefile, that seams to be there for that very purpose (the suffix rules for .d.c extention). So I suppose there is a standard way to compile ocaml in debugging mode. Do someone have some knowledge to share about this ? ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Native toplevel?
Alain Frisch al...@frisch.fr writes: On 11/16/2010 03:52 PM, Benedikt Meurer wrote: A further step to improve this native toplevel is to avoid the call to the external assembler and linker. To do that, one basically needs to replace the assembly code emitters (emit.mlp/emit_nt.mlp) with native code emitters and do the relocation/dynamic loading by hand, which is quite easy. Maybe this could possibly help with the emission part (at least for X86) (regarding OCamlJIT as well): https://github.com/danmey/ocaml-inline-x86/ The project is `not yet there', however I am currently putting some effort to make it more usable. If anybody is interested please shout. -- Alain Wojciech ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] SMP multithreading
Yes, actually. :P On Wed, Nov 17, 2010 at 11:15 PM, Jon Harrop jonathandeanhar...@googlemail.com wrote: Can you cite any papers from this century? ;-) Cheers, Jon. *From:* Eray Ozkural [mailto:examach...@gmail.com] *Sent:* 17 November 2010 13:41 *To:* Eray Ozkural; Jon Harrop; caml-list@yquem.inria.fr *Subject:* Re: [Caml-list] SMP multithreading On Wed, Nov 17, 2010 at 8:50 AM, Gabriel Kerneis kern...@pps.jussieu.fr wrote: On Wed, Nov 17, 2010 at 06:27:14AM +0200, Eray Ozkural wrote: As I said even in C good results can be achieved, I've seen that, so I know it's doable with ocaml, just a difficult kind of compiler. The functional features would expose more concurrency. Could you share a pointer to a paper describing this compiler? I can't reveal much, but just to point out that there are indeed more sophisticated compilers than gcc: http://www.research.ibm.com/vliw/compiler.html So, uh, there are compilers that turn loops into threads, and also parallelize independent blocks Both coarse-grain and fine-grain parallelization strategies in existing compiler research can be effectively applied to the multi-core architectures. In fact, some of the more advanced compilers (like that of the RAW architecture) must be able to target it already, but who knows. :) Just consider that most of the parallelization technology is language independent, they can be applied to any imperative language. So, would such a thing be able to work on ocaml generated binaries? Most definitely, I believe, it is in principle possible to start from the sequential binary and emit parallel code! Best, -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
RE: [Caml-list] SMP multithreading
Edgar Friendly wrote: It looks like high-performance computing of the near future will be built out of many machines (message passing), each with many cores (SMP). One could use message passing for all communication in such a system, but a hybrid approach might be best for this architecture, with use of shared memory within each box and message passing between. Of course the best choice depends strongly on the particular task. Absolutely - and the problem in OCaml seems to be that shared memory parallelism is just branded as evil and ignored... In the long run, it'll likely be a combination of a few large, powerful cores (Intel-CPU style w/ the capability to run a single thread as fast as possible) with many many smaller compute engines (GPGPUs or the like, optimized for power and area, closely coupled with memory) that provides the highest performance density. I think the central thing that we can be utterly sure about is that desktops will always have * 1* general purpose CPU. Maybe not be an ever-increasing number of cores but definitely more than one. The question of how to program such an architecture seems as if it's being answered without the functional community's input. What can we contribute? It has often seemed to me when SMP has been discussed in the past on this list that it almost gets dismissed out of hand because it doesn't look future-proof or because we're worried about what's round the corner in technology terms. To me the principal question is not about whether a parallel/thread-safe GC will scale to 12, 16 or even the 2048 cores on something like http://www.hpc.cam.ac.uk/services/darwin.html but whether it will hurt a single-threaded application - i.e. whether you will still be able to implement message passing libraries and other scalable techniques without the parallel GC getting in the way of what you're doing. A parallel/thread-safe GC should be aiming to provide the same sort of contract as the present one - it just works for most things and in a few borderline cases (like HPC - yes, it's a borderline case) you'll need to tune your code or tweak GC parameters because it's causing some problems or because in your particular application squeezing every cycle out of the CPU is important. As long as the GC isn't (hugely) slower than the present one in OCaml then we can continue to use libraries, frameworks and technologies-still-to-come on top of a parallel/thread-safe GC which simply ignores shared memory thread-level parallelism just by not instantiating threads. The argument always seems to focus on utterly maxing out all possible available resources (CPU time, memory bandwidth, etc.) rather than on whether it's simply faster than what we're doing able to do at the moment on the same system. Of course, it may be that the only way to do that is to have different garbage collectors - one invoked when threads.cmxa is linked and then the normal one otherwise (that's so easy to type out as a sentence, summarising a vast amount of potential work!!) Multithreading in OCaml seems to be focused on jumping the entire width of the river of concurrency in one go, rather than coming up with stepping stones to cross it in bits... David ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] SMP multithreading
Hello, And OCaml on GPU ? We just tested a recent GPU card with 480 processors at 900Mhz ... this is qui impressive ... and supported by matlab via cuda-lapack (http://www.culatools.com/) ... I imagine we could at least use cuda-lapack from OCaml ? Cheers, Christophe signature.asc Description: OpenPGP digital signature ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
RE: [Caml-list] SMP multithreading
Can you cite any papers from this century? ;-) Cheers, Jon. From: Eray Ozkural [mailto:examach...@gmail.com] Sent: 17 November 2010 13:41 To: Eray Ozkural; Jon Harrop; caml-list@yquem.inria.fr Subject: Re: [Caml-list] SMP multithreading On Wed, Nov 17, 2010 at 8:50 AM, Gabriel Kerneis kern...@pps.jussieu.fr wrote: On Wed, Nov 17, 2010 at 06:27:14AM +0200, Eray Ozkural wrote: As I said even in C good results can be achieved, I've seen that, so I know it's doable with ocaml, just a difficult kind of compiler. The functional features would expose more concurrency. Could you share a pointer to a paper describing this compiler? I can't reveal much, but just to point out that there are indeed more sophisticated compilers than gcc: http://www.research.ibm.com/vliw/compiler.html So, uh, there are compilers that turn loops into threads, and also parallelize independent blocks Both coarse-grain and fine-grain parallelization strategies in existing compiler research can be effectively applied to the multi-core architectures. In fact, some of the more advanced compilers (like that of the RAW architecture) must be able to target it already, but who knows. :) Just consider that most of the parallelization technology is language independent, they can be applied to any imperative language. So, would such a thing be able to work on ocaml generated binaries? Most definitely, I believe, it is in principle possible to start from the sequential binary and emit parallel code! Best, -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] SMP multithreading
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1650134 This is one of the more recent papers a quick search turns up, but you have to keep in mind that thread extraction is only one problem among many for a parallelizing compiler. I think the keyword you are looking for is thread extraction. And here probably, it's the simplest kind of extraction... Food for some thought: assume that you have a very good compiler pass that extracts all possible threads in the sequential code, can you name any other problems the compiler must solve to achieve good performance? I can't talk at all about the project I worked on, but as I mentioned previously, familiarize yourself with the RAW project, it was similar in some respects to the project I worked in: http://groups.csail.mit.edu/cag/raw/ This should, at least a bit, dispel the illusion that parallelizing compilers are helpless when they confront C code. Reading the OS/400 book had opened my mind about OS design, perhaps reading about recent computer architecture research projects will open others' eyes about compilers, and how useful they really can be! Also, I believe there ought to be some textbooks about multi-core architectures and relevant compilation strategies, let me post it if I find a comprehensive reference. The dream compiler would have all the cool linear algebra capabilities of HPF + the more general/free-form kinds of parallelization strategies in recent compilers. Ok, so what you really want to do is, parallelize applications that can benefit from them. Not file utils or web browsers. If you are so curious, stuff like povray would be in the test suite. Sometimes the parallelizing compiler parallelizes computations that a programmer wouldn't bother due to program complexity, here a basic block, there a basic block, some pipelining communication/computation overlap there I think it's a safe bet to say that, with all the general lameness surrounding parallel programming languages, parallelizing compilers will be very important in the near future. Cheers, On Thu, Nov 18, 2010 at 2:28 AM, Eray Ozkural examach...@gmail.com wrote: Yes, actually. :P On Wed, Nov 17, 2010 at 11:15 PM, Jon Harrop jonathandeanhar...@googlemail.com wrote: Can you cite any papers from this century? ;-) Cheers, Jon. *From:* Eray Ozkural [mailto:examach...@gmail.com] *Sent:* 17 November 2010 13:41 *To:* Eray Ozkural; Jon Harrop; caml-list@yquem.inria.fr *Subject:* Re: [Caml-list] SMP multithreading On Wed, Nov 17, 2010 at 8:50 AM, Gabriel Kerneis kern...@pps.jussieu.fr wrote: On Wed, Nov 17, 2010 at 06:27:14AM +0200, Eray Ozkural wrote: As I said even in C good results can be achieved, I've seen that, so I know it's doable with ocaml, just a difficult kind of compiler. The functional features would expose more concurrency. Could you share a pointer to a paper describing this compiler? I can't reveal much, but just to point out that there are indeed more sophisticated compilers than gcc: http://www.research.ibm.com/vliw/compiler.html So, uh, there are compilers that turn loops into threads, and also parallelize independent blocks Both coarse-grain and fine-grain parallelization strategies in existing compiler research can be effectively applied to the multi-core architectures. In fact, some of the more advanced compilers (like that of the RAW architecture) must be able to target it already, but who knows. :) Just consider that most of the parallelization technology is language independent, they can be applied to any imperative language. So, would such a thing be able to work on ocaml generated binaries? Most definitely, I believe, it is in principle possible to start from the sequential binary and emit parallel code! Best, -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] a syntax error that I don't get
Hello caml'ers This one is short, I'm writting a signal handler: 205: let sig_exit () = 206:Unix.kill !chpid Sys.sigterm ; 207:match (Unix.waitpid [] !chpid) with 208:| pid, Unix.WSIGNALED (Sys.sigterm) - log `LOG_NOTICE child terminated by parent 209:| pid, pstat - log `LOG_ERR child asked to terminate by parent, but unexpected process status received and compilation chokes on: phi...@air:~/mysrc/deconvolution/jackond$ make ocamlfind opt -thread -package unix,threads,netsys,netstring,netplex,rpc,equeue,jack,str,syslog -c patcher.ml -o patcher.cmx File patcher.ml, line 208, characters 28-35: Error: Syntax error make: *** [patcher.cmx] Error 2 replacing Sys.sigterm by -11 or signumber, it compiles fine. I don't get it. It's using ocaml 3.12 of godi, on osx 10.6.5, x86_64 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] ocamlbuild and packs
Hi all, I've got a source tree with the following patterns : A/a.ml (defines 'let x = 1') B/a.ml (use A.x and defines 'let y = 2') B/b.ml B/b.mlpack (contains 'A B') C/a.ml (use A.x) C/b.ml (use B.A.y) C/c.mlpack (contains 'A B') Is there any way with ocamlbuild to build that tree (if possible using only _tags files, but if not I would be happy to have at least a solution :-) ? Cheers, Thomas ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] ocamlbuild and packs
Hi all, I've got a source tree with the following patterns : A/a.ml (defines 'let x = 1') B/a.ml (use A.x and defines 'let y = 2') B/b.ml B/b.mlpack (contains 'A B') C/a.ml (use A.x) C/b.ml (use B.A.y) C/c.mlpack (contains 'A B') Is there any way with ocamlbuild to build that tree (if possible using only _tags files, but if not I would be happy to have at least a solution :-) ? Cheers, Thomas PS: apologies if you receive this email twice ... ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] Optimizing garbage collection
A program I wrote constructs a lot of small lists, and strings and discards them. It's a search algorithm. I profiled this code and saw that garbage collection takes significant time. In C++, we can write custom allocators to optimize the data structures that cause such slowdowns. Any recommended strategies in ocaml? Best, Call graph in gprof output on linux: granularity: each sample hit covers 2 byte(s) for 0.00% of 8680.47 seconds index % timeself childrencalled name 0.001.313777/13406323 caml_alloc [139] 0.002.457076/13406323 caml_alloc_small [94] 0.006.42 18532/13406323 caml_copy_double [119] 0.05 405.30 1169992/13406323 caml_alloc_string [16] 0.10 721.02 2081381/13406323 caml_check_urgent_gc [27] 0.48 3507.65 10125565/13406323 caml_garbage_collection [3] [1] 53.50.63 4644.15 13406323 caml_minor_collection [1] 2.13 2625.29 13406323/13406323 caml_major_collection_slice [5] 2.25 2014.36 26812646/26812646 caml_empty_minor_heap [6] 0.130.00 13406323/13406323 caml_final_do_calls [308] --- spontaneous [2] 40.40.66 3508.19 caml_call_gc [2] 0.03 3508.16 10125565/10125565 caml_garbage_collection [3] --- 0.03 3508.16 10125565/10125565 caml_call_gc [2] [3] 40.40.03 3508.16 10125565 caml_garbage_collection [3] 0.48 3507.65 10125565/13406323 caml_minor_collection [1] 0.030.00 10125565/10125627 caml_process_pending_signals [429] --- -- Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara http://groups.yahoo.com/group/ai-philosophy http://myspace.com/arizanesil http://myspace.com/malfunct ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Native toplevel? (was: OCamlJit 2.0)
On Wed, Nov 17, 2010 at 3:44 AM, Alain Frisch al...@frisch.fr wrote: Does performance really matter that much for rapid prototyping/development? Rapid prototyping for me often involves a couple of lines of code that read in a very large file and do something with it. I have to keep compiling these small programs to native code because the performance of the toplevel is too slow. Then, I have to recompile and re-read the whole file for every little additional thing I want to compute. A high-performance toplevel would help in this kind of work. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] Looking for stubs for sendmsg/recvmsg
Hi, I'm looking for stubs for ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags); ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags); Specifically I need those to send (among normal messages) an Unix.file_descr over a Unix Domain Socket. Does anyone know of a module that has them? MfG Goswin ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] Re: Native toplevel? (was: OCamlJit 2.0)
On Nov 17, 2010, at 09:44 , Alain Frisch wrote: There is actually already a native top-level in the distribution, even though it is undocumented and unmaintained. You can build it with the make ocamlnat target. The implementation is based on the same approach as native dynlink. The toplevel embeds the native compiler; for each phrase, the toplevel produces assembly code, calls the assembler and the linker to produce a dynamic/shared library, and then dynamically load and execute the resulting code. This gives some latency, but it's not so bad in practice, and you get the full speed of native code. A further step to improve this native toplevel is to avoid the call to the external assembler and linker. To do that, one basically needs to replace the assembly code emitters (emit.mlp/emit_nt.mlp) with native code emitters and do the relocation/dynamic loading by hand, which is quite easy. As it turns out, LexiFi uses on a daily basis such direct binary code emitters for x86/amd64 on Windows, and that would be easy to port to other x86/amd64 platforms. The x86 version was developed internally, and the amd64 version was done by Fabrice Le Fessant. There is also some code to wrap the binary code into COFF objects (and flexdll has a stand-alone mode to produce .cmxs files without an external linker), but that would be useless for a native toplevel. The goal was to have a compiler which can be more easily embedded in our applications and deployed to our customers, without depending on any external tool. If you Benedikt, or someone else, is willing to work on a native top-level without the need to call an external assembler/linker, we are ready to extract the binary code emitters from our code base and share them with the community (with an open-source license to be determined). This requires some packaging work on our side, so we're going to do it only if there is interest in the project. This is indeed very interesting. I haven't thought of the native top-level. I haven't done any measurements yet, but from my experience with ocamlopt, I know that the optimizing native compiler is somewhat slower than the byte-code compiler. I doubt that this is related solely to the external as/ld invocations, so there's certainly more work to do than just replacing the calls to as/ld with an in-process assembler/linker. -- Alain Benedikt ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
