RE: [Caml-list] Native toplevel? (was: OCamlJit 2.0)

2010-11-20 Thread Jon Harrop
Also for interactive data massaging and visualization.

 

From: caml-list-boun...@yquem.inria.fr
[mailto:caml-list-boun...@yquem.inria.fr] On Behalf Of Ashish Agarwal
Sent: 18 November 2010 16:50
To: Alain Frisch
Cc: caml-list@yquem.inria.fr
Subject: Re: [Caml-list] Native toplevel? (was: OCamlJit 2.0)

 

On Wed, Nov 17, 2010 at 3:44 AM, Alain Frisch al...@frisch.fr wrote:

Does performance really matter that much for rapid prototyping/development?

 

Rapid prototyping for me often involves a couple of lines of code that read
in a very large file and do something with it. I have to keep compiling
these small programs to native code because the performance of the toplevel
is too slow. Then, I have to recompile and re-read the whole file for every
little additional thing I want to compute. A high-performance toplevel would
help in this kind of work.

 

___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


RE: [Caml-list] SMP multithreading

2010-11-20 Thread Jon Harrop
 This is actually a quick way to use multiple cores with ocaml. Find a
 often called function that takes considerable time and offload it to C

Or HLVM, F#, Scala, Clojure or any of the other languages that permit shared
memory parallelism. C is particularly poor in this regard so I would not
just restrict yourself to C...

Cheers,
Jon.


___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


[Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Vincent Balat
On Nov 19, 2010 21:20:01, Yoann Padioleau wrote:
 On Nov 19, 2010, at 11:46 AM, Dario Teixeira wrote:
...
  Actually, Facebook has a compiler that transforms PHP source code into
  C++ [1], and they claim a 50% reduction in CPU usage.
 
 Yes, which is good. But if you think about it is only a x2 speedup vs a
 really slow bytecode interpreter (the Zend PHP interpreter). PHP is known
 for being more than 30 times slower than C.
 It's even slower than Ruby on
 http://shootout.alioth.debian.org/u32/which-programming-languages-are-fast
 est.php There are lots of opportunities to do better IMHO.

Would it be completely inconceivable for a company like facebook to 
reimplement everything using a fast well designed typed language
instead of spending hundreds of millions of dollars on machines that run PHP 
bytecode interpreters? (quoting Yoann)

Vincent [not completely joking]

___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Benedikt Meurer

On Nov 20, 2010, at 16:19 , Vincent Balat wrote:

 On Nov 19, 2010 21:20:01, Yoann Padioleau wrote:
 On Nov 19, 2010, at 11:46 AM, Dario Teixeira wrote:
 ...
 Actually, Facebook has a compiler that transforms PHP source code into
 C++ [1], and they claim a 50% reduction in CPU usage.
 
 Yes, which is good. But if you think about it is only a x2 speedup vs a
 really slow bytecode interpreter (the Zend PHP interpreter). PHP is known
 for being more than 30 times slower than C.
 It's even slower than Ruby on
 http://shootout.alioth.debian.org/u32/which-programming-languages-are-fast
 est.php There are lots of opportunities to do better IMHO.
 
 Would it be completely inconceivable for a company like facebook to 
 reimplement everything using a fast well designed typed language
 instead of spending hundreds of millions of dollars on machines that run PHP 
 bytecode interpreters? (quoting Yoann)

It's probably not a technical decision, but more likely a marketing decision. 
If you tell Joe that your webservices run on Java, PHP or .NET, he'll say 
great, sure or wow (not because Joe's familiar with the technology or the 
theory, but because he's familiar with the terms). Tell Joe your webservices 
run on OCaml or Haskell and the best answer you can get will be what?.

 Vincent [not completely joking]

Benedikt
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] OCamlJit 2.0

2010-11-20 Thread Yoann Padioleau

On Nov 19, 2010, at 11:10 AM, Benedikt Meurer wrote:

 
 Well, it is a research project, and it was driven by actual demand. A JIT 
 engine for PHP is something less interesting from a university point of view, 
 unless there are companies willing to sponsor/help the development.
 
 But from my personal experience, there is not really a lot to gain w.r.t. 
 PHP. Delivering website content does not involve complex computations or 
 processing, it is mostly I/O bound, depending on a fast database engine, a 
 fast webserver, decent text processing throughput, etc. I may be wrong here,

I think you are. As said by someone previously facebook got a 2x speedup on CPU 
by optimizing PHP. Websites are not just
getting something from a database and echoing it.

 but I doubt that you'd see relevant speedups on large websites by simply 
 JITting the PHP code.
 
 Also PHP code is less likely to change at runtime, so there's no real need to 
 acutally JIT compile it. You could use a lot simpler techniques here to 
 improve performance. For example, just write a simple PHP to C compiler, 
 compiling your PHP code to native code via C, and let the webserver run the 
 native code instead. With some clever compilation scheme, this should 
 outperform any JIT engine, with a lot less effort.

I think it may actually be more effort to do an efficient compiler than a JIT 
on languages like PHP
For instance the team behind the Druby project ( 
http://www.cs.umd.edu/projects/PL/druby/publications.html )
first tried to do some static type inference on ruby code but then switched to 
a dynamic approach
because it was too hard to infer statically.

 
 Anyway, this seems to be off-topic here...

Sure

 
 Benedikt
 ___
 Caml-list mailing list. Subscription management:
 http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
 Archives: http://caml.inria.fr
 Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
 Bug reports: http://caml.inria.fr/bin/caml-bugs

___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Yoann Padioleau

On Nov 20, 2010, at 7:42 AM, Benedikt Meurer wrote:

 
 On Nov 20, 2010, at 16:19 , Vincent Balat wrote:
 
 On Nov 19, 2010 21:20:01, Yoann Padioleau wrote:
 On Nov 19, 2010, at 11:46 AM, Dario Teixeira wrote:
 ...
 Actually, Facebook has a compiler that transforms PHP source code into
 C++ [1], and they claim a 50% reduction in CPU usage.
 
 Yes, which is good. But if you think about it is only a x2 speedup vs a
 really slow bytecode interpreter (the Zend PHP interpreter). PHP is known
 for being more than 30 times slower than C.
 It's even slower than Ruby on
 http://shootout.alioth.debian.org/u32/which-programming-languages-are-fast
 est.php There are lots of opportunities to do better IMHO.
 
 Would it be completely inconceivable for a company like facebook to 
 reimplement everything using a fast well designed typed language

I think it is inconceivable. I doubt facebook will switch to ocaml and ocsigen 
tomorrow :)
The problem is how to migrate code to another language smoothly ? You can not
stop all development for a month and say Hey everybody, we are porting
our 10 millions lines of code of PHP to X.. The reason C++ succeeded was
because there was a smooth migration path. C code is valid C++ code. You
can incrementally add objects to an existing codebase.

Do we have example of big companies porting their whole codebase to another 
language ?


 instead of spending hundreds of millions of dollars on machines that run 
 PHP 
 bytecode interpreters? (quoting Yoann)

Apparently they decided to keep PHP and switch from a slow bytecode interpreter
to a 2x-faster compiler.

 
 It's probably not a technical decision, but more likely a marketing decision. 
 If you tell Joe

Who is Joe ? A developer ? A user ? A venture capitalist ?

 that your webservices run on Java, PHP or .NET, he'll say great, sure or 
 wow (not because Joe's familiar with the technology or the theory, but 
 because he's familiar with the terms). Tell Joe your webservices run on OCaml 
 or Haskell and the best answer you can get will be what?.

I doubt any user care about how facebook is implemented. Twitter and Foursquare 
run on Scala and
this is not a very popular language.


 
 Vincent [not completely joking]
 
 Benedikt
 ___
 Caml-list mailing list. Subscription management:
 http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
 Archives: http://caml.inria.fr
 Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
 Bug reports: http://caml.inria.fr/bin/caml-bugs



___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Benedikt Meurer

On Nov 20, 2010, at 17:10 , Yoann Padioleau wrote:

 It's probably not a technical decision, but more likely a marketing 
 decision. If you tell Joe
 
 Who is Joe ? A developer ? A user ? A venture capitalist ?

Joe is 99.9% of world's population, excluding experts in this special area.

 that your webservices run on Java, PHP or .NET, he'll say great, sure or 
 wow (not because Joe's familiar with the technology or the theory, but 
 because he's familiar with the terms). Tell Joe your webservices run on 
 OCaml or Haskell and the best answer you can get will be what?.
 
 I doubt any user care about how facebook is implemented. Twitter and 
 Foursquare run on Scala and
 this is not a very popular language.

You can develop web services using probably any programming language available 
in the world. That's what I was about to say, it doesn't matter from a 
technological point of view. So if the programming language is irrelevant, but 
you have to pick one, you'll start looking for arguments to prefer one over the 
other (based on available libraries, marketing, etc.). Some argument made 
Facebook pick PHP (instead of Perl, Java, Ruby, C/C++, OCaml, Haskell, Standard 
ML, whatever), probably something trivial like availability of PHP developers, 
or simply because PHP was popular at that time (i.e. there were people who knew 
the term PHP).

Now the world looks different of course, there are hundreds of millions lines 
of existing code, and really porting all that code to a new language would 
involve a lot of effort, money and time.

Benedikt
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


RE: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Jon Harrop
 Do we have example of big companies porting their whole codebase to
 another language ?

Yes, of course. Companies modernise all the time. We have a client who just
started porting 1MLOC of C++ to F#. Flying Frog have ported hundreds of
thousands of lines of OCaml to F#. It happens all the time but it is even
more likely to happen as a consequence of multicore.

Cheers,
Jon.


___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Gerd Stolpmann
Am Samstag, den 20.11.2010, 08:10 -0800 schrieb Yoann Padioleau:
 On Nov 20, 2010, at 7:42 AM, Benedikt Meurer wrote:
 
  
  On Nov 20, 2010, at 16:19 , Vincent Balat wrote:
  
  On Nov 19, 2010 21:20:01, Yoann Padioleau wrote:
  On Nov 19, 2010, at 11:46 AM, Dario Teixeira wrote:
  ...
  Actually, Facebook has a compiler that transforms PHP source code into
  C++ [1], and they claim a 50% reduction in CPU usage.
  
  Yes, which is good. But if you think about it is only a x2 speedup vs a
  really slow bytecode interpreter (the Zend PHP interpreter). PHP is known
  for being more than 30 times slower than C.
  It's even slower than Ruby on
  http://shootout.alioth.debian.org/u32/which-programming-languages-are-fast
  est.php There are lots of opportunities to do better IMHO.
  
  Would it be completely inconceivable for a company like facebook to 
  reimplement everything using a fast well designed typed language
 
 I think it is inconceivable. I doubt facebook will switch to ocaml and 
 ocsigen tomorrow :)
 The problem is how to migrate code to another language smoothly ? You can not
 stop all development for a month and say Hey everybody, we are porting
 our 10 millions lines of code of PHP to X.. The reason C++ succeeded was
 because there was a smooth migration path. C code is valid C++ code. You
 can incrementally add objects to an existing codebase.

It's not a matter of the technical migration. You simply wouldn't find
enough engineers that can master a fast well designed typed
language (assuming here you have the more advanced ones in mind :-) ).
Also, there is the question whether it is the best approach to let the
most talented developers in the company program the user interface, with
day-to-day tasks like hey Joe, can you please fix this broken link.
(Not that there are also complicated UI's but these are rare.)

At Mylife.com (where I'm currently consulting for) large parts of the
backend are now running in Ocaml, but the web frontend is done by
another team, and there is PHP involved. IMHO, it would be a
misallocation of human resources if we (the Ocaml team) also did the
frontend.

 Do we have example of big companies porting their whole codebase to another 
 language ?

Usually languages are only switched when a completely new project is
started.

Gerd

  instead of spending hundreds of millions of dollars on machines that run 
  PHP 
  bytecode interpreters? (quoting Yoann)
 
 Apparently they decided to keep PHP and switch from a slow bytecode 
 interpreter
 to a 2x-faster compiler.
 
  
  It's probably not a technical decision, but more likely a marketing 
  decision. If you tell Joe
 
 Who is Joe ? A developer ? A user ? A venture capitalist ?
 
  that your webservices run on Java, PHP or .NET, he'll say great, sure 
  or wow (not because Joe's familiar with the technology or the theory, but 
  because he's familiar with the terms). Tell Joe your webservices run on 
  OCaml or Haskell and the best answer you can get will be what?.
 
 I doubt any user care about how facebook is implemented. Twitter and 
 Foursquare run on Scala and
 this is not a very popular language.
 
 
  
  Vincent [not completely joking]
  
  Benedikt
  ___
  Caml-list mailing list. Subscription management:
  http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
  Archives: http://caml.inria.fr
  Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
  Bug reports: http://caml.inria.fr/bin/caml-bugs
 
 
 
 ___
 Caml-list mailing list. Subscription management:
 http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
 Archives: http://caml.inria.fr
 Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
 Bug reports: http://caml.inria.fr/bin/caml-bugs
 


-- 

Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
g...@gerd-stolpmann.de  http://www.gerd-stolpmann.de
Phone: +49-6151-153855  Fax: +49-6151-997714


___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Yoann Padioleau

On Nov 20, 2010, at 8:25 AM, Benedikt Meurer wrote:
 You can develop web services using probably any programming language 
 available in the world. That's what I was about to say, it doesn't matter 
 from a technological point of view. So if the programming language is 
 irrelevant, but you have to pick one, you'll start looking for arguments to 
 prefer one over the other (based on available libraries,

I think this is the key, available libraries and tools around the language.

 marketing, etc.). Some argument made Facebook pick PHP (instead of Perl, 
 Java, Ruby, C/C++, OCaml, Haskell, Standard ML, whatever), probably something 
 trivial like availability of PHP developers

You think smart people in startups make big decisions based on such trivial 
things ? 
I guess they've chosen PHP in 2005 because it made sense to choose PHP in the 
context of what they were doing: prototyping
a social website as soon as possible by reusing as much as possible (Apache, 
Mysql, etc).
I know OCaml, I know PHP, I know Java, and in 2005 I've written my homepage in 
PHP, not OCaml, not Java.

 , or simply because PHP was popular at that time (i.e. there were people who 
 knew the term PHP).
 
 Now the world looks different of course, there are hundreds of millions lines 
 of existing code, and really porting all that code to a new language would 
 involve a lot of effort, money and time.
 
 Benedikt
 ___
 Caml-list mailing list. Subscription management:
 http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
 Archives: http://caml.inria.fr
 Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
 Bug reports: http://caml.inria.fr/bin/caml-bugs
 



___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Yoann Padioleau

On Nov 20, 2010, at 9:08 AM, Jon Harrop wrote:

 Do we have example of big companies porting their whole codebase to
 another language ?
 
 Yes, of course. Companies modernise all the time. We have a client who just
 started porting 1MLOC of C++ to F#.

How they do that ? Are they using compiler frontends to assist them
in automatically translating part of the code to F# ?

 Flying Frog have ported hundreds of
 thousands of lines of OCaml to F#.

OCaml and F# are very similar language, so such porting does not look that hard.
Translating from C++ to F#,  or from PHP to Java seems more complicated.
I wonder what kind of techniques people have developed to help such 
language-to-language
translation.

 It happens all the time but it is even
 more likely to happen as a consequence of multicore.
 
 Cheers,
 Jon.
 
 



___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


[Caml-list] Re: [Was: OCamlJit 2.0]

2010-11-20 Thread Sylvain Le Gall
On 20-11-2010, Yoann Padioleau pada...@wanadoo.fr wrote:

 On Nov 20, 2010, at 9:08 AM, Jon Harrop wrote:

 Do we have example of big companies porting their whole codebase to
 another language ?
 
 Yes, of course. Companies modernise all the time. We have a client who just
 started porting 1MLOC of C++ to F#.

 How they do that ? Are they using compiler frontends to assist them
 in automatically translating part of the code to F# ?


I worked for Metaware (http://www.metaware.fr). This company does source
to source migration of COBOL code. We also tried once to do COBOL to
Java translation using internal tools... It works, but probably not the
way you expect... Or maybe the same way obrowser works in fact: you
create a VM that interpret something in between COBOL and Java. You
loose comments and meanings in between. The result is 100%
unmaintainable/unreadable and at least 2x bigger. 

We stopped this kind of migration, because the result was not exactly
great (but it works). 

 It happens all the time but it is even
 more likely to happen as a consequence of multicore.

I doubt an old code, not written with multicore in mind is easily
portable to multicore. So basically, the migration you are talking about
is starting a new project that will replace one software/library by
another. 

Regards,
Sylvain Le Gall

___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


RE: [Caml-list] [Was: OCamlJit 2.0]

2010-11-20 Thread Jon Harrop
Yoann Padioleau wrote:
 On Nov 20, 2010, at 9:08 AM, Jon Harrop wrote:
  Do we have example of big companies porting their whole codebase to
  another language ?
 
  Yes, of course. Companies modernise all the time. We have a client
  who just
  started porting 1MLOC of C++ to F#.
 
 How they do that ? Are they using compiler frontends to assist them
 in automatically translating part of the code to F# ?

We do it for them for a fee, using magic. We tried automated translation
(from Fortran) but it is usually easier just to knuckle down by hand. Old
code bases are usually 90% cruft anyway. That 1MLOC of C++ could have been
100kLOC of C++ but now it will be 10kLOC of F# instead.

  Flying Frog have ported hundreds of thousands of lines of OCaml to F#.
 
 OCaml and F# are very similar language, so such porting does not look
 that hard.

Yes. The main thing is that OCaml source is nice and readable so it is easy
to translate even if it makes heavy use of features that don't port
(polymorphic variants, macros). C++ source can be a nightmare so you really
need to pin down a precise specification for what it does using advanced
techniques like poking it with a stick.

 Translating from C++ to F#,  or from PHP to Java seems more
 complicated.
 I wonder what kind of techniques people have developed to help such
 language-to-language translation.

The only viable option seems to be to do it by hand. You can churn through
an enormous amount of code when you put your mind to it...

Cheers,
Jon.


___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


RE: [Caml-list] Re: [Was: OCamlJit 2.0]

2010-11-20 Thread Jon Harrop
Sylvain Le Gall:
 I doubt an old code, not written with multicore in mind is easily
 portable to multicore. So basically, the migration you are talking
 about
 is starting a new project that will replace one software/library by
 another.

Yes, the systems are kept loosely coupled during the transition so it is
more like a new project to supercede the old one than a gradual transition
between code bases. However, only the developers need be aware of this
distinction. From the point of view of everyone else in the company, the
implementation of a product is being modernized. Management get improved
productivity/maintainability. Sales get shiny new things. HR get to fire any
developers who resist (but most are assimilated ;-).

The avalanche happens when the cost of rewriting falls below the cost of
adding essential features to a legacy code base.

Cheers,
Jon.


___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] Re: Looking for stubs for sendmsg/recvmsg

2010-11-20 Thread Goswin von Brederlow
Sylvain Le Gall sylv...@le-gall.net writes:

 On 19-11-2010, Goswin von Brederlow goswin-...@web.de wrote:
 Sylvain Le Gall sylv...@le-gall.net writes:
 On 19-11-2010, Goswin von Brederlow goswin-...@web.de wrote:
 Sylvain Le Gall sylv...@le-gall.net writes:
 On 18-11-2010, Goswin von Brederlow goswin-...@web.de wrote:

 Not that much, if you proceed in another way. I think what you are
 looking for is a fd leak detector?

 Here is a small modules that I have written for this purpose:

 File UnixExt.ml:
 (** Count open/close call *)

 [...]

 [...override other functions that open/close fd...]

 Then in the modules using this features, you just have to open UnixExt
 after Unix...

 You can even probably design a library that will transparently hide Unix
 with a custom Unix module providing this feature.

 Much less usefull.

 Using a custom block with finalizer means that the FD will be closed
 relative close to where/when it was leaked. Makes it easier to find
 where it was leaked and adding GC.compact calls at strategic locations
 can narrow it down even more. Leaking FDs also becomes much less
 serious. The GC will clean them up and close them. So you can use an app
 that leaks FDs just fine.


 It all depends on what you want: fix your program that leaks or live
 with it. The former piece of code helps to fail if there are leaked FD.
 On Unix FD leaks is not that problematic, but on Windows it turns to be
 another problem.

 The best example about this: you cannot delete a file that has an FD
 still open on it. This makes harder to remove temporary file (and this
 piece of code was precisely made to track FD on temporary files, that
 let 1000s of unremoved temp file). 

 Regards,
 Sylvain Le Gall

Which again speaks for my solution. The leaked FD will be closed much
faster (before the program terminates) and one can remove the tempfiles
while the program is still running.

MfG
Goswin

___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


Re: [Caml-list] SMP multithreading

2010-11-20 Thread Goswin von Brederlow
Jon Harrop jonathandeanhar...@googlemail.com writes:

 This is actually a quick way to use multiple cores with ocaml. Find a
 often called function that takes considerable time and offload it to C

 Or HLVM, F#, Scala, Clojure or any of the other languages that permit shared
 memory parallelism. C is particularly poor in this regard so I would not
 just restrict yourself to C...

 Cheers,
 Jon.

I'm not talking about any shared memory parallelism here. The
parallelism is completly restricted to the ocaml side. You just find
some single threaded job that takes long, rewrite it as external
function and release the ocaml lock while it is running.

For example in my code I compute the sha256 sum of a block of
data. Since I use a C library for sha256 anyway the function is already
external. All I had to do was switch the interface from using string to
Bigarray and add enter/leave_blocking_section(). After that multiple
threads can compute the sha256 sum for blocks of data in parallel and my
code run 3.7 times faster with 4 cores.

MfG
Goswin


___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


[Caml-list] Re: Looking for stubs for sendmsg/recvmsg

2010-11-20 Thread Sylvain Le Gall
On 20-11-2010, Goswin von Brederlow goswin-...@web.de wrote:
 Sylvain Le Gall sylv...@le-gall.net writes:
 On 19-11-2010, Goswin von Brederlow goswin-...@web.de wrote:
 Sylvain Le Gall sylv...@le-gall.net writes:
 On 19-11-2010, Goswin von Brederlow goswin-...@web.de wrote:
 Sylvain Le Gall sylv...@le-gall.net writes:
 On 18-11-2010, Goswin von Brederlow goswin-...@web.de wrote:

 The best example about this: you cannot delete a file that has an FD
 still open on it. This makes harder to remove temporary file (and this
 piece of code was precisely made to track FD on temporary files, that
 let 1000s of unremoved temp file). 

 Regards,
 Sylvain Le Gall

 Which again speaks for my solution. The leaked FD will be closed much
 faster (before the program terminates) and one can remove the tempfiles
 while the program is still running.


I think, this totally off-topic. But anyway, when a program create a
temporary files it needs to remove ASAP, itself. I wouldn't 
deliver a program with a notice like sometimes FD leaked, not a
problem, just remove $TMP/myprogram-*. 

For the little story, the leaked FD (hence the temporary files) was
400MB each and it quickly get noticed after a few run (and a full HD).

Regards,
Sylvain Le Gall

___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs