Re: [WORK] std.file.update function

2016-10-18 Thread Patrick Schluter via Digitalmars-d
On Tuesday, 18 October 2016 at 13:51:48 UTC, R wrote: On Monday, 19 September 2016 at 02:57:01 UTC, Chris Wright wrote: You have an operating system that automatically checksums every file? There are a few filesystems that keep checksums of blocks, but I don't see one that keeps file

Re: [WORK] std.file.update function

2016-10-18 Thread R via Digitalmars-d
On Monday, 19 September 2016 at 02:57:01 UTC, Chris Wright wrote: You have an operating system that automatically checksums every file? There are a few filesystems that keep checksums of blocks, but I don't see one that keeps file checksums.

Re: [WORK] std.file.update function

2016-09-19 Thread Walter Bright via Digitalmars-d
One way to implement it is to open the existing file as a memory-mapped file. Memory-mapped files only get paged into memory as the memory is referenced. So if you did a memcmp(oldfile, newfile, size), it will stop once the first difference is found, and the rest of the file is never read.

Re: [WORK] std.file.update function

2016-09-19 Thread Walter Bright via Digitalmars-d
On 9/19/2016 7:04 AM, Andrei Alexandrescu wrote: On 09/19/2016 01:16 AM, Walter Bright wrote: The compiler currently creates the complete object file in a buffer, then writes the buffer to a file in one command. The reason is mostly because the object file format isn't incremental, the

Re: [WORK] std.file.update function

2016-09-19 Thread Stefan Koch via Digitalmars-d
On Monday, 19 September 2016 at 14:04:03 UTC, Andrei Alexandrescu wrote: Interesting. What happens e.g. if one makes a change to a function whose generated code is somewhere in the middle of the object file? If it doesn't alter the call graph, doesn't the new .o file share a common prefix

Re: [WORK] std.file.update function

2016-09-19 Thread Andrei Alexandrescu via Digitalmars-d
On 09/18/2016 10:05 PM, Brad Roberts via Digitalmars-d wrote: This is nice in the case of no changes, but problematic in the case of some changes. The standard write new, rename technique never has either file in a half-right state. The file is atomically either old or new and nothing in

Re: [WORK] std.file.update function

2016-09-19 Thread Andrei Alexandrescu via Digitalmars-d
On 09/19/2016 01:16 AM, Walter Bright wrote: On 9/18/2016 8:20 AM, Andrei Alexandrescu wrote: On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: Simplest case is - source file is being changed, therefore a new object file is being produced, therefore a new executable is being produced.

Re: [WORK] std.file.update function

2016-09-19 Thread ketmar via Digitalmars-d
On Monday, 19 September 2016 at 06:53:47 UTC, Walter Bright wrote: Doing a linker inside DMD means that object files imported from other C/C++ compilers have to be correctly interpreted. I could do it, but I couldn't do that and continue to work on D. yeah. there is a reason for absense of

Re: [WORK] std.file.update function

2016-09-19 Thread Walter Bright via Digitalmars-d
On 9/18/2016 11:33 PM, Stefan Koch wrote: However the maintenance burden is a bit heavy we don't have enough menpower as it is. A major part of the problem (that working with Optlink has made painfully clear) is that although linking is conceptually a rather trivial task, the people who've

Re: [WORK] std.file.update function

2016-09-19 Thread Stefan Koch via Digitalmars-d
On Monday, 19 September 2016 at 05:16:37 UTC, Walter Bright wrote: I'd love to design our own high speed formats, but then they'd be incompatible with everybody else's. I'd like that as well. I recently had a look at the ELF and the COFF file formats both are definitely in need of rework

Re: [WORK] std.file.update function

2016-09-19 Thread Jacob Carlborg via Digitalmars-d
On 2016-09-19 07:16, Walter Bright wrote: I'd love to design our own high speed formats, but then they'd be incompatible with everybody else's. You already mentioned in an other post [1] that the compiler could do the linking as well. In that case you would need to write some form of

Re: [WORK] std.file.update function

2016-09-18 Thread Walter Bright via Digitalmars-d
On 9/18/2016 7:05 PM, Brad Roberts via Digitalmars-d wrote: This is nice in the case of no changes, but problematic in the case of some changes. The standard write new, rename technique never has either file in a half-right state. The file is atomically either old or new and nothing in

Re: [WORK] std.file.update function

2016-09-18 Thread Walter Bright via Digitalmars-d
On 9/18/2016 8:20 AM, Andrei Alexandrescu wrote: On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: Simplest case is - source file is being changed, therefore a new object file is being produced, therefore a new executable is being produced. Forgot to mention a situation here: if you change

Re: [WORK] std.file.update function

2016-09-18 Thread Chris Wright via Digitalmars-d
On Mon, 19 Sep 2016 04:24:41 +1200, rikki cattermole wrote: > On 19/09/2016 3:41 AM, Andrei Alexandrescu wrote: >> On 9/18/16 11:24 AM, rikki cattermole wrote: >>> On 19/09/2016 3:20 AM, Andrei Alexandrescu wrote: On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: > Simplest case is -

Re: [WORK] std.file.update function

2016-09-18 Thread Brad Roberts via Digitalmars-d
On 9/18/2016 8:17 AM, Andrei Alexandrescu via Digitalmars-d wrote: There is actually an even better way at the application level. Consider a function in std.file: updateS, Range)(S name, Range data); updateFile does something interesting: it opens the file "name" for reading AND writing, then

Re: [WORK] std.file.update function

2016-09-18 Thread rikki cattermole via Digitalmars-d
On 19/09/2016 3:41 AM, Andrei Alexandrescu wrote: On 9/18/16 11:24 AM, rikki cattermole wrote: On 19/09/2016 3:20 AM, Andrei Alexandrescu wrote: On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: Simplest case is - source file is being changed, therefore a new object file is being produced,

Re: [WORK] std.file.update function

2016-09-18 Thread Andrei Alexandrescu via Digitalmars-d
On 9/18/16 12:15 PM, Chris Wright wrote: This will produce different behavior with hard links. With hard links, the temporary file mechanism you mention will result in the old file being accessible via the other path. With your recommended strategy, the data accessible from both paths is

Re: [WORK] std.file.update function

2016-09-18 Thread Chris Wright via Digitalmars-d
This will produce different behavior with hard links. With hard links, the temporary file mechanism you mention will result in the old file being accessible via the other path. With your recommended strategy, the data accessible from both paths is updated. That's probably acceptable, and hard

Re: [WORK] std.file.update function

2016-09-18 Thread Andrei Alexandrescu via Digitalmars-d
On 9/18/16 11:24 AM, rikki cattermole wrote: On 19/09/2016 3:20 AM, Andrei Alexandrescu wrote: On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: Simplest case is - source file is being changed, therefore a new object file is being produced, therefore a new executable is being produced.

Re: [WORK] std.file.update function

2016-09-18 Thread Stefan Koch via Digitalmars-d
On Sunday, 18 September 2016 at 15:17:31 UTC, Andrei Alexandrescu wrote: There are quite a few situations in rdmd and dmd generally when we compute a dependency structure over sets of files. Based on that, we write new files that overwrite old, obsoleted files. Those changes in turn trigger

Re: [WORK] std.file.update function

2016-09-18 Thread rikki cattermole via Digitalmars-d
On 19/09/2016 3:20 AM, Andrei Alexandrescu wrote: On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: Simplest case is - source file is being changed, therefore a new object file is being produced, therefore a new executable is being produced. Forgot to mention a situation here: if you change

Re: [WORK] std.file.update function

2016-09-18 Thread Andrei Alexandrescu via Digitalmars-d
On 09/18/2016 11:17 AM, Andrei Alexandrescu wrote: Simplest case is - source file is being changed, therefore a new object file is being produced, therefore a new executable is being produced. Forgot to mention a situation here: if you change the source code of a module without influencing

[WORK] std.file.update function

2016-09-18 Thread Andrei Alexandrescu via Digitalmars-d
There are quite a few situations in rdmd and dmd generally when we compute a dependency structure over sets of files. Based on that, we write new files that overwrite old, obsoleted files. Those changes in turn trigger other dependencies to go stale so more building is done etc. Simplest case