Re: Button: A fast, correct, and elegantly simple build system.

2016-06-27 Thread Jason White via Digitalmars-d-announce

On Monday, 27 June 2016 at 06:43:26 UTC, Rory McGuire wrote:
FYI, I implemented this feature today (no Batch/PowerShell 
output yet

though):

http://jasonwhite.github.io/button/docs/commands/convert

I think Bash should work on most Unix-like platforms.



And there is this[0] for windows, if you wanted to try bash on 
windows:



[0]: https://msdn.microsoft.com/en-us/commandline/wsl/about


Thanks, but I'll be sticking to bash on Linux. ;)

I'll add Batch (and maybe PowerShell) output when Button is 
supported on Windows. It should be very easy.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-26 Thread Jason White via Digitalmars-d-announce

On Monday, 20 June 2016 at 08:21:29 UTC, Dicebot wrote:

On Monday, 20 June 2016 at 02:46:13 UTC, Jason White wrote:
This actually sounds nice. Main problem that comes to my mind 
is that there is no cross-platform shell script. Even if it 
is list of plain unconditional commands there are always 
differences like normalized path form. Of course, one can 
always generate `build.d` as a shell script, but that would 
only work for D projects and Button is supposed to be a 
generic solution.


I'd make it so it could either produce a Bash or Batch script. 
Possibly also a PowerShell script because error handling in 
Batch is awful. That should cover any platform it might be 
needed on. Normalizing paths shouldn't be a problem either.


This should actually be pretty easy to implement.


Will plain sh script script also work for MacOS / BSD flavors? 
Committing just two scripts is fine but I wonder how it scales.


FYI, I implemented this feature today (no Batch/PowerShell output 
yet though):


http://jasonwhite.github.io/button/docs/commands/convert

I think Bash should work on most Unix-like platforms.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-19 Thread Jason White via Digitalmars-d-announce

On Sunday, 19 June 2016 at 15:47:21 UTC, Dicebot wrote:
Let me propose another idea where maybe we can remove the 
extra dependency for new codebase collaborators but still have 
access to a full-blown build system: Add a sub-command to 
Button that produces a shell script to run the build. For 
example, `button shell -o build.sh`. Then just run 
`./build.sh` to build everything. I vaguely recall either Tup 
or Ninja having something like this.


This actually sounds nice. Main problem that comes to my mind 
is that there is no cross-platform shell script. Even if it is 
list of plain unconditional commands there are always 
differences like normalized path form. Of course, one can 
always generate `build.d` as a shell script, but that would 
only work for D projects and Button is supposed to be a generic 
solution.


I'd make it so it could either produce a Bash or Batch script. 
Possibly also a PowerShell script because error handling in Batch 
is awful. That should cover any platform it might be needed on. 
Normalizing paths shouldn't be a problem either.


This should actually be pretty easy to implement.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-19 Thread Jason White via Digitalmars-d-announce

On Saturday, 18 June 2016 at 23:52:00 UTC, H. S. Teoh wrote:

I did a quick investigation, which found something interesting.
 If compiling straight to executable, the executable is 
identical each time with the same md5sum.  However, when 
compiling to object files, the md5sum is sometimes the same, 
sometimes different.  Repeating this several time reveals that 
the md5sum changes every second, meaning that the difference is 
a timestamp in the object file.


Maybe we could file an enhancement request for this?


Done: https://issues.dlang.org/show_bug.cgi?id=16185


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-18 Thread Jason White via Digitalmars-d-announce

On Saturday, 18 June 2016 at 14:23:39 UTC, H. S. Teoh wrote:
Moral of the story is, if you're writing a compiler, for the 
sake of build systems everywhere, make the output 
deterministic! For consecutive invocations, without changing 
any source code, I want the hashes of the binaries to be 
identical every single time. DMD doesn't do this and it 
saddens me greatly.


DMD doesn't? What does it do that isn't deterministic?


I have no idea. As a simple test, I compiled one my source files 
to an object file, and ran md5sum on it. I did this again and the 
md5sum is different. Looking at a diff of the hexdump isn't very 
fruitful either (for me at least). For reference, I'm on Linux 
x86_64 with DMD v2.071.0.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-18 Thread Jason White via Digitalmars-d-announce

On Friday, 17 June 2016 at 20:36:53 UTC, H. S. Teoh wrote:

- Assuming that a revision control system is in place, and a
  workspace is checked out on revision X with no further
  modifications, then invoking the build tool should ALWAYS,
  without any exceptions, produce exactly the same outputs, bit
  for bit.  I.e., if your workspace faithfully represents
  revision X in the RCS, then invoking the build tool will
  produce the exact same binary products as anybody else who
  checks out revision X, regardless of their initial starting
  conditions.


Making builds bit-for-bit reproducible is really, really hard to 
do, particularly on Windows. Microsoft's C/C++ compiler embeds 
timestamps and other nonsense into the binaries so that every 
time you build, even when no source changed, you get a different 
binary. Google wrote a tool to help eliminate this 
non-determinism as a post-processing step called 
zap_timestamp[1]. I want to eventually include something like 
this with Button on Windows. I'll probably have to make a PE 
reader library first though.


Without reproducible builds, caching outputs doesn't work very 
well either.


Moral of the story is, if you're writing a compiler, for the sake 
of build systems everywhere, make the output deterministic! For 
consecutive invocations, without changing any source code, I want 
the hashes of the binaries to be identical every single time. DMD 
doesn't do this and it saddens me greatly.


[1] 
https://github.com/google/syzygy/tree/master/syzygy/zap_timestamp


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-18 Thread Jason White via Digitalmars-d-announce

On Friday, 17 June 2016 at 10:24:16 UTC, Dicebot wrote:
However, I question the utility of even doing this in the 
first place. You miss out on the convenience of using the 
existing command line interface. And for what? Just so 
everything can be in D? Writing the same thing in Lua would be 
much prettier. I don't understand this dependency-phobia.


It comes from knowing that for most small to average size D 
projects you don't need a build _tool_ at all. If full clean 
build takes 2 seconds, installing extra tool to achieve the 
same thing one line shell script does is highly annoying.


Your reasoning about makefiles seems to be flavored by C++ 
realities. But my typical D makefile would look like something 
this:


build:
dmd -ofbinary `find ./src`

test:
dmd -unittest -main `find ./src`

deploy: build test
scp ./binary server:

That means that I usually care neither about correctness nor 
about speed, only about good cross-platform way to define 
pipelines. And for that fetching dedicated tool is simply too 
discouraging.


In my opinion that is why it is so hard to take over make place 
for any new tool - they all put too much attention into 
complicated projects but to get self-sustained network effect 
one has to prioritize small and simple projects. And ease of 
availability is most important there.


I agree that a sophisticated build tool isn't really needed for 
tiny projects, but it's still really nice to have one that can 
scale as the project grows. All too often, as a project gets 
bigger, the build system it uses buckles under the growing 
complexity, no one ever gets around to changing it because 
they're afraid of breaking something, and the problem just gets 
worse.


I realize you might be playing devil's advocate a bit and I 
appreciate it. Let me propose another idea where maybe we can 
remove the extra dependency for new codebase collaborators but 
still have access to a full-blown build system: Add a sub-command 
to Button that produces a shell script to run the build. For 
example, `button shell -o build.sh`. Then just run `./build.sh` 
to build everything. I vaguely recall either Tup or Ninja having 
something like this.


The main downside is that it'd have to be committed every time 
the build changes. This could be automated with a bot, but it's 
still annoying. The upsides are that there is no need for any 
other external libraries or tools, and the superior build system 
can still be used by anyone who wants it.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-18 Thread Jason White via Digitalmars-d-announce

On Friday, 17 June 2016 at 20:59:46 UTC, jmh530 wrote:
I found the beginning of the tutorial very clear. I really 
liked that it can produce a png of the build graph. I also 
liked the Lua build description for DMD. Much more legible than 
the make file.


However, once I got to the "Going Meta: Building the Build 
Description" section of the tutorial, I got a little confused.


I found it a little weird that the json output towards the end 
of the tutorial don't always match up. Like, where did the .h 
files go from the inputs? (I get that they aren't needed for 
running gcc, but you should mention that) Why is it displaying 
cc instead of gcc? I just feel like you might be able to split 
things up a little and provide a few more details. Like, this 
is how to do a base version, then say this is how you can 
customize what is displayed. Also, it's a little terse on the 
details of things like what the cc.binary is doing. Always err 
on the side of explaining things too much rather than too 
little, IMO.


Thank you for the feedback! I'm glad someone has read the 
tutorial.


I'm not happy with that section either. I think I'll split it up 
and go into more depth, possibly moving it to a separate page. I 
also still need to write docs on the Lua parts (like cc.binary), 
but that API is subject to change.


Unlike most people, I kind of actually enjoy writing 
documentation.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-17 Thread Jason White via Digitalmars-d-announce

On Friday, 17 June 2016 at 06:18:28 UTC, H. S. Teoh wrote:
For me, correctness is far more important than speed. Mostly 
because at my day job, we have a Make-based build system and 
because of Make's weaknesses, countless hours, sometimes even 
days, have been wasted running `make clean; make` just so we 
can "be sure".  Actually, it's worse than that; the "official" 
way to build it is:


svn diff > /tmp/diff
\rm -rf old_checkout
mkdir new_checkout
cd new_checkout
svn co http://svnserver/path/to/project
patch -p0 because we have been bitten before by `make clean` not *really* 
cleaning *everything*, and so `make clean; make` was actually 
producing a corrupt image, whereas checking out a fresh new 
workspace produces the correct image.


Far too much time has been wasted "debugging" bugs that weren't 
really there, just because Make cannot be trusted to produce 
the correct results. Or heisenbugs that disappear when you 
rebuild from scratch. Unfortunately, due to the size of our 
system, a fresh svn checkout on a busy day means 15-20 mins 
(due to everybody on the local network trying to do fresh 
checkouts!), then make takes about 30-45 mins to build 
everything.  When your changeset touches Makefiles, this could 
mean a 1 hour turnaround for every edit-compile-test cycle, 
which is ridiculously unproductive.


Such unworkable turnaround times, of course, causes people to 
be lazy and just run tests on incremental builds (of unknown 
correctness), which results in people checking in changesets 
that are actually wrong but just happen to work when they were 
testing on an incremental build (thanks to Make picking up 
stray old copies of obsolete libraries or object files or other 
such detritus). Which means *everybody*'s workspace breaks 
after running `svn update`. And of course, nobody is sure 
whether it broke because of their own changes, or because 
somebody checked in a bad changeset; so it's `make clean; make` 
time just to "be sure". That's n times how many man-hours (for 
n = number of people on the team) straight down the drain, 
where had the build system actually been reliable, only the 
person responsible would have to spend a few extra hours to fix 
the problem.


Make proponents don't seem to realize how a seemingly 
not-very-important feature as build correctness actually adds 
up to a huge cost in terms of employee productivity, i.e., 
wasted hours, AKA wasted employee wages for the time spent 
watching `make clean; make` run.


I couldn't agree more! Correctness is by far the most important 
feature of a build system. Second to that is probably being able 
to make sense of what is happening.


I have the same problems as you in my day job, but magnified. 
Some builds take 3+ hours, some nearly 24 hours, and none of the 
developers can run full builds themselves because the build 
process is so long and complicated. Turn-around time to test 
changes is abysmal and everyone is probably orders of magnitude 
more unproductive because of it. All of this because we can't 
trust Make or Visual Studio to do incremental builds correctly.


I hope to change that with Button.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-16 Thread Jason White via Digitalmars-d-announce

On Thursday, 16 June 2016 at 13:39:20 UTC, Atila Neves wrote:
It would be a worthwhile trade-off, if those were the only two 
options available, but they're not. There are multiple build 
systems out there that do correct builds whilst being faster 
than make. Being faster is easy, because make is incredibly 
slow.


I didn't even find out about ninja because I read about it in a 
blog post, I actively searched for a make alternative because I 
was tired of waiting for it.


Make is certainly not slow for full builds. That is what I was 
testing.


I'm well aware of Ninja and it is maybe only 1% faster than Make 
for full builds[1]. There is only so much optimization that can 
be done when spawning processes as dictated by a DAG. 99% of the 
CPU's time is spent on running the tasks themselves.


Where Make gets slow is when checking for changes on a ton of 
files. I haven't tested it, but I'm sure Button is faster than 
Make in this case because it checks for changed files using 
multiple threads. Using the file system watcher can also bring 
this down to a near-zero time.


Speed is not the only virtue of a build system. A build system 
can be amazeballs fast, but if you can't rely on it doing 
incremental builds correctly in production, then you're probably 
doing full builds every single time. Being easy to use and robust 
is also pretty important.


[1] 
http://hamelot.io/programming/make-vs-ninja-performance-comparison/


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-16 Thread Jason White via Digitalmars-d-announce

On Thursday, 16 June 2016 at 12:34:26 UTC, Kagamin wrote:

On Sunday, 12 June 2016 at 23:27:07 UTC, Jason White wrote:
However, I question the utility of even doing this in the 
first place. You miss out on the convenience of using the 
existing command line interface.


Why the build script can't have a command line interface?


It could, but now the build script is a more complicated and for 
little gain. Adding command line options on top of that to 
configure the build would be painful.


It would be simpler and cleaner to write a D program to generate 
the JSON build description for Button to consume. Then you can 
add a command line interface to configure how the build 
description is generated. This is how the Lua build descriptions 
work[1].


[1] 
http://jasonwhite.github.io/button/docs/tutorial#going-meta-building-the-build-description


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-15 Thread Jason White via Digitalmars-d-announce
On Wednesday, 15 June 2016 at 12:02:56 UTC, Andrei Alexandrescu 
wrote:
In all likelihood. One issue with build systems is there's no 
clear heir to make. There are so many, including a couple (!) 
by our community, each with its pros and cons. Which one should 
we choose?


You should choose mine, obviously. ;)

In all seriousness, Make will probably live as long as C. There 
are a *ton* of Makefiles out there that no one wants translate to 
a new build system. Part of the reason for that is probably 
because they are so friggin' incomprehensible and its not exactly 
glamorous work. This is why I'm working on that tool to allow 
Button to build existing Makefiles [1]. It may not work 100% of 
the time, but it should help a lot with migrating away from Make.


[1] https://github.com/jasonwhite/button-make


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-15 Thread Jason White via Digitalmars-d-announce
On Wednesday, 15 June 2016 at 12:00:52 UTC, Andrei Alexandrescu 
wrote:
I'd say the gating factor is -j. If an build system doesn't 
implement the equivalent of make -j, that's a showstopper.


Don't worry, there is a --threads option and it defaults to the 
number of logical cores.


I just did some tests and the reason it is slower than Make is 
because of the automatic dependency detection on every single 
command. I disabled the automatic dependency detection and 
compared it with Make again. Button was then roughly the same 
speed as Make -- sometimes it was faster, sometimes slower. 
Although, I think getting accurate dependencies at the cost of 
slightly slower builds is very much a worthwhile trade-off.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-14 Thread Jason White via Digitalmars-d-announce

On Monday, 13 June 2016 at 20:12:27 UTC, Walter Bright wrote:

On 6/12/2016 4:27 PM, Jason White wrote:

I don't understand this dependency-phobia.


It's the "first 5 minutes" thing. Every hiccup there costs us 
maybe half the people who just want to try it out.


I suppose you're right. It is just frustrating that people are 
unwilling to adopt clearly superior tools simply because it would 
introduce a new dependency. I'm sure D itself has the same exact 
problem.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-14 Thread Jason White via Digitalmars-d-announce
On Tuesday, 14 June 2016 at 14:57:52 UTC, Andrei Alexandrescu 
wrote:

On 6/12/16 8:27 PM, Walter Bright wrote:

On 5/30/2016 12:16 PM, Jason White wrote:

Here is an example build description for DMD:


https://github.com/jasonwhite/dmd/blob/button/src/BUILD.lua


I'd say that's a lot easier to read than this crusty thing:

https://github.com/dlang/dmd/blob/master/src/posix.mak


Yes, the syntax looks nice.


Cool. Difference in size is also large. Do they do the same 
things? -- Andrei


Not quite. It doesn't download a previous version of dmd for 
bootstrapping and it doesn't handle configuration (e.g., x86 vs 
x64). About all it does is the bare minimum work necessary to 
create the dmd executable. I basically ran `make all -n` and 
converted the output because it's easier to read than the 
Makefile itself.


Building from scratch takes about 7 seconds on my machine (using 
8 cores and building in /tmp). Make takes about 5 seconds. Guess 
I need to do some optimizing. :-)


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-14 Thread Jason White via Digitalmars-d-announce

On Tuesday, 14 June 2016 at 10:47:58 UTC, Fool wrote:
Switching the compiler version seems to be a valid use case. 
You might have other means to detect this, though.


If you want to depend on the compiler version, then you can add a 
dependency on the compiler executable. It might be a good idea to 
have Button do this automatically for every command. That is, 
finding the path to the command's executable and making it a 
dependency.


A possible use case is creating object files first and packing 
them into a library as a second step. Then single object files 
are of not much interest anymore. Imagine, you want to 
distribute a build to several development machines such that 
their local build environments are convinced that the build is 
up to date. If object files can be treated as secondary or 
intermediate targets you can save lots of unnecessary network 
traffic and storage.


You're right, that is a valid use case. In my day job, we have 
builds that produce 60+ GB of object files. It would be wasteful 
to distribute all that to development machines.


However, I can think of another scenario where it would just as 
well be incorrect behavior: Linking an executable and then 
running tests on it. The executable could then be seen by the 
build system as the "secondary" or "intermediate" output. If it 
gets deleted, I think we'd want it rebuilt.


I'm not sure how Make or Shake implement this without doing it 
incorrectly in certain scenarios. There would need to be a way to 
differentiate between necessary and unnecessary outputs. I'll 
have to think about this more.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-12 Thread Jason White via Digitalmars-d-announce

On Sunday, 12 June 2016 at 20:03:06 UTC, Walter Bright wrote:

On 6/3/2016 1:26 AM, Dicebot wrote:
From that perspective, the best build system you could 
possibly have would look

like this:

```
#!/usr/bin/rdmd

import std.build;

// define your build script as D code
```


Yeah, I have often thought that writing a self-contained D 
program to build D would work well. The full power of the 
language would be available, there'd be nothing new to learn, 
and all you'd need is an existing D compiler (which we already 
require to build).


The core functionality of Button could be split off into a 
library fairly easily and there would be no dependency on Lua. 
Using it might look something like this:


import button;

immutable Rule[] rules = [
{
inputs: [Resource("foo.c"), Resource("baz.h")],
task: Task([Command(["gcc", "-c", "foo.c", "-o", 
"foo.o"])]),

outputs: [Resource("foo.o")]
},
{
inputs: [Resource("bar.c"), Resource("baz.h")],
task: Task([Command(["gcc", "-c", "bar.c", "-o", 
"bar.o"])]),

outputs: [Resource("bar.o")]
},
{
inputs: [Resource("foo.o"), Resource("bar.o")],
task: Task([Command(["gcc", "foo.o", "bar.o", "-o", 
"foobar"])]),

outputs: [Resource("foobar")]
}
];

void main()
{
build(rules);
}

Of course, more abstractions would be added to make creating the 
list of rules less verbose.


However, I question the utility of even doing this in the first 
place. You miss out on the convenience of using the existing 
command line interface. And for what? Just so everything can be 
in D? Writing the same thing in Lua would be much prettier. I 
don't understand this dependency-phobia.


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-11 Thread Jason White via Digitalmars-d-announce

On Saturday, 11 June 2016 at 02:48:59 UTC, H. S. Teoh wrote:
Finally got around to looking at this (albeit just briefly). It 
looks very nice!  Perhaps I'll try using it for my next project.


If you do end up using it, I'd be happy to iron out any 
irritations in Button that you encounter.


Button really needs a large project using it to help drive 
refinements.


I'm particularly pleased with the bipartite graph idea. It's a 
very nice way of sanely capturing the idea of build commands 
that generate multiple outputs.  Also big plusses in my book 
are implicit dependencies and use of inotify to eliminate the 
infamous "thinking pause" that older build systems all suffer 
from (this idea was also advanced by tup, but IMO Button looks 
a tad more polished than tup in terms of overall design).  Of 
course, being written in D is a bonus in my book. :-D Though 
realistically speaking it probably doesn't really matter to me 
as an end user, other than just giving me warm fuzzies.


Tup has had a big influence on the design of Button (e.g., a 
bipartite graph, deleting unused outputs, implicit dependencies, 
using Lua, etc.). Overall, I'd say Button does the same or better 
in every respect except maybe speed.


About it being written in D: If Rust had been mature enough when 
I first started working on it, I might have used it instead. All 
I knew is that I didn't want to go through the pain of writing it 
in C/C++. :-)


Unfortunately I don't have the time right now to actually do 
anything non-trivial with it... but I'll try to give feedback 
when I do get around to it (and I definitely plan to)!


Thanks! I look forward to it!


Re: Button: A fast, correct, and elegantly simple build system.

2016-06-01 Thread Jason White via Digitalmars-d-announce

On Wednesday, 1 June 2016 at 06:41:17 UTC, Jacob Carlborg wrote:
So, Lua is a build dependency? Seems that Sqlite is a build 
dependency as well.


Actually, SQLite more of a run-time dependency because 
etc.c.sqlite3 comes with DMD.


$ ldd button
linux-vdso.so.1 (0x7ffcc474c000)
--> libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 
(0x7f2d13641000)
libpthread.so.0 => /usr/lib/libpthread.so.0 
(0x7f2d13421000)

libm.so.6 => /usr/lib/libm.so.6 (0x7f2d13119000)
librt.so.1 => /usr/lib/librt.so.1 (0x7f2d12f11000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x7f2d12d09000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f2d12af1000)
libc.so.6 => /usr/lib/libc.so.6 (0x7f2d12749000)
/lib64/ld-linux-x86-64.so.2 (0x7f2d13951000)


Re: Reddit announcements

2016-05-31 Thread Jason White via Digitalmars-d-announce

On Tuesday, 31 May 2016 at 18:57:29 UTC, o-genki-desu-ka wrote:

Many nice announcements here last week. I put some on reddit.


Thank you for doing this! I agree previous posts though, that 
this is too many at once. Also, I think posting a link directly 
to the project instead of the forum post would have been better.


Re: Button: A fast, correct, and elegantly simple build system.

2016-05-31 Thread Jason White via Digitalmars-d-announce

On Tuesday, 31 May 2016 at 14:28:02 UTC, Dicebot wrote:
Can it be built from just plain dmd/phobos install available? 
One of major concernc behind discussion that resulted in Atila 
reggae effort is that propagating additional third-party 
dependencies is very damaging for build systems. Right now 
Button seems to fail rather hard on this front (i.e. Lua for 
build description + uncertain amount of build dependencies for 
Button itself).


Building it only requires dmd+phobos+dub.

Why is having dependencies so damaging for build systems? Does it 
really matter with a package manager like Dub? If there is 
another thread that answers these questions, please point me to 
it.


The two dependencies Button itself has could easily be moved into 
the same project. I kept them separate because they can be useful 
for others. These are the command-line parser and IO stream 
libraries.


As for the dependency on Lua, it is statically linked into a 
separate executable (called "button-lua") and building it is 
dead-simple (just run make). Using the Lua build description 
generator is actually optional, it's just that writing build 
descriptions in JSON would be horribly tedious.


Re: Button: A fast, correct, and elegantly simple build system.

2016-05-31 Thread Jason White via Digitalmars-d-announce

On Tuesday, 31 May 2016 at 10:15:14 UTC, Atila Neves wrote:

On Monday, 30 May 2016 at 19:16:50 UTC, Jason White wrote:
I am pleased to finally announce the build system I've been 
slowly working on for over a year in my spare time:


snip
In fact, there is some experimental support for automatic 
conversion of Makefiles to Button's build description format 
using a fork of GNU Make itself: 
https://github.com/jasonwhite/button-make


I'm going to take a look at that!


I think the Makefile converter is probably the coolest thing 
about this build system. I don't know of any other build system 
that has done this. The only problem is that it doesn't do well 
with Makefiles that invoke make recursively. I tried compiling 
Git using it, but Git does some funky stuff with recursive make 
like grepping the output of the sub-make.


- Can automatically build when an input file is modified 
(using inotify).


Nope, I never found that interesting. Possibly because I keep 
saving after every edit in OCD style and I really don't want 
things running automatically.


I constantly save like a madman too. If an incremental build is 
sufficiently fast, it doesn't really matter. You can also specify 
a delay so it accumulates changes and then after X milliseconds 
it runs a build.


- Recursive: It can build the build description as part of the 
build.


I'm not sure what that means. reggae copies CMake here and runs 
itself when the build description changes, if that's what you 
mean.


It means that Button can run Button as a build task (and it does 
it correctly). A child Button process reports its dependencies to 
the parent Button process via a pipe. This is the same mechanism 
that detects dependencies for ordinary tasks. Thus, there is no 
danger of doing incorrect incremental builds when recursively 
running Button like there is with Make.



- Lua is the primary build description language.


In reggae you can pick from D, Python, Ruby, Javascript and Lua.


That's pretty cool. It is possible for Button to do the same, but 
I don't really want to support that many languages. In fact, the 
Make and Lua build descriptions both work the same exact way - 
they output a JSON build description for Button to use. So long 
as someone can write a program to do this, they can write their 
build description in it.


Re: Button: A fast, correct, and elegantly simple build system.

2016-05-31 Thread Jason White via Digitalmars-d-announce

On Tuesday, 31 May 2016 at 03:40:32 UTC, rikki cattermole wrote:

Are you on Freenode (no nick to name right now)?
I would like to talk to you about a few ideas relating to lua 
and D.


No, I'm not on IRC. I'll see if I can find the time to hop on 
this weekend.


Re: Button: A fast, correct, and elegantly simple build system.

2016-05-30 Thread Jason White via Digitalmars-d-announce

On Monday, 30 May 2016 at 20:58:51 UTC, poliklosio wrote:

- Lua is the primary build description language.


Why not D?


Generating the JSON build description should entirely 
deterministic. With Lua, this can be guaranteed. You can create a 
sandbox where only certain operations are permitted. For example, 
reading files is permitted, but writing to them is not. I can 
also intercept all file reads and mark the files that get read as 
dependencies.


It certainly could be done in D, or any other language for that 
matter. All that needs to be done is to write a program that can 
output the fundamental JSON build description.


Button: A fast, correct, and elegantly simple build system.

2016-05-30 Thread Jason White via Digitalmars-d-announce
I am pleased to finally announce the build system I've been 
slowly working on for over a year in my spare time:


Docs:   http://jasonwhite.github.io/button/
Source: https://github.com/jasonwhite/button

Features:

- Correct incremental builds.
- Automatic dependency detection (for any build task, even shell 
scripts).

- Build graph visualization using GraphViz.
- Language-independent. It can build anything.
- Can automatically build when an input file is modified (using 
inotify).
- Recursive: It can build the build description as part of the 
build.

- Lua is the primary build description language.

A ton of design work went into this. Over the past few years, I 
went through many different designs and architectures. I finally 
settled on this one about a year ago and then went to work on 
implementing it. I am very happy with how it turned out.


Note that this is still a ways off from being production-ready. 
It needs some polishing. Feedback would be most appreciated (file 
some issues!). I really want to make this one of the best build 
systems out there.


Here is an example build description for DMD:

https://github.com/jasonwhite/dmd/blob/button/src/BUILD.lua

I'd say that's a lot easier to read than this crusty thing:

https://github.com/dlang/dmd/blob/master/src/posix.mak

In fact, there is some experimental support for automatic 
conversion of Makefiles to Button's build description format 
using a fork of GNU Make itself: 
https://github.com/jasonwhite/button-make


Finally, a few notes:

- I was hoping to give a talk on this at DConf, but sadly my 
submission was turned down. :'(


- I am aware of Reggae, another build system written in D. 
Although, I admit I haven't looked at it very closely. I am 
curious how it compares.


- You might also be interested in the two other libraries I wrote 
specifically for this project:


  - https://github.com/jasonwhite/darg (A command-line parser)
  - https://github.com/jasonwhite/io (An IO streams library)


Re: Argon: an alternative parser for command-line arguments

2016-03-02 Thread Jason White via Digitalmars-d-announce

On Wednesday, 2 March 2016 at 19:50:30 UTC, Markus Laker wrote:

https://github.com/markuslaker/Argon

Let me know if you do something interesting with it.

Markus


Looks nice! Can it support sub-commands (e.g., git status)? I 
suppose that can be done by passing through unused arguments and 
parsing those again.


Also, you'll get more users if it's a dub package and on 
code.dlang.org.


I was also dissatisfied with std.getopt and wrote a command line 
argument parser (competition!):


https://github.com/jasonwhite/darg

Though it's not quite feature-complete yet, it does everything I 
need it to. It uses compile-time introspection to fill out the 
fields of a struct. It can also generate the help and usage 
strings at compile-time.


Re: reggae v0.5.0: new features in the D meta-build system

2015-09-25 Thread Jason White via Digitalmars-d-announce
I rarely visit the D forums and even more rarely make a post, but 
this thread caught my eye.


I've been writing a build system in D too: 
https://github.com/jasonwhite/brilliant-build (I'm not very fond 
of the name. Naming is hard!)


It is a general build system with an emphasis on correctness.

It is a work in progress at this point, but I'm very happy with 
how it is turning out. I'm interested in your guys' thoughts on 
it.