Re: [racket-dev] (reposted from users) Noisy compiler at PLTSTDERR=info
Hi Matthew, Thanks for those thoughts. On 01/22/2015 09:11 AM, Matthew Flatt wrote: On 1 above: I've been uncertain of the best way to organize logging from the start, but the idea of grouping topics hierarchically (such as a compilation topic group) doesn't sound promising. I agree. Better perhaps would be a more relational approach. There are all sorts of cross-cutting ways one might like to filter log messages: - by package author (all the messages in code I wrote) - by package - by collection - by module - by specific subroutine? - by level (debug, info, warning etc) - by phase - by place - by predicate (e.g. regexp) on log message or log datum I don't know how to do this yet. Certainly not with the efficiency of the current logging system! I guess there's often a mismatch between the producer's original idea of organization the and consumer's eventual idea. Yes, that's a very good point. On 2 above: I'm not sure why compilation or phase 1 is special. As more and more libraries use logging, it seems like they will generate noise (from your perspective) at run time, too. On reflection, I agree with you. Phase is just one of the many axes listed above. In this sense, the info label is ambiguous. [...] My only idea, then, is that we're missing a level somewhere between debug and warning. To me it feels like there is one major divison when it comes to levels: that between errors/warnings and informational messages. Within each grouping, less important subdivisions exist. If I can turn on and off the informational messages as a group, for individual *subsystems*, I think I'm happy. So my current workaround works OK for me. Adding an extra gradation between info and debug wouldn't add much in this example. I think I'm after finer control of selecting messages on axes *other* than level/severity. -- Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] (reposted from users) Noisy compiler at PLTSTDERR=info
[Reposted from -users, since it seems likely I erred in my initial selection of that list] Hi all, Over the past few months, more and more subsystems have started logging at info level as part of regular compilation. I prefer having PLTSTDERR=info in order to catch log-info that happens at runtime, and find the compile-time log output quite distracting. My current workaround is to set PLTSTDERR=info warning@cm warning@compiler/cm warning@module-prefetch warning@setup/parallel-build warning@cm-accomplice warning@online-check-syntax but this is quite verbose, and as subsystem log messages come and go I'll need to keep updating it. Could one of the following ideas be worth exploring? 1. Have a superlogger for these (and other?) compile-time-ish loggers, so that I could write info warning@compilation, excluding the noise in one fell swoop 2. Have a phase number associated with logging, so I could say info@0 warning@1, or similar -- Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] regexp.c and lookahead
Hi all, At the moment, when regexp.c runs out of buffered lookahead during a regexp-try-match, it peeks a few bytes. However, it looks like it will never peek *fewer* than 16 bytes (unless eof occurs before then). I have written the package incremental-input which lets a blocking read (e.g. read-json) be fed input as it becomes available, event-style. When testing using read-json from the json collect, I find that it blocks unnecessarily even though a complete input is available. See https://github.com/tonyg/racket-incremental-input/blob/master/incremental-input/main.rkt#L148-L157 to see how it manifests. Deleting any of the whitespace in the byte-string on line 147 causes an unnecessary suspension. The proximal problem is the greedy lookahead buffering in regexp-try-match. Would it be possible for regexp.c to be satisfied with a partially full lookahead buffer, so long as it is long enough to properly evaluate the regexp under consideration? The specific regexp-try-match being used here is just ^], which needs just one byte of lookahead. Alternatively, perhaps I'm overlooking something simpler. Perhaps there's something I can do in (make-wrapper) with progress-events or similar, to convince regexp.c to work with what it has before asking for more? After all, (read-json) at the REPL seems to detect the end of a JSON term without gratuitous whitespace or eof! Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] raco make-and-run
Hi all, I have a little shell script I use frequently: #!/bin/sh raco make $1 exec racket $@ It first compiles the program I wish to run, then runs it with the arguments I've given. Should this be something that raco or racket does itself? Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] JIT buffer overflow
Seen just now while make CPUS=7 on racket git rev 1f1d1a38aae9f4994f76f69948f1feaca73ba57f: raco setup: 2 rendering: pkgs/syntax-color-doc/syntax-color/syntax-color.scrbl JIT buffer overflow: 0x7f804194064c [0x7f804193f020,0x7f8041940648] (1)!! Makefile:52: recipe for target 'plain-in-place' failed I ran make CPUS=7 again and the build seemed to continue through to successful conclusion. We'll see how things are once I start using the built racket... Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] make clean target for top-level makefile?
On 03/11/2014 01:50 PM, John Clements wrote: I just had a problem compiling Racket which was successfully resolved by axing the racket/build subdirectory. Would in make sense to have a “clean” target for the top-level makefile that does this automatically? As an aside, without wishing to comment on possible makefile changes, I've been using git clean -fxd to sterilize a built checkout with good results for a few years now. (Of course, it wipes out ALL untracked files and directories, not just the build/ directory.) Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Recompiling on wrong version for compiled code
Hi all, Is there some reason the compiler can't either (a) ignore or (b) replace the outdated .zos when it comes across a situation like the following? compiled/html-utils_rkt.zo::0: read (compiled): wrong version for compiled code compiled version: 5.90.0.5 expected version: 5.90.0.9 context...: standard-module-name-resolver standard-module-name-resolver standard-module-name-resolver I just ran into this issue with a webservice I run: I had updated Racket but forgotten to blow away the compiled .zos of the webservice, so when I restarted the service, it repeatedly failed until I went and deleted the .zos by hand. Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] A story of a package split
Hi all, Matthias asked me to write a few words about an experience I had splitting a large repository of code up into smaller repositories and then building a mechanism to tie them together again. == A short story == Once upon a time, RabbitMQ (www.rabbitmq.com) was held in a single, monolithic Mercurial repository, including the server, the Java client library, the .NET client library, the Erlang client library, the protocol codec compiler, the documentation, adapters for other related messaging protocols, and so on. We decided for various reasons to split the monolithic repository into separate repositories. The approach we ended up taking was to have a single repository, the umbrella, which included a Makefile and a handful of scripts which checked out, updated, compiled etc. a number of other repositories from various places. You can still see the umbrella today here: http://hg.rabbitmq.com/rabbitmq-public-umbrella/file/default The workflow for someone working on RabbitMQ is now: 1. Check out the umbrella, and `cd` into it. 2. Run `make checkout`. 3. Run `make`. 4. Edit, compile, debug, commit and push in the subdirectories resulting from step 2. 5. Occasionally run `make update` in the umbrella. (There's also some ugly makefile machinery to do cross-subrepository dependency tracking to let `make` in a subrepo recompile just the right things. Mostly.) Personally, I frequently use a script, `foreachrepo` (git variant attached) that lets me operate on all repositories found under the umbrella at once. For example, $ foreachrepo pwd would tell me where all the checkouts live, and $ foreachrepo git status would show me their status. When a configuration is found that works nicely and is to be released, a tag is made across all the currently-checked-out repositories: $ foreachrepo git tag my_release_2.3.4 $ foreachrepo git push --tags The split into completely separate repositories, linked informally by action of a script, worked out well for RabbitMQ, and the RabbitMQ project seems to be living happily ever after. == Comment == The problem addressed here is *configuration management*. RabbitMQ takes a very loose approach to configuration management, where individual modules evolve independently and are only connected to each other by happening to be in sibling directories within the umbrella. Tags are used to take a snapshot of a group of repositories at the same time. Another approach to configuration management uses an explicitly *versioned* manifest, where an umbrella repository names other repositories *and specific versions* of their contents to pull into scope. This is taken by systems like rebar, and is essentially how git submodules work. You could frame the contrast between the two by saying that the RabbitMQ approach is essentially *optimistic*, freezing configurations only when needed, and with occasional frankenconfigurations (when you `git pull` one subrepo but not one of its siblings) a risk during development, whereas the `git submodule` approach is *pessimistic*, keeping configurations frozen until explicitly moved forward into the next frozen configuration. An intermediate form could be imagined, where the Makefile checks out specific versions or branches but otherwise leaves them free to evolve in a way `git submodule` prohibits. Vincent has recently run into issues of configuration management: he wishes to assemble a specific collection of packages at specific versions to run a particular application (namely, some benchmarks). Others on this list do similar things, assembling specific versions of libraries into complete applications. I think it's interesting that both releasing applications and releasing the Racket system itself have this problem of describing a collection of related packages. Cheers, Tony #!/bin/sh for gitdir in $(find . -type d -name .git) do repo=$(dirname $gitdir) echo = $repo (cd $repo; $@) echo done _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] A story of a package split
On 08/13/2013 04:42 PM, Nick Shelley wrote: Can you elaborate on your intermediate form? I don't understand how git submodules prohibit or restrict submodule evolution. The only difference I see with the submodule approach is that it requires an extra commit to update the submodule versions (and subsequently a pull followed by a submodule update in other clones), whereas the makefile approach only requires a 'make update' in the umbrella clones. Is there something else I'm missing? Perhaps you're right. Maybe my impression that git submodules are frozen is out-of-date. It has been a while since I used them. Perhaps they're first-class checkouts that can be manipulated independently of their... supermodule. If that's true, perhaps they're an OK way of specifying a versioned subrepo manifest. Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] A story of a package split
On 08/13/2013 04:49 PM, Tony Garnock-Jones wrote: Perhaps you're right. Maybe my impression that git submodules are frozen is out-of-date. It has been a while since I used them. Perhaps they're first-class checkouts that can be manipulated independently of their... supermodule. If that's true, perhaps they're an OK way of specifying a versioned subrepo manifest. The Makefile-based approach doesn't suffer some of the gotchas wrt submodules listed at pages like http://blogs.atlassian.com/2013/03/git-submodules-workflows-tips/. It's generally just a slightly looser coupling between the umbrella repo and the child repos. Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] A story of a package split
On 08/13/2013 05:05 PM, Nick Shelley wrote: I was mainly asking about the intermediate form because it seems like it could be useful, but I didn't understand how it would work. One major difference I've just spotted is that git submodules are tied to particular commit IDs, leading to detached-HEAD checkouts, where a looser approach can check out subrepos on real branches. Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] raco pkg install should default to linking for filesystem packages
Hi all, When I have a directory foo/, containing a single-collect package, and I install it with raco pkg install foo/, it seems to copy the contents of the directory, meaning local edits are not found by the system. Instead I have to raco pkg install --link foo/. I would prefer it if --link were the default. Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] package name restrictions - why?
Hi all, Package names are restricted as follows, per the documentation: a package name — a string made of the characters a through z, A through Z, 0 through 9, _, and -. Why does this restriction exist? Programmers never see package names when using (require); Operators see package names only when they use raco pkg; otherwise they're just strings, surely? Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] package name restrictions - why?
On 07/31/2013 02:11 PM, Jay McCarthy wrote: The restriction is primarily because they appear in URLs as single path segments. Could we then widen the restriction to be [-A-Za-z0-9._~!$'()*+,;=], following http://tools.ietf.org/html/rfc3986#section-3.3 (but disallowing percent-escaping)? I ask because I find the occasional package whose git repo is named foobar.rkt/, and it's inconvenient to have to (a) make up and (b) remember a package name different from foobar.rkt for that case. Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] package name restrictions - why?
On 07/31/2013 02:42 PM, Matthew Flatt wrote: Package names show up in all sorts of other contexts, too, such as filesystem paths. Sure, but if there's something that won't work as a filesystem path, the developer finds that out awfully quickly. Note that the programming language in which we'd have to deal with encoding is sometimes not nearly as nice as Racket. For example, including a package with $ in its name as part of a `PKGS=...' argument to `make' is going to fail in lots of ways (since the `PKGS' value flows back and worth between `make' and `sh' or `cmd.exe'). Yes. (Does this say something about the PKGS= argument to make?) I don't care about $ so much, but I do care about .. If there are good reasons for a particular restriction, so be it, but the current restriction feels unnecessarily tight. Finally, there's the question of inference that `raco pkg install' and other tools perform on a string that represents a package source. Keeping the package-name grammar simple makes that inference more predictable. Do you have an example in mind? I can imagine permitting : in package names would make this difficult, but other than that, my imagination is failing me in coming up with something that'd make inference trickier. I see what you mean, and yet foobar.rkt seems like a strange name for a Racket package. In particular, I'd expect it to be inferred as a filename source, instead of a package-name source, if we ever manage to make individual files act as packages (as some have suggested). That's a problem we don't have right now. But I do have the problem of a git repo named something.rkt which I'd like to straightforwardly use as a package. Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] hex decoding?
Here's the one I've been using in racl (https://github.com/tonyg/racl, raco pkg install racl). It ignores non-hex characters but is less efficient than Stephen's version. Um, and it expects *bytes* as input, not strings. I'm not sure why, actually. (define (hex-string-bytes . strs) (define cleaned (bytes-string/utf-8 (regexp-replace* #rx[^0-9a-fA-F]+ (apply bytes-append strs) ))) (define count (/ (string-length cleaned) 2)) (define bs (make-bytes count 0)) (for ((i (in-range count))) (bytes-set! bs i (string-number (substring cleaned (* i 2) (+ 2 (* i 2))) 16))) bs) On 2013-06-09 8:14 PM, Stephen Chang wrote: There doesn't appear to be a library function for going the reverse direction but here's one way to write it up: #lang racket (define ASCII-ZERO (char-integer #\0)) ;; [0-9A-Fa-f] - Number from 0 to 15 (define (hex-char-number c) (if (char-numeric? c) (- (char-integer c) ASCII-ZERO) (match c [(or #\a #\A) 10] [(or #\b #\B) 11] [(or #\c #\C) 12] [(or #\d #\D) 13] [(or #\e #\E) 14] [(or #\f #\F) 15] [_ (error 'hex-char-number invalid hex char: ~a\n c)]))) (define (hex-string-bytes str) (list-bytes (hex-string-bytelist str))) (define (hex-string-bytelist str) (with-input-from-string str (thunk (let loop () (define c1 (read-char)) (define c2 (read-char)) (cond [(eof-object? c1) null] [(eof-object? c2) (list (hex-char-number c1))] [else (cons (+ (* (hex-char-number c1) 16) (hex-char-number c2)) (loop))]) (require file/sha1) (hex-string-bytes (bytes-hex-string #turtles)) Welcome to DrRacket, version 5.3.4.6 [3m]. Language: racket [custom]. #turtles On Sun, Jun 9, 2013 at 6:30 PM, David Vanderson david.vander...@gmail.com wrote: I'm doing some cryptography exercises that involve a lot of hex encoding/decoding. I see there's a bytes-hex-string function in file/sha1 and openssl/sha1, but I can't find a decode function. Is a hex decode function in the distribution? Thanks, Dave _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] amd64 out of memory
On 2013-04-16 11:50 AM, Jon Rafkind wrote: The PPA build has been failing the past 2 days or so for amd64 machines due to the Racket VM running out of memory, usually during 'raco setup' but not correlated with any specific collection. I have seen this behaviour after a 'raco pkg remove': during the post-removal re-setup, Racket ends up consuming all memory and swap on my machine. This machine has 32G of RAM and 40-something G of swap, so that's quite a lot of memory to fill :-) I'm not sure, but I think rerunning 'raco setup' after that has happened gets stuck doing the same thing. Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] amd64 out of memory
On 04/16/2013 12:42 PM, Neil Van Dyke wrote: On these out-of-memory conditions, you might want to look at the kernel logs for what the OOM-killer said about what processes were running, their sizes, and who it thought the culprit was. It was *definitely* Racket: I was watching htop at the time. Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] amd64 out of memory
On 04/16/2013 03:50 PM, Matthew Flatt wrote: Meanwhile, I've been meaning to make `raco pkg remove' setup only the same collections that were setup for `raco pkg install' (minus any that are gone after the uninstall), and I've pushed that change. I don't expect that the change fixes the underlying problem that you're seeing. Thanks, that's great! I'll update to master and let you know if I spot it again. Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Performance regression testing?
Hi all, I have a speculative change to match that I'd like to propose, but before I do I'd like to make sure it has no negative performance impact. How do people go about checking for performance regressions in core racket collections? Is there a standard benchmark suite? Regards, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Bug in Racket's I/O?
Hi all, I think I've found a bug in Racket's I/O. Please interpret the following text as relating to the attached tarball with a small example of the problem. The problem manifests for me on both platforms I've tried, OS X and Linux. When `sync`ing on `read-bytes-evt` at the same time as on `alarm-evt`, it seems that if the `read-bytes-evt` would have come ready but the `alarm-evt` is the one actually chosen, the input from `read-bytes-evt` gets corrupted. This only seems to happen if a single `read-bytes-evt` instance is reused! If a fresh `read-bytes-evt` is constructed every time, the input stream remains uncorrupted. To see this, run `racket buggy-server.rkt 0`, which uses a single `read-bytes-evt` instance but does not use any `alarm-evt` instances. In another terminal, run `./run-test-osx.sh` (or `./run-test-linux.sh`, as appropriate). The program `./run-test-osx.sh` should terminate without producing any output. Now, kill off the instance of `buggy-server.rkt`, and start another, this time by running `racket buggy-server.rkt 1`. This instance uses both a single `read-bytes-evt` instance and an `alarm-evt` instance. Now, running `./run-test-osx.sh` should still produce no output, but in fact on most runs, some output (from diffing the input against the output of the socket) is produced, meaning that the echoed output differed from the input. Regards, Tony io-bug.tar.gz Description: GNU Zip compressed data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Bug in Racket's I/O?
Sorry, I should have filed a bug report instead of mailing the list. I'm filing the report now. On 2012-06-20 8:22 PM, Tony Garnock-Jones wrote: Hi all, I think I've found a bug in Racket's I/O. Please interpret the following text as relating to the attached tarball with a small example of the problem. The problem manifests for me on both platforms I've tried, OS X and Linux. When `sync`ing on `read-bytes-evt` at the same time as on `alarm-evt`, it seems that if the `read-bytes-evt` would have come ready but the `alarm-evt` is the one actually chosen, the input from `read-bytes-evt` gets corrupted. This only seems to happen if a single `read-bytes-evt` instance is reused! If a fresh `read-bytes-evt` is constructed every time, the input stream remains uncorrupted. To see this, run `racket buggy-server.rkt 0`, which uses a single `read-bytes-evt` instance but does not use any `alarm-evt` instances. In another terminal, run `./run-test-osx.sh` (or `./run-test-linux.sh`, as appropriate). The program `./run-test-osx.sh` should terminate without producing any output. Now, kill off the instance of `buggy-server.rkt`, and start another, this time by running `racket buggy-server.rkt 1`. This instance uses both a single `read-bytes-evt` instance and an `alarm-evt` instance. Now, running `./run-test-osx.sh` should still produce no output, but in fact on most runs, some output (from diffing the input against the output of the socket) is produced, meaning that the echoed output differed from the input. Regards, Tony _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Are There More String Functions?
On 04/18/2012 03:28 PM, Eli Barzilay wrote: `string-normalize-spaces', which takes a string and a regexp for the spaces, and turns all spaces into single ones. Same principles as above. This one is getting a `#:trim?' keyword that says whether spaces at the edges should be dropped (the default) or normalized. BTW, I hate that name -- it makes the `string-' prefix looks even uglier... Any suggestions for a better name? string-compress-repeats ? Cheers, Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Any notion of .jar files for Racket?
On 03/06/2012 02:14 PM, Danny Yoo wrote: The Java folks have a notion of .jar files that pack a collection of class files into a single archive, but also make it possible to load classes directly out of .jars without an intermediate unpacking stage. .plt archives can serve a role to pack files together. Does Racket have a mechanism for loading modules out of them without unpacking? Python and Erlang both support a similar thing, fwiw. Tony _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Racket home page proposal
On 2011-12-20 4:07 AM, Robby Findler wrote: I like the idea, but I think our twitter feed and blog aren't updated enough. When we decided to put a twitter feed on the rabbitmq.com homepage, we went with simply including *any tweet mentioning rabbitmq*. This has its upside and its downside, of course, and there was a lot of discussion at the time as to whether this was appropriate, shooting ourselves in the foot, etc. By and large, it has worked out. Negative tweets are few and far between. The impression given is of a vibrant and fairly diverse community. Trickier, perhaps, with Racket, which isn't such a unique search term. Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] scheme_sema_post_all
Hi all, It looks like scheme_sema_post_all has the effect of an infinite number of scheme_sema_post calls. Is that right? If so, that's exactly what I need: would it make sense to expose the functionality, perhaps as (semaphore-post-infinite) or (semaphore-post-all)? (Context: I want to build a kind of box that starts empty, is able to be given a value exactly once, and that threads can wait on until its value is set by some other thread. Currently I'm using a channel and a newly-spun-up thread that loops in a channel-put, but it's not kill-safe I don't think, and creating a thread whenever the box is filled is quite heavy compared to doing a scheme_sema_post_all. Incidentally, the behaviour I'm after is very similar to (thread-dead-evt). I'll be using it, among other things, to hold the fatal exception that killed a thread, if any.) Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] scheme_sema_post_all
On 2011-10-22 9:41 AM, Tony Garnock-Jones wrote: (Context: I want to build a kind of box that starts empty, is able to be given a value exactly once, and that threads can wait on until its value is set by some other thread. Currently I'm using a channel and a newly-spun-up thread that loops in a channel-put, but it's not kill-safe I don't think, and creating a thread whenever the box is filled is quite heavy compared to doing a scheme_sema_post_all. Incidentally, the behaviour I'm after is very similar to (thread-dead-evt). I'll be using it, among other things, to hold the fatal exception that killed a thread, if any.) Nothing like the 20 seconds or so after a post to make one question oneself. Could it be that semaphore-peek-evt could be used to get what I need? I'll experiment. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Kill-safe, single-write, blocking box (was Re: scheme_sema_post_all)
On 2011-10-22 9:43 AM, Tony Garnock-Jones wrote: Nothing like the 20 seconds or so after a post to make one question oneself. Could it be that semaphore-peek-evt could be used to get what I need? I'll experiment. The answer is almost, i.e. no. But scheme_sema_post_all doesn't do what I want either. And I don't think having a thread issue an infinite sequence of (channel-put)s can be used either. I think I need something else. Something primitive, maybe. - If I use semaphore-peek-evt or scheme_sema_post_all, I still have a problem with kill safety, because I have to do something like: (when (semaphore-try-wait? (blocking-box-used b)) (set-blocking-box-cell! b the-value) (semaphore-post (blocking-box-ready b))) ...which might be killed between the try-wait and the post. - If I use a thread issuing an infinite sequence of channel-puts, (thread (lambda () (when (semaphore-try-wait? (blocking-box-used b)) (let loop () (channel-put c v) (loop) ...the custodian could be shut down at some point. Trying the same trick as the buffered async channels doesn't work here, because I'd need to know which thread to thread-resume when I checked the box's value, and to do that I'd need a kill-safe box that can be written into only once, which is an infinite regress. It looks like I need something like a cross between CAS and a semaphore. Perhaps I'm having imagination failure here. Is there something I'm overlooking that would get me an event to wait on until a value arrives, and that enforces that second and subsequent value-setting attempts do not succeed? (This is closely related to E's Promises and less closely related to Scheme's delay/force.) Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Thread exit status
Hi all, What do people think of the idea of threads preserving their exit status? The exit status would be either - a raised exception, or - a normally-returned value, or - nothing as yet, because the thread is still running. (thread-died-with-exception? thd) - boolean? Returns #t if thd died from a raised exception, #f if thd exited through a normal return, or raises exn:fail:contract if thd hasn't exited yet. (thread-exception thd) - (or/c exn? false?) Returns the exception that killed thd, #f if the thread returned normally, or raises exn:fail:contract if thd hasn't exited yet. (thread-result thd) - any? Returns the value that thd's thunk returned, or raises exn:fail:contract if either thd hasn't exited yet or thd died from a raised exception. The existing (thread-dead?) and (thread-dead-evt) would be used unchanged. (Hmm - if a thread was killed, what would (thread-died-with-exception?) return?) Would it be difficult to implement? It'd require a change to the C code. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Thread exit status
On 2011-10-22 11:14 AM, Jay McCarthy wrote: I don't think any change is necessary, because this can be done as a library: https://github.com/jeapostrophe/exp/blob/master/ethread.rkt I was hoping for something that works for threads created using (thread) as well. Using either your approach or something like https://gist.github.com/1306106 works well for programs where you control all the threads, but I wonder about whether it composes well with other code. Maybe it doesn't matter as much as I think, though. (Incidentally, there are deadlocks lurking in (update-thread!), but I think those can be worked around. One way is via the gist above; another is using the kill-safe single-write box I mentioned earlier today.) Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Kill-safe, single-write, blocking box (was Re: scheme_sema_post_all)
On 2011-10-22 11:42 AM, Matthew Flatt wrote: I think you could get this behavior by creating a manager thread when you create the new kind of box. If threads are too heavyweight, though, you can get the effect of a primitive by using `ffi/unsafe/atomic'. Of course! Using a thread to manage the cell is straightforward and will obviously work. I guess my imagination failed me indeed! Thanks for the pointer to ffi/unsafe/atomic. I'll see if something interesting can be done with that, as well. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Thread exit status
On 2011-10-22 12:02 PM, Jay McCarthy wrote: Eh, I think imposing this on all the threads would be a bad idea because it would increase memory usage unexpectedly for a lot of programs, I would guess Only those holding references to their threads well beyond the lifetime of said threads. Is there a reason why that might be common? If it really is a problem, though, then I wonder if keeping the exit value on successful conclusion of the thread could be dispensed with. Erlang doesn't do that, for instance. It'd still be useful to keep the killing exception, if there is one, though. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Thread exit status
On 2011-10-22 5:10 PM, Eli Barzilay wrote: I don't see how why that cannot be built on top of `delay/thread'. Because 1. the threads may not have been started via delay/thread, and 2. there is no way to get from (current-thread) to the promise. My best bet so far seems to be something along the lines of https://gist.github.com/1306106, perhaps simplified to only deal with exceptions. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] async-channels use a manager thread
Hi all, Why does the implementation of async-channels use a manager thread internally? Is it for kill-safety? Some other reason? Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] online check syntax deadlock: can you help?
Is that expected? It looks like the OS is preventing not-full-fledged-app-things from opening windows and handling input. This is OS 10.6,8. It probably needs to run from inside the .app folder. OS X uses the full path to the binary to figure out the name to use to dispatch events, or something similarly odd! I've tried a couple of quick experiments along the lines of $ /Applications/Racket v5.1.3/DrRacket.app/Contents/MacOS/DrRacket -W info -l- drracket but that binary doesn't like the -W switch. [...] Aha, removing the DrRacket binary from the Contents/MacOS directory and symlinking plain racket there as DrRacket instead works for me: $ mv DrRacket DrRacket.moved-out-of-the-way $ ln -s /Applications/Racket\ v5.1.3/bin/racket DrRacket $ /Applications/Racket v5.1.3/DrRacket.app/Contents/MacOS/DrRacket -W info -l- drracket I launch that all from an xterm in X11, though, so if it doesn't work for you that's an interesting clue that Terminal.app is doing something weird perhaps. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Search by example (was Re: Roogle?)
On 2011-08-05 12:13 AM, Robby Findler wrote:
So I wonder if anyone has a positive experience with this kind of
searching in an in anger kind of setting?
Not with search by type, but at the risk of derailing the thread, I have
had quite a few positive experiences with search by example.
For instance, many's the time I've been unable to remember the name of
the selector that tells me whether a string starts with some other
substring in Smalltalk. So I open the method finder, and enter
'abcde'. 'abc'. true
for the first and second arguments and the expected result,
respectively, and start it ticking, and it answers with suggested
methods such as
'abc' 'abcde' -- true
'abcde' 'abc' -- true
'abcde' beginsWith: 'abc' -- true
'abcde' includesSubString: 'abc' -- true
and so on. See attached picture. Clicking on the beginsWith: entry in
the results shows me all the classes implementing that method, in a
list. Clicking on an entry of that list opens up the relevant source code.
It has been useful to me on more than one occasion (e.g. entering
'cbabad'. $a. 2 gives #occurrencesOf:).
There's nothing magic about Smalltalk either. It's implemented as a
giant hackish crock rather than as anything remotely viewable as elegant
:-) The implementation simply has a list of safe methods that it runs
through in order, collecting non-erroring results that compare equal to
the expected output, which it then displays in a list.
Something similar could work for Racket. Useful for students and others
learning or transitioning to the language, perhaps.
You could even use contracts, perhaps: check each given argument against
the positional arguments of a candidate's contract, check the given
result; you don't even have to run the contracted procedure, which could
after all be dangerous, and thus you get to avoid some of the
crockishness of the Squeak implementation.
(You could also, in both Squeak's and this imagined Racket solution,
permit additional examples to be given to narrow the field after some
first candidate is found. #('abcde' 'xyz' false), for instance, for the
original example.)
Cheers,
Tony
attachment: Screen shot 2011-08-06 at 3.01.24 PM.png_
For list-related administrative tasks:
http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Search by example (was Re: Roogle?)
[Oops - forgot to reply to all.] To build a sketch of this, I've found module-exports, dynamic-require and friends, but I think I'd need procedures along the lines of function-contract? : any - boolean function-contract-domain-contracts : function-contract? - (listof contract?) function-contract-range-contracts : function-contract? - (listof contract?) I see contract?, flat-contract? etc, but I don't see anything that could let me take apart a contract to use its pieces. Is there any such thing? (Also there are, as usual, interesting wrinkles with search-by-example with higher-order functions.) Regards, Tony On 2011-08-06 3:13 PM, Tony Garnock-Jones wrote: Not with search by type, but at the risk of derailing the thread, I have had quite a few positive experiences with search by example. _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Search by example (was Re: Roogle?)
On 2011-08-06 4:41 PM, Robby Findler wrote: But I think in Racket we have better ways of avoiding such danger (namely using the sandbox library or the pieces it builds upon). In general, in fact, you'd have to do that anyways, since a contract can be some arbitrary predicate (and don't forget do-dads like #:pre and friends). OK - so rather than using the contracts as a kind of executable documentation, just go ahead and try out each possibility in the sandbox? Not a bad approach. Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Search by example (was Re: Roogle?)
On 2011-08-06 8:25 PM, Eli Barzilay wrote: *However*, there's an awful lot of functions to try -- going over them will make a very slow search... There's not *that* many. How long does 100,000 function calls take? I'd imagine you'd search the exports from racket/base before looking further afield, as well. Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Racket on Rockets
On 2011-07-27 4:01 PM, Sam Tobin-Hochstadt wrote: If you have a sufficiently powerful inspector, you can traverse any structure. In principle, you can even traverse closures this way, but no inspector with the needed power exists. See `struct-vector'. That sounds fantastic! Especially the traversal of closures. What kinds of dark alchemy would I have to engage in to get hold of the necessary inspector? Would such an inspector be able to represent the structure of chaperoned and impersonated values accurately? Finally, what about the other direction? I notice there's no obvious `vector-struct`. Is there a general way of moving from the description of a structure to the structure itself? Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Racket on Rockets
On 2011-07-27 4:17 PM, Sam Tobin-Hochstadt wrote: No such alchemy exists, it's just intended as part of the conceptual framework. Would it be fair to say that were such a thing to come into existence, the VM would need to be changed as part of that work? No, `struct-vector' uses the chaperoned accessors, but doesn't represent their structure. OK. And there's nothing else that can expose that structure? No. OK. Thanks, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Inline caching (was Re: my '312' this semester, how we compare to others)
On 2011-05-04 12:04 PM, Matthias Felleisen wrote: I still believe that the Java implementation (just under 1s without their 'Google' contract) benefits from typed dispatches. Maybe it does, but it's almost certain that it is benefiting from inline caching at send sites (i.e. dynamic type information) much more than it will be benefiting from static type information. A quick-and-dirty comparison of raw send performance on my Mac: Language Virtual machine Nanoseconds per send Java Hotspot 64-bit 1.6.0_24 1.4 SmalltalkCog r2382 21 SmalltalkSqueakVM 4.2.4beta1U 122 Racket Racket v5.1 ~350 Note that Cog is a JITting VM and SqueakVM is a plain (but very well optimised) interpreter. Both Cog and SqueakVM use a couple of levels of method lookup cache. A simple experiment I just performed suggests that a monomorphic inline cache hit can reduce the time needed for a send in Racket from 350ns to around 60ns, which is a massive win. I've attached the program I used to measure this, FWIW. (Run it using command-line Racket, not DrRacket: I got some *very* weird timing artifacts out of DrRacket during this experiment!) The question, then, is: how do we implement MICs or PICs using Racket's macro system? Each send site needs to expand into - a piece of global state - a test involving that global state - a possible update to that global state Hypothesising some kind of (let-static) form that introduces a lexically-scoped piece of global state, this kind of thing might Just Work to provide a speedup of almost six-fold on almost-monomorphic send-heavy code: (define-syntax cached-send (syntax-rules () ((_ obj msg arg ...) (let-static ((bc (box #f)) (bm (box #f))) (let* ((tmp obj) (cls (object-ref tmp))) (if (eq? (unbox bc) cls) ((unbox bm) tmp arg ...) (let-values (((method _) (find-method/who 'send tmp 'msg))) (set-box! bc cls) (set-box! bm method) (method tmp arg ... Regards, Tony #lang racket (require racket/private/class-internal) ;; An ordinary Racket class. (define a% (class* object% () (super-new) (define/public (op x) (+ x 1 ;; Representation of a trivial vtable. (struct ob (vt state) #:transparent) ;; A simple vtable providing a single method named op. (define (b%-vt selector) (case selector ((op) (lambda (self x) (+ x 2))) (else (error 'dnu ;; A simple class, using b%-vt as its behaviour. (define (b%) (ob b%-vt 'no-state)) ;; An uncached send to a struct ob. (define-syntax unmemo-send (syntax-rules () ((_ obj msg arg ...) (let ((tmp obj)) (((ob-vt tmp) 'msg) tmp arg ...) ;; A quasi-cached send to a struct ob. ;; ;; A real cache would have per-send-site state rather than a single ;; (!) global variable. (define *memo-class* #f) (define *memo-method* #f) (define-syntax memo-send (syntax-rules () ((_ obj msg arg ...) (let* ((tmp obj) (cls (ob-vt tmp))) (if (eq? *memo-class* cls) (*memo-method* tmp arg ...) (let ((method (cls 'msg))) (set! *memo-class* cls) (set! *memo-method* method) (method tmp arg ...))) ;; Test objects. (define a0 (new a%)) (define b0 (b%)) ;; Syntax: (measure-ns exp) ;; ;; Expands to an expression that repeats exp NREPEATS times, ;; measuring the elapsed time, and returns the number of nanoseconds ;; of CPU time used *per iteration*, excluding any GC time. (define NREPEATS 500) (define-syntax measure-ns (syntax-rules () ((_ exp) (call-with-values (lambda () (pretty-print `(measuring exp)) (time-apply (lambda () (do ((i 0 (+ i 1))) ((= i NREPEATS)) exp)) '())) (lambda (results cpu real gc) (/ (* 10.0 (/ (- cpu gc) 1000.0)) NREPEATS)) ;; Main program. ;; Measure the time for a null measure-ns loop first, then measure the ;; operations of interest, subtracting the null-time overhead ;; measurement from each to get an estimate of the time taken for the ;; interesting operation. (let ((null-time (measure-ns 123))) (define (report-on t) (let ((name (first t)) (ns/op (second t))) (write (list name (- ns/op null-time))) (newline))) (for-each report-on `( ;; Report on the loop overhead for sanity checking. (constant ,null-time) ;; How long does a plain Scheme addition operation take? (simple-add
Re: [racket-dev] Inline caching (was Re: my '312' this semester, how we compare to others)
The attached (highly experimental) patch seems to improve the performance of normal sends (in the case of a cache hit) by roughly 100% - 150%. The difference between this mere factor of two improvement and the factor of six-through-ten I was seeing earlier is, I speculate, related to object-ref taking a lot of time. Interestingly, the performance of (send something message) is, with this patch, seemingly roughly en par with the performance of generics captured using generic and invoked using send-generic. I haven't yet tried running a full racket system with this patch applied. I wonder if it makes a difference to the interactive performance of DrRacket? Major weaknesses of the approach: - overhead on cache miss is unknown - not safe-for-space yet (weak boxes are immutable) The approach should generalize easily to polymorphic inline caches. Regards, Tony diff --git a/collects/racket/private/class-internal.rkt b/collects/racket/private/class-internal.rkt index d00ef10..0a96cba 100644 --- a/collects/racket/private/class-internal.rkt +++ b/collects/racket/private/class-internal.rkt @@ -3734,19 +3734,30 @@ (let () (define (do-method traced? stx form obj name args rest-arg?) - (with-syntax ([(sym method receiver) - (generate-temporaries (syntax (1 2 3)))]) + (with-syntax ([(sym in-object in-class state method receiver) + (generate-temporaries (syntax (1 2 3 4 5 6)))] + [*cached-state* +(syntax-local-lift-expression + (syntax (cons #f #f)))]) (quasisyntax/loc stx (let*-values ([(sym) (quasiquote (unsyntax (localize name)))] -[(method receiver) - (find-method/who '(unsyntax form) - (unsyntax obj) - sym)]) + [(in-object) (unsyntax obj)] + [(in-class) (and (object? in-object) (object-ref in-object))] + [(state) *cached-state*] +[(method) +(if (and in-class (eq? (car state) in-class)) +(cdr state) +(let-values ([(m r) + (find-method/who '(unsyntax form) + in-object + sym)]) + (set! *cached-state* (cons in-class m)) + m))]) (unsyntax (make-method-call traced? stx - (syntax/loc stx receiver) + (syntax/loc stx in-object) (syntax/loc stx method) (syntax/loc stx sym) args _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Inline caching (was Re: my '312' this semester, how we compare to others)
On 2011-05-04 6:54 PM, Asumu Takikawa wrote: Wow, impressive! I've been benchmarking with the DrRacket interactive tests already for contracts, so I can run my test driver and get some numbers for that. That'd be great. I mean, it'll probably just break, but if it doesn't... it'd be interesting :-) Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] [racket-bug] all/11365: syntax definitions in interactions -- namespace mismatch: errortrace-key
Eli Barzilay wrote: * If it's OSX, and no `git' was found in the path, then try /opt/local/bin/git. This looks like the common place to get it installed on OSX (please correct me if it's wrong). Homebrew (and a simple ./configuremake install) will put it in /usr/local/bin. It might be worth checking there too, if /usr/local/bin isn't on the default OS X path. Regards, Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Possible build platform for ARM
Noel Welsh wrote: Also, if someone had an accessible ARM device any crazy people who were developing, e.g., assemblers, on Racket would be able to target that platform (and hence iPhone, iPad, and Android devices). You know qemu's ARM emulation is fairly complete, right? Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Possible build platform for ARM
Neil Van Dyke wrote: For testing, I think you need not just the target processor but also various OS and chipset/firmware stuff. Indeed. It's a thorny issue. The Openmoko phone, for instance, had a custom configuration of qemu that simulated the GSM radio, the GPS, the sound chip etc etc. Qemu comes with a configuration that's able to boot some generic ARM Debian, I think, which could make using Racket on the platform quite palatable... Tony _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
