Re: Guile 3 and wip-elisp/Emacs

[Christopher Allan Webber]

Christine Lemmer-Webber  writes:

> Ludovic Courtès  writes:
>
>> Hello!
>>
>> Christine Lemmer-Webber  skribis:
>>
>>> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
>>> again, making some naming adjustments for myself and a couple of
>>> adjustments having talked to Robin.
>>>
>>> If nobody objects, I'd like to merge this into main.  Maintainers, if
>>> you have any objections, speak now or forever hold these commits!
>>
>> I haven’t looked at the branch, but I think it’s great to see it live
>> and it’s great if it can be merged!
>
> I just compiled the rebased version and will be playing with it little
> bits over the next few days to make sure it's reasonably good.

Well, *something* isn't ready to go:

scheme@(guile-user)> (+ 1 2)
$3 = 3
scheme@(guile-user)> ,L elisp
Happy hacking with Emacs Lisp!  To switch back, type `,L scheme'.
elisp@(guile-user)> (+ 1 2)
$4 = 3
elisp@(guile-user)> (cons 'foo '())
$25 = (foo)
elisp@(guile-user) [10]> 

Hoo, 10 levels deep!

>> Some things to pay attention to before merging to ‘main’, since it
>> corresponds to the current 3.0 stable branch:
>>
>>   • Make sure no backward incompatibilities are introduced in
>> preexisting modules;
>>
>>   • Make sure the ABI of libguile-3.0.so and that of public modules
>> is unchanged, or is changed compatibly;
>
> There are, I think, two commits that could use review, but I am NOT the
> right person to do this.
>
>   4e96211eb666751b8666beb918bf3108aa1c725b intern arbitrary constants
>   433fc448ddb018767906f8808203c9668c68cd83 multiple obarrays

Re: guile 3 desires: guile-3-0, and debugging

[Christopher Allan Webber]

Matt Wette writes:

> Here are a couple desires:
>
> 1) more cases for cond-expand, in case 3.2 has items 3.0 does not (e.g., 
> srfi-199)
>
> 2) better debugging.  
>Maybe I'm not doing it right, but I struggle in this area: I mostly resort 
> to printing.
>For example, add scheme level hook, or command arg, to turn off 
> optimization.

You aren't alone... read this thread:

  https://lists.gnu.org/archive/html/guile-user/2017-05/msg00068.html

And Andy's suggestion of what we need to do to make it better:

  https://lists.gnu.org/archive/html/guile-user/2017-05/msg00070.html

Re: guile 3 update

[Christopher Allan Webber]

Andy Wingo writes:

> Hi :)
>
> On Sun 22 Oct 2017 15:22, Christopher Allan Webber <cweb...@dustycloud.org> 
> writes:
>
>>  - Could native code compilation also be a step towards WASM, assuming
>>they lend us their GC?
>
> Regarding this question: yes!  Specifically with the "GC" proposal
> (which in reality is much more:
> https://github.com/WebAssembly/gc/blob/master/proposals/gc/Overview.md)
> I think that we will approach WASM's semantic level.  There are some
> open questions of course, like how to deal with the stack and delimited
> continuations, how to do bignums with overflow, etc; but yeah we're
> getting there.
>
> Andy

This is very exciting!

Re: Weak tables harmful to GC?

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Ludovic Courtès writes:
>>
>>> Also, it no longer displays the pathological behavior shown in
>>> <https://bugs.gnu.org/28590>.
>>>
>>> Of course, even better if people could test the two patches and confirm
>>> that it works for them.
>>>
>>> Then if there are no objections I’d like to merge them in ‘stable-2.2’.
>>
>> Sounds great indeed, but it didn't apply to master or stable-2.2 for me?
>
> Really?  The two patches should apply to stable-2.2, though you need to
> apply them in the right order (I have it applied over
> 80696023620eae12f9b2f167aee834f632a32739.)
>
> Ludo’.

Huh?  What object is this?  I don't see it in my git repo.

This is the latest commit I see to stable-2.2, which is also what
Savannah sees:

  
https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.2=a74d4ee4f6e062ff640f2532c9cfc9977bb68a49

Re: Weak tables harmful to GC?

[Christopher Allan Webber]

Ludovic Courtès writes:

> Also, it no longer displays the pathological behavior shown in
> .
>
> Of course, even better if people could test the two patches and confirm
> that it works for them.
>
> Then if there are no objections I’d like to merge them in ‘stable-2.2’.

Sounds great indeed, but it didn't apply to master or stable-2.2 for me?

Re: Weak tables harmful to GC?

[Christopher Allan Webber]

+  else
+{
+  /* The move to BDW-GC with Guile 2.0 introduced some bugs
+ related to weak hash tables, threads, memory usage, and the
+ alloc lock.  We were unable to fix these issues
+ satisfactorily in 2.0 but have addressed them via a rewrite
+ in 2.2.  If you see this message often, you probably want
+ to upgrade to 2.2.  */
+  fprintf (stderr, "guile: warning: weak hash table corruption "
+   "(https://bugs.gnu.org/19180)");
+  len = 0;
+}

Guess reverting this patch means this comment also should be amended!

Re: guile 3 update

[Christopher Allan Webber]

Andy Wingo writes:

> Thoughts welcome if I have forgotten something.  Cheers :)

Super exciting, Andy!  I only have two thoughts:

 - What an exciting future we have ahead of us!
 - Could native code compilation also be a step towards WASM, assuming
   they lend us their GC?

Woo woo, keep up the good work! :)

Re: Weak tables harmful to GC?

[Christopher Allan Webber]

Ludovic Courtès writes:

> I’ve come to the conclusion that the 2.2 weak-table implementation
> strategy cannot work efficiently with libgc.
>
> I’m also skeptical (perhaps that’s also because I’m insufficiently
> informed, tell me!) about the “open-addressed” strategy that is used.
> To me, it’s necessarily less space-efficient than a regular hash table
> with chaining since we always have at least 10% more weak entries than
> the number of actual entries in the table (and in practice it’s usually
> much more than 10% AFAICS, because of the gap between subsequent sizes.)
>
> All in all, given that these issues are very likely causes of the
> execution time and memory consumption issues that plague the compiler
> (where we have huge symbol and source property tables), I’m in favor of
> switching back to the 2.0 implementation of weak hash tables.  That can
> be done in an API-compatible way, I think.

If ~reverting this one thing will fix our current major problems
(especially if api compatibility can be preserved), I would say it's a
good idea for now.  Presumably including a re-iteration of the weak hash
tables that include the new work could be still done in the future once
these problems are solved?

Thanks for all your hard work on this Ludo'!

Re: Wisp as shipped language in Guile?

[Christopher Allan Webber]

Matt Wette writes:

>> On Sep 17, 2017, at 4:22 PM, Arne Babenhauserheide  wrote:
>> 
>> 
>> Matt Wette  writes:
>>> Looks promising, especailly if the footprint is small.  
>>> Can someone post the sample guix specification?
>>> I could not access from the gnunet log.
>> 
>> Do you mean the wisp package for Guix?
>
> No, I meant the guix package spec for package foo written in wisp.
>
> define-public foo
>package 
>   name "guile-wisp"
>   source : origin 
>  ...

https://dustycloud.org/blog/wisp-lisp-alternative/

Re: Static linking of Guile extensions

[Christopher Allan Webber]

Maxim Cournoyer writes:

> Hi,
>
> Today I tried investigating if was possible to build Guile so that even
> its extension modules could be built in statically rather than
> dynamically loaded. I didn't succeed; even when building Guile the way
> its described by the %guile-static package in Guix (see (gnu packages
> make-bootstrap)), I would still get the following message when attemting
> activation of readline:
>
> scheme@(guile-user)> (use-modules (ice-9 readline))
> While compiling expression:
> ERROR: In procedure dynamic-link: file: "libguilereadline-v-18", message: 
> "file not found"
>
> Interestingly a static library for readline seems to have been built:
>
> /gnu/store/xxi8scxxx8g6i9h0lm57i5kc3xwcgj5r-guile-static-2.0.14/lib/libguilereadline-v-18.a
>
> One use case would be to have readline support for a statically linked
> Guile such as used for the GuixSD init ram disk; I'm currently attemting
> to troubleshoot a problem at the init prompt (Guile) and it would be
> great to have line editing support there!
>
> Maxim

As an aside, you may want to look at 6.3 on Ian's update here:
https://lists.gnu.org/archive/html/guile-user/2017-08/msg00070.html

Maybe there can be some alignment on this work?

Re: Adding (ice-9 suspendable-ports) support to https / custom ports

[Christopher Allan Webber]

So since writing this initial email I had a conversation with Wingo and
have done a lot more research.  I'm a lot more informed about what needs
to happen, but unfortunately, I'm also very stuck.  This is unfortunate
because this is basically blocking the release of the test suite
relating to the standards work I'm doing, which was due out this
Tuesday!  So any help is *greatly* appreciated.

So here's the situation:
 - We have suspendable-ports which make asynchronous I/O work nicely and
   cooperatively, yay
 - HTTPS requests now work because we wrap underlying ports in gnutls
   using custom-binary-i/o-ports, yay
 - Oh no but the gnutls-wrapped ports aren't suspendable and block
 - Oh no the ActivityPub test suite sends out a request to a foreign
   server and the foreign server sends its own request to the test suite
   before it gives a response but the test suite is blocked on its
   initial request and oh no the whole thing is deadlocked oops, take
   that cooperative model!
 - To make things more complicated, when we call C code that calls into
   Scheme code, we can no longer abort to a prompt, which is how
   suspendable ports works.  (For more on that see Wingo's blogposts
   on delimited continuations and the Scheme/C stacks.)

Wingo gave some helpful advice on IRC:

 there are two general options.  one is, if this port is always
operated on from scheme, *and* you arrange for it to implement the
read/write functions via the scm_read / scm_write members and not the
c_read/c_write --
 then in that case, the read/write functions written in scheme can
themselves use suspendable ports, transparently blocking.
 the second option is to change the read/write functions to allow them
to return -1 when they would block, and in that case you implement the
read_wait_fd / write_wait_fd methods.
 i suspect the latter is going to be easier but i don't know

I've been looking into solutions.

1. The first option would be the nicest if it were possible; we could
   simply do the "set the port nonblocking with fcntl" dance on the
   wrapped port, and when we try to write to the wrapped port, it should
   automatically suspend.

   Unfortunately there are problems with this route afaict:
- It wouldn't be possible to use custom-binary-i/o-ports anyway,
  because scm_read/scm_write are properties not of the port instance
  itself but of its port type, as far as I can tell.  The whole
  point of custom-binary-i/o-ports is to be able to set up
  procedures on the instance, so I'm not sure how to get around
  this.
- Even if we didn't do this, the low-level port-read/port-write
  procedures in ports.c are, well, in C.  So I'm afraid we're going
  to pass through C anyway.

2. Okay, well let's add the read_wait_fd and write_wait_fd methods
   (which would really just wrap the wrapped port's file descriptors)
   and allow the existing read/write functions from the current
   tls-wrapping port thing we've got to just return -1, indicating that
   they'd like to suspend please so as to not block.  In theory, this
   would just be passed along from the underlying port.  Okay, sounds
   great!  Except... well let's look at how the current tls-wrapping
   code looks:

   #+BEGIN_SRC scheme
   (define (tls-wrap port server)
 "Return PORT wrapped in a TLS connection to SERVER.  SERVER must be a DNS
   host name without trailing dot."

   ;;  gnutls setup stuff here ***

   ;; @@: Not sure if this comment would help

   ;; FIXME: It appears that session-record-port is entirely
   ;; sufficient; it's already a port.  The only value of this code is
   ;; to keep a reference on "port", to keep it alive!  To fix this we
   ;; need to arrange to either hand GnuTLS its own fd to close, or to
   ;; arrange a reference from the session-record-port to the
   ;; underlying socket.
   (let ((record (session-record-port session)))
 (define (read! bv start count)
   (define read-bv (get-bytevector-some record))
   (if (eof-object? read-bv)
   0  ; read! returns 0 on eof-object
   (let ((read-bv-len (bytevector-length read-bv)))
 (bytevector-copy! read-bv 0 bv start (min read-bv-len count))
 (when (< count read-bv-len)
   (unget-bytevector record bv count (- read-bv-len count)))
 read-bv-len)))
 (define (write! bv start count)
   (put-bytevector record bv start count)
   (force-output record)
   count)
 ;;  Some more stuff for close, etc here 
 (make-custom-binary-input/output-port "gnutls wrapped port" read! 
write!
   get-position set-position!
   close
   #+END_SRC

   Okay, so what are the issues here?

- In order to pass along whether or not these 

Adding (ice-9 suspendable-ports) support to https / custom ports

[Christopher Allan Webber]

Hiya,

For the project I'm working on, I'll need to have (ice-9
suspendable-ports) work with https.  This is a bit more urgent than I
realized.  I'm willing to attempt the work, but I don't even know where
to start.

Could someone give me pointers as to where to begin spelunking?  I'm
guessing I have to both follow the lead of the suspendable-ports
adjustments to normal ports and also enable fnctl support for custom
ports?  Specifically, I imagine that this needs to be supported:

 (let ((flags (fcntl socket F_GETFL)))
   (fcntl socket F_SETFL (logior O_NONBLOCK flags)))

Either that or we need some generalized procedure that can either do
this to a port which does set fnctl or... I don't know what it would do
for something which doesn't need to run that operation.

Advice appreciated!
 - Chris

Re: Including sjson (formerly (ice-9 json)) and fash.scm in guile proper?

[Christopher Allan Webber]

Mark H Weaver writes:

> Hi Chris,
>
> I'm terribly sorry for the long delay on this.  For better or worse,
> I've become extremely concerned about computer security, and so I feel a
> heavy responsibility to be extra careful about code that is expected to
> parse hostile data.

No worries.  I also know you've been working hard to keep us secure in
your work on Guix and etc, and all that work is valued / appreciated!

> I was also looking for a cleaner way to express this parser, and to add
> better error reporting, while allowing flexibility for users to
> customize the Scheme representation.  Toward these ends, I ended up
> re-implementing the parser from scratch.
>
> I've attached my current draft of the new parser.  By default, JSON
> objects are represented as (@ . ) where  has dotted pairs,
> but it's easy to ask for your preferred two-element lists using
> 'make-json-parser'.

I like the interface to make-json-reader.  It looks like it would work
well by default, but it's also obvious to me how to extend it.  It also
looks far less disasterous than my attempt to be flexible on using
fashes or s-exps via an overdose of if clauses. ;)  Looks good!

> The json writer is mostly okay, but it also needs to be generalized to
> support the customizable representation (and maybe I went too far here
> with the parser, dunno).

I don't see the writer included below?

> Also, there are a few cases where it will generate invalid JSON,
> notably: if ASCII control characters are present in strings, hex
> escapes must be printed instead of the raw character, and real numbers
> cannot simply be printed using 'display' because of infinities and
> NaNs.
>
> Are you okay with this general direction?

I'm good with it!  It's a bit more complicated to read than David's
original design but it looks well formed (and I followed it well still),
and I'm confident in your reasoning behind its design.  I feel like the
flexibility of plugging in different datastructures as needed is a big
win, anyway.

Speaking of security issues, I've wondered before about someone giving a
json structure so large or deeply nested that it's hard to fit in memory
(maybe a structure that even just looks like "[[...").
Maybe a recursive limit, or some other type of limit, would be useful?
but I imagine that could complexify the design considerably, and maybe
it's not worth it.

I tested this on a few examples from
https://www.w3.org/TR/activitystreams-vocabulary/
and they all seemed to work fine.

Thanks for all your hard work on this!  Looking forward to seeing this land!

 - Chris

PS:

> ;; XXX Consider using conditions and exceptions from SRFI-34 + SRFI-35
> ;; or R6RS (they should be merged!)

Yeah I've wondered about this... I've come to the point where I needed
the kind of condition subtyping that srfi-35 and r6rs conditions
provide, but I'm not sure which one to use.  Advice?

Re: [ Porting Guile on Android 4.4 ]

[Christopher Allan Webber]

Robert Dazi writes:

> Hello,I tried to port Guile 2.2.1 on Android 4.4.

Wow nice!

> As you might know, Android is not GNU/Linux and many functions (and
> even types such as localeconv) are missing or not acting as expected

I can't comment to the rest of it, but getting Guile ported to Android
seems desirable if possible.

Re: Including sjson (formerly (ice-9 json)) and fash.scm in guile proper?

[Christopher Allan Webber]

Mark H Weaver writes:

> I wrote:
>> Most of the modifications you've made are good, but I'm very
>> uncomfortable with the use of #nil in this API.  [...]
>
> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>> Oh!  No you got it backwards, the library *was* using #nil initially,
>> and I modified it to use 'null now instead. :)
>
> Ah, my mistake.  Excellent!
>
> Having now looked more closely, I'm mostly happy with the API, except
> for one issue: I don't like the way fash support was hacked in, with the
> 'use-fash' flag and the (if use-fash [fash-code] [alist-code]) sprinkled
> around.  If this truly needs to be done within the json library itself
> (which I wonder), then I'd prefer to generalize it to support any
> dictionary data structure, and thereby remove the dependency on fashes.

I agree that it's pretty hacky.  Allowing other dictionary structures is
fine by me.

> My main concern about fashes, besides the fact that Andy hasn't yet
> proposed adding them to Guile himself, is that the implementation is
> very complex, and I'd like to achieve some degree of confidence in its
> correctness before adding it.  I'd also tend to favor adding a simpler,
> truly immutable dictionary data structure based on Phil Bagwell's HAMTs
> (Hash Array Mapped Tries) to eliminate the need for thread
> synchronization, but I'm open to suggestions.

I don't really understand enough of the field to really know what the
right direction is.  I do know that I need something that's not O(n) for
json-ld processing, though I guess one option always would have been to
read in the sexp structure and transform it before doing all that
processing.   I've long wanted a better immutable dictionary
structure in Guile though, but am open to what it would be.

> Anyway, since writing my previous message in this thread, I've started
> carefully reviewing the code, making modifications as I go.  At this
> point, my proposed modifications have become quite extensive.  So far,
> I've reworked the code to greatly reduce heap allocations, support
> arbitrary dictionary types (removing the fash dependency, while still
> allowing its use), and fix various bugs (e.g. relying on unspecified
> evaluation order, failure to handle 12-character hex escapes properly,
> producing and accepting invalid JSON in some cases, etc).
>
> I'll followup with another message when I've completed my proposed
> revisions.  Feel free to ping me if it takes more than a week.

Wow, exciting!

>>> Otherwise, I'm generally in favor of incorporating this library into
>>> Guile, after we make sure that it is robust against malicious inputs.
>>
>> Okay, cool!  The other thing is to add more specific error messages, as
>> discussed.
>
> Indeed, better error messages would be a good thing.
>
>> What examples of malicious inputs should we test against?
>
> I'm mostly trying to address that by careful code review.

Yay!  Thank you for doing it.

Re: Including sjson (formerly (ice-9 json)) and fash.scm in guile proper?

[Christopher Allan Webber]

Mark H Weaver writes:

> Hi Chris,
>
> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>> So a while ago, David Thompson submitted (ice-9 json) to Guile proper.
>> A few changes were requested, so it hadn't made it in.  In the meanwhile
>> I began using it for a number of projects.  I also added some
>> modifications and extensions: #nil became 'null for the representation
>> of null values,
>
> Most of the modifications you've made are good, but I'm very
> uncomfortable with the use of #nil in this API.  #nil is a terrible hack
> which may not even be adequate for its intended use case.  Its existence
> in any data structure is likely to cause misbehavior in other Scheme
> code that is exposed to it, because it violates a longstanding fact in
> Scheme that there is only one value that is treated as "false".  It
> would also make it difficult or impossible to port this library, and
> thus anything that depends on this library, to other Scheme systems.  We
> should not promote its use by incorporating it into new APIs.
>
> What do you think?

Oh!  No you got it backwards, the library *was* using #nil initially,
and I modified it to use 'null now instead. :)

So I think you'd be probably pretty happy!

> Otherwise, I'm generally in favor of incorporating this library into
> Guile, after we make sure that it is robust against malicious inputs.

Okay, cool!  The other thing is to add more specific error messages, as
discussed.

What examples of malicious inputs should we test against?

Re: 2.2 git users should switch to stable-2.2 branch

[Christopher Allan Webber]

Andy Wingo writes:

> Hello Guilefolk,
>
> As we did with Guile 1.8 and 2.0, I just pushed a stable-2.2 branch.
> This is the branch for Guile 2.2 development -- the branch that keeps
> a stable API and ABI during the 2.2.x series.  It's the one you want to
> be using unless you are doing development of Guile itself.
>
> If you have a checkout of Guile master, probably right now you want to
> do this:
>
>   git fetch
>   git checkout stable-2.2
>
> Assuming your default remote is the Guile repo, that will fetch the new
> stable-2.2 branch from git.sv.gnu.org, and then the "checkout" command
> will check out a new branch locally to track upstream stable-2.2.
>
> As far as a roadmap goes... I suspect the next Guile stable series will
> be 3.0 and I suspect it will feature ahead-of-time compilation to
> machine code instead of bytecode.  I think there will still be a
> bytecode backend as well for targets that are missing the machine code
> backend.  Here are some thoughts:
>
>   https://wingolog.org/archives/2016/02/04/guile-compiler-tasks
>
> I will probably start within the next month or three on "instruction
> explosion"; see that blog post for details.  It's like Abdulaziz
> Ghoulum's "incremental approach to compiler development", except
> completely in reverse :P

This is really exciting!

> Farther on I would expect a 3.0 after about 2 years or so.  There are
> many things to do.  If someone is interested in taking a task, that blog
> post has many ideas that are still valid.

I hope, eventually, I will be in the right time and place to be helpful
on this.

> Incidentally if you use Guile commercially and would like to support my
> work on it, let me know.  My employer Igalia continues to be happy with
> me working on it a day or two a week, and I work on it as a hobby
> additionally, but it would be nice to be able to devote all of my
> attention on Guile, at least for consecutive periods of a month or two
> so I can work on whole features.  I expect a general perf improvement
> for Guile 3.0 of about 4x relative to Guile 2.2, and with support, this
> can become reality sooner.  There are lots of details of course but if
> this is something that interests you, let's talk.
>
> Happy hacking, and don't forget to switch your Guile git branches!
>
> Andy

I don't know what the right answer is to it (maybe eventually there
would be a company making money off of Guix deployment that can manage
to pay for full-time Guile work) but obviously this would be nice to see
happen.

Thank you for all your leadership and hacking in Guile-land!  We
love/appreciate it!

Including sjson (formerly (ice-9 json)) and fash.scm in guile proper?

[Christopher Allan Webber]

Hello!

So a while ago, David Thompson submitted (ice-9 json) to Guile proper.
A few changes were requested, so it hadn't made it in.  In the meanwhile
I began using it for a number of projects.  I also added some
modifications and extensions: #nil became 'null for the representation
of null values, the representation got a bit easier to read for deeply
nested lists-of-dicts-of-lists-of-dicts (moved from '(@ (key . val)) to
'(@ (key val)) after some discussion with David Thompson), I added a
pretty printer, and I also added fash support, snarfing fash.scm from
Andy Wingo (I had a number of cases where I had json documents with a
*lot* of key / value pairs and I was operating on them, and being able
to read/write from a constant time datastructure as an option was
needed).

I released this as an independed library, which is even now packaged in
Guix as guile-sjson.

However, I wonder if we should package this in Guile proper.  There is
still one issue to resolve iirc, I should add more specific exception
names.

The biggest problem to me seems that we would also want to include
Wingo's fash.scm in Guile proper.  Personally, I think this would be a
big win: highly performant immutable hashmaps are desirable (and though
we have vhashes, setting an existing value keeps the old value, and are
not as fast as fashes in my experience).

So:
 - Are Guile's developers open to having an (ice-9 fash) module?
 - And should I submit (ice-9 json), with my changes?

Thanks!
 - Chris

Re: Wisp as shipped language in Guile?

[Christopher Allan Webber]

Arne Babenhauserheide writes:

> Hi,
>
>
> A few weeks ago I asked in IRC whether wisp[1] could be included with
> Guile in modules/language/wisp to allow every guile users to run wisp
> code from any Guile installation via
>
>
> $ guile --language=wisp []
>
>
> Essentially this is about making wisp as language part of the
> "batteries" of Guile. I think it currently is at a point in its
> development where that’s useful, because wisp itself shouldn’t need
> major changes (and hopefully not many minor ones either, since it only
> transforms its syntax to Scheme and then lets the compiler tower take
> over).
>
>
> I got the answer that a viable path would be to add a wip-wisp branch in
> savannah. What would be required for that? Basics I know are:
>
> - copyright assignment (who can get me the papers to sign?)
> - access to https://savannah.gnu.org/projects/guile — should I get push
>   access or rather send patches here?
>
> Did I miss something?
>
>
> Best wishes,
> Arne
>
> [1]: http://draketo.de/english/wisp

+1 to wisp in Guile proper :)

Re: "guix potluck", a moveable feast

[Christopher Allan Webber]

Andy Wingo writes:

> Christopher Webber asks about breakage due to version skew between peer
> channels and channels and Guix itself.  I think I would like to just
> ignore this problem for now: if you add channels and things break
> somehow due to an update in Guix or an update in some channel, then the
> workaround is to disable channels until developers fix things.

It isn't an impossible problem to get around anyway, at least from a
channel to channel perspective... since we're building channels around
git repositories, *eventually* we could allow a channel to include
another channel as a dependency with a specific hash.  But we can work
on that once we get things working.

Re: "guix potluck", a moveable feast

[Christopher Allan Webber]

Andy Wingo writes:

> Hi!

Hi!

> potluck.guixsd.org needs to be isolated from other hosts because it will
> load potluck.scm files from untrusted sources; we hope the sandbox works
> but we need a bit of defense-in-depth.

Well now I see the motivation behind (ice-9 sandbox) ... :)

> As I mentioned, I think it would be nice to be able to install some
> potluck packages directly from git, without requiring those packages to
> make releases and update the potluck.scm.  But until then, we can make
> it so that the source is fixed in the potluck.scm as it is with other
> Guix packages, and therefore that any update to potluck.scm in the
> source git branch registered with potluck.guixsd.org constitutes a new
> release which replaces the old one.  A developer should signal
> potluck.guixsd.org about the update via a re-invocation of "guix potluck
> add".  Maybe "guix potluck add" could remember the branch, dunno.
>
> Anyway!  The result of the "guix potluck channel-manager" is a stream of
> guix modules as a continually updated git tree -- a guix channel.  I am
> thinking that we need to rewrite these files to be more "normal" -- like
> starting with a (define-module), but a #:pure module and an appropriate
> set of imports to enforce the sandbox.  We should be able to compile
> this module, to prevent the potluck channel from slowing things down.
> So basically the channel-manager rewrites the potluck.scm files.

It sounds nice!

One challenge though... what do we do about multiple channels
introducing version skew?  (Maybe I'm abusing that term?)  This isn't
something we've dealt with before in Guix... if my channel adds
something that depends on your channel's package definition, do I
explicitly set a revision for your channel?  Otherwise else, your
channel could change as you upgrade your software version, and that
might unexpectedly break my channel...

Re: RFC: (ice-9 sandbox)

[Christopher Allan Webber]

Wow!  With this I suppose we could implement something like 
  http://mumble.net/~jar/pubs/secureos/secureos.html
?

Re: Guile-2.2 - goops slot redefinition at subclass level

[Christopher Allan Webber]

Andy Wingo writes:

> On Mon 27 Feb 2017 01:09, David Pirotte  writes:
>
>> 2-   slot redefinition at subclass level
>>
>> See bug#20423 for a full description.
>
> I think the correct solution here is to implement the slot combination
> protocol; see http://mop.lisp.se/dictionary.html#compute-slots.  This
> can be done in 2.2.  It's all in Scheme now so hopefully it will be
> easier to hack on.
>
> I do not have any time to devote to this area, however, and as it's
> possible to implement this in the stable series and it's not a
> regression, I don't think it's a release blocker.
>
> Andy

Okay, it's good to know that you're open to this being implemented as a
feature (and that there's a clear explaination of how to do it above).
Maybe in the not too distant future I can take a crack at it, as it's
something I'd love to have myself.

 - Chris

Re: [PATCH] Add preliminary versions of the R7RS libraries along with documentation and tests

[Christopher Allan Webber]

Freja Nordsiek writes:

> I've written preliminary versionf of all the R7RS libraries along with
> documentation and unit tests for most of the procedures I had to add
> (none for those that were re-exports/renames of existing procedures in
> Guile). Sorry for the bad timing (right before the 2.2 release).

Wow! :O

> I was inspired by the work in the r7rs-wip branch
> (https://git.savannah.gnu.org/cgit/guile.git/log/?h=r7rs-wip). I
> figured it would be a lot of work to get that branch up to date since
> the last commit was in mid 2014, so I decided to write just the
> libraries themselves but not work on adding any of the R7RS syntax
> that hasn't already been added by others to the master branch. I wrote
> the libraries in pure scheme and did not modify any other scheme
> modules or C code to simplify inclusion (they won't have side effects
> on other modules or code). Note that I did grab Mark H Weaver's
> implementation of the load procedure/syntax. I didn't copy more code
> because I wanted to make adding this to Guile not dependent on any
> changes to the parts in C or other scheme modules (would have taken a
> long time to sort out which changes in r7rs-wip branch depend on such
> modifications and which do not).
>
> The tests for what I added check out and the documentation compiles to
> info and html (compiling texinfo to pdf is broken on my computer for
> all projects so I wasn't able to test that).
>
> Note, I was not able to make unit tests for load, include, and
> include-ci. file-error? and read-error? are not 100% reliable, and my
> implementation of the bytevector output ports is a bit of a hack job.
> Also, I did not update cond-expand or include the R7RS syntax for
> declaring libraries. Those things are still missing.
>
> Patch is rather large.

I'm the wrong person to comment on candidate for inclusion-ness, but it
looks like impressive work.  Doing things in pure scheme is pretty cool.

Re: Guile-2.2 - goops setters should be inherited, no matter what :)

[Christopher Allan Webber]

David Pirotte writes:

>   - both the accessor and the setter should be inherited

I've run into unexpected issues where I redefined a slot and didn't
remember to set the same getter/setter/accessor.  I'd also love to see
that be an inherited.  It would lead to less goofups in my code.

But why just setters and accessors?  Why not also getters?

  scheme@(guile-user)> (use-modules (oop goops))
  scheme@(guile-user)> (define-class  ()
 (da-slot #:init-keyword #:da-slot
  #:getter get-da-slot))
  scheme@(guile-user)> (define-class  ()
 (da-slot #:init-value "moop"))
  scheme@(guile-user)> (get-da-slot (make ))
  ERROR: In procedure scm-error:
  ERROR: No applicable method for #< get-da-slot (1)> in call 
(get-da-slot #< 22180a0>)

  Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
  scheme@(guile-user) [1]> ,q

It may also be worth asking, what other slot fields get inherited?
Would #:init-keyword?  #:init-value?  #:init-thunk?  Inheriting
#:init-value and #:init-thunk both could be confusing (what does CLOS
do?).  What about #:allocation?

Re: Fixing non-reproducibility in some guile packages

[Christopher Allan Webber]

Andy Wingo writes:

> In some future (is it near or far?), the source -> compiled function
> needs additional inputs: checksums or timestamps of "build inputs" or
> so, so that when for-syntax definitions (like macros) change, users of
> those definitions will recompile.  That is a harder problem though.

Ah yeah, I've been bit by this before.  Though, I imagine we'd run into
a problem where we'd never know how to "garbage collect" anything in
~/.cache/guile/ .  "Maybe not a problem", except if you're hacking some files
constantly... :)

Maybe every guile hacker needs to get in the habit of
`rm -rf ~/.cache/guile/' though? :)

Re: guile2.2-gdbm-ffi issue

[Christopher Allan Webber]

Ugh, sorry, wrong list.

guile2.2-gdbm-ffi issue

[Christopher Allan Webber]

Hiya,

I pushed guile2.2-gdbm because it was just a couple of lines and it
seemed to work and didn't affect anything.  I made a mistake though that
I didn't realize until I ran
"guix environment --ad-hoc guile-next guile2.2-gdbm-ffi --pure".
The trivial-build-system in guile-gdbm-ffi dumps the built module into
(string-append out "/share/guile/site/2.0") ... oops!  Before I ran
--pure, the 2.0 directory was on my load path, and thus I didn't bump
into the error.  (Maybe I should have asked for review anyway...)

I'm trying to think of how to fix this.  I have a very kludgy solution
attached.  I don't feel great about it but I don't know how to signal to
the builder whether it's using guile 2.2 or 2.0 otherwise.  Thoughts?

From 711a23036417807d444729eaa778c9cadffa8646 Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Fri, 10 Feb 2017 19:24:57 -0600
Subject: [PATCH 3/3] guile-gdbm-ffi: Write to correct guile output directory.

* gnu/packages/guile.scm (make-guile-gdbm-ffi): New variable.
(guile-gdbm-ffi, guile2.2-gdbm-ffi): Use make-guile-gdbm-ffi.
---
 gnu/packages/guile.scm | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 3e8ab007b..3e62949f2 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -825,9 +825,11 @@ inspired by the SCSH regular expression system.")
 ;; There are two guile-gdbm packages, one using the FFI and one with
 ;; direct C bindings, hence the verbose name.
 
-(define-public guile-gdbm-ffi
+(define (make-guile-gdbm-ffi guile-2.2?)
   (package
-(name "guile-gdbm-ffi")
+(name (if guile-2.2?
+  "guile2.2-gdbm-ffi"
+  "guile-gdbm-ffi"))
 (version "20120209.fa1d5b6")
 (source (origin
   (method git-fetch)
@@ -848,7 +850,9 @@ inspired by the SCSH regular expression system.")
   (system base compile))
 
  (let* ((out (assoc-ref %outputs "out"))
-(module-dir (string-append out "/share/guile/site/2.0"))
+(module-dir (string-append out "/share/guile/site/"
+   ,(if guile-2.2?
+"2.2" "2.0")))
 (source (assoc-ref %build-inputs "source"))
 (doc (string-append out "/share/doc"))
 (guild (string-append (assoc-ref %build-inputs "guile")
@@ -877,7 +881,8 @@ inspired by the SCSH regular expression system.")
(compile-file gdbm.scm-dest
  #:output-file gdbm.go-dest)
 (inputs
- `(("guile" ,guile-2.0)))
+ `(("guile" ,(if guile-2.2?
+ guile-next guile-2.0
 (propagated-inputs
  `(("gdbm" ,gdbm)))
 (home-page "https://github.com/ijp/guile-gdbm;)
@@ -887,8 +892,11 @@ inspired by the SCSH regular expression system.")
 Guile's foreign function interface.")
 (license gpl3+)))
 
+(define-public guile-gdbm-ffi
+  (make-guile-gdbm-ffi #f))
+
 (define-public guile2.2-gdbm-ffi
-  (package-for-guile-2.2 guile-gdbm-ffi))
+  (make-guile-gdbm-ffi #t))
 
 (define-public guile-sqlite3
   (let ((commit "607721fe1174a299e45d457acacf94eefb964071"))
-- 
2.11.0



signature.asc
Description: PGP signature

Re: FOSDEM 2017 what a success!!

[Christopher Allan Webber]

Ludovic Courtès writes:

> It confirmed my feeling that one of the greatest things about Guile and
> Guix is the people.  Thanks for the great talks and the good time we’ve
> had!

+1.  The Guile/Guix devroom has been some of the most fun I've ever had
at a conference.  I'm sure 2018 will be great too, and I'm looking
forward to all the things that will happen in the next year! :)

 - Chris

Re: FOSDEM social dinner

[Christopher Allan Webber]

Alex Sassmannshausen writes:

> Hello,
>
> To confirm, I've now placed a reservation for Saturday 4 February at
> 19:30 at a Lebanese restaurant called Al Jannah.  I haven't been there
> before, but the menu looks diverse, the prices good and the location
> relatively central.
>
> https://www.tripadvisor.com/Restaurant_Review-g188644-d2039550-Reviews-Al_Jannah-Brussels.html
>
> Address:
> Rue Blaes 59, Brussels 1000, Belgium
>
> There is a direct tram line from ULB to Louise, and then it's a 5-10 min
> walk.  From there it's probably around 15 min walk to the central
> station where there are trains to Antwerp.
>
> List of attendees:
> Leo, Catonano, Amirouche, Efraim, Tomáš, Thomas, Tobias, Christopher
> Baines, Manolis, Ludo, Ricardo, Matias and myself.

Is it possible to add me a bit late to the attendees? :)

`guile --listen' broken on guile master

[Christopher Allan Webber]

Hello,

If you use the latest Guile, you'll find that "guile --listen" has
broken.  If you try:
  ./meta/guile --listen=/tmp/guile-socket

then you'll find that you have a very short window in which you can do:
  M-x geiser-connect-local  guile  /tmp/guile-socket 

but then the main repl, the one you launched in the shell, hangs.
And if you wait too long or disconnect then reconnect, you won't be
able to connect again.

What you'll see in the guile process is this:

  scheme@(guile-user)> In thread:
  ERROR: In procedure select: Interrupted system call

Here's the commit that introduces the bug:
206dced87f425af7eed628530313067a45bee2c2

I've verified that it works right before that commit.

The commit does some things involving changing the code involving the
"select" and etc in the repl code, so maybe that's related.

Re: GNU Guile 2.1.5 released (beta)

[Christopher Allan Webber]

Matt Wette writes:

>> On Dec 16, 2016, at 5:50 AM, Matt Wette  wrote:
>> 
>> OK.  This is gonna be hard to trace down.
>> 
>> I am on a Mac running Sierra, 10.12.2.  At work I am running 10.9 I believe, 
>> and I see the same behavior.
>> 
>> 
>> If I compile with gcc 6.2 or gcc4.9, but no extra flags besides —prefix, I 
>> get the following failure:
>> 
>> FAIL: numbers.test: string->number: valid complex number strings
>> 
>> and trace this down to 
>> 
>> meta/guile -c '(display (eqv? (imag-part (string->number "#i1@-0")) -0.0))' 
>> => #f
>
> I am now compiling with clang-3.9 and getting fatal errors.  Stay tuned …

Is clang supported?  I didn't think it was.

Re: GNU Guile 2.1.5 released (beta)

[Christopher Allan Webber]

Matt Wette writes:

>> On Dec 7, 2016, at 11:55 AM, Andy Wingo  wrote:
>> 
>> We are pleased to announce GNU Guile release 2.1.5.
>> 
>> Guile 2.1.5 is the fifth pre-release in what will eventually become the
>> 2.2 release series.  We encourage you to test this release and provide
>> feedback to guile-devel@gnu.org.
>
> I am building on my Mac running macOS 10.12.1, aka Sierra.  I will post my 
> patches when I get through “make check”.  
>
> As a heads-up:
> 1) macOS does not have clock_getcpuclockid(), stime.c line 833, so patch 
> skips that check.
> 2) macOS mkostemp() (filesys.c line 1496) will fail if any non-supported bits 
> are passed in open_flags, so patch masks out unsupported bits.
>
> Make check is failing on test-language, so I will be chasing that down.
>
> Matt

Thanks for working on Guile portability!

Re: Re-using bootstrap in the half-strap

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hi!
>
> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> If you could package *just* the bootstrapped compiler, if a user doesn't
>> feel like waiting through the whole bootstrap process, they could use
>> the existing bootstrap package, like:
>>
>>   ./configure --with-bootstrap=/path/to/guile-bootstrap
>
> That’s essentially what happens when cross-compiling Guile, where the
> GUILE_FOR_BUILD variable must point to a same-version Guile.
>
> What you’re suggesting seems equivalent to bundling the pre-built .go
> files, which is what is done with release tarballs.
>
> Or am I missing something?
>
> Thanks,
> Ludo’, who sympathizes.  :-)

I'm suggesting we don't bundle them *with* it... because that wouldn't
be helpful when building from a git checkout.  But if we could point to
the prebuilt .go files from a separate package, then yeah!

Note that I wouldn't want to *require* building from a separate
package... I agree that it's valuable to be able to not rely on Guile to
build Guile :)

But maybe I'm just being silly!
 - Chris

Re-using bootstrap in the half-strap

[Christopher Allan Webber]

Hello,

A thought this morning... one challenge with Guile's 2.1 branch is it
takes a log time to bootstrap.  There are good reasons for this, and
Andy Wingo has documented them:

  http://wingolog.org/archives/2016/01/11/the-half-strap-self-hosting-and-guile

However, I can't help but wonder if we could reuse the initial
bootstrapped compiler.  What if we had an external package that
contained *just* the scheme interpreter used to bootstrap the rest of
the system?  We might not need to separate the codebase, but have this
be a separate make output.

If you could package *just* the bootstrapped compiler, if a user doesn't
feel like waiting through the whole bootstrap process, they could use
the existing bootstrap package, like:

  ./configure --with-bootstrap=/path/to/guile-bootstrap

and maybe we could have a guile-bootstrap package in guix, etc.  If you
want to do the whole bootstrap yourself in the Guile package, you could
do so, but it wouldn't be a requirement, assuming you had that separate
package...

Thoughts?  Useful?  Badly informed?
 - Chris

Re: Status of elisp support in Guile

[Christopher Allan Webber]

Wilfred Hughes writes:

>> you'd definitely need to do copyright assignment for Guile.  I think
>> other "establishing yourself in the community" things apply,
>> informally(??)
>
> I have copyright assignment all in order now :). I don't have any wild
> ambitions for sweeping changes, I just thought it might be easier to land
> simple docs patches without making work for the core Guile folks.

Again, can't speak to giving out commit access... that's not my area.

But if you have documentation patches, and since you've done copyright
assignment, those would be most welcome.  If you could post them to the
list, I could help get them into the wip-elisp branch!

Re: Guile security vulnerability w/ listening on localhost + port (with fix)

[Christopher Allan Webber]

Lizzie Dixon writes:

> Hi,
>
> On 10/11, Christopher Allan Webber wrote:
>> The default in Guile has been to expose a port over localhost to which
>> code may be passed.  The assumption for this is that only a local user
>> may write to localhost, so it should be safe.  Unfortunately, users
>> simultaneously developing Guile and operating modern browsers are
>> vulnerable to a combination of an html form protocol attack [1] and a
>> DNS rebinding attack [2].  How to combine these attacks is published in
>> the article "How to steal any developer's local database" [3].
>
>> 
>> In Guile's case, the general idea is that you visit some site which
>> presumably loads some javascript code (or tricks the developer into
>> pressing a button which performs a POST), and the site operator switches
>> the DNS from their own IP to 127.0.0.1.  Then a POST is done from the
>> website to 127.0.0.1 with the body containing scheme code.  This code is
>> then executed by the Guile interpreter on the listening port.
>
> You don't need to rebind DNS to exploit this bug, or other bugs like
> it. I wrote some details here:
>
> <https://blog.lizzie.io/exploiting-CVE-2016-8606.html>
>
> Best,
>
> Lizzie.

Hi Lizzie!  Thanks for the post.  Interesting to see you figured out how
to do it with a GET request, not just a POST.  So, I guess this will
work from a public site as well?  I'm always a bit fuzzy about what
browsers do and don't allow, but I'm stunned that a browser will let a
request from some http://foo.example/ to http://localhost:37146/, even
for just a GET.  It seems like there are all sorts of daemons you can
exploit that way.

Anyway, thanks for the interesting blogpost, and kudos for using Guile
to write your example!
 - Chris

Re: Status of elisp support in Guile

[Christopher Allan Webber]

Wilfred Hughes writes:

> I've noticed that NEWS in Guile trunk says:
>
> ** Complete Emacs-compatible Elisp implementation
>
> However, I can see that there are 36 commits on the wip-elisp branch
> that aren't in master. For example, defsubst support[1] seems only to
> be on wip-elisp branch. It's still the case the guile-emacs docs[2]
> recommend using the wip-elisp branch.
>
> I can merge master into wip-elisp only one trivial conflict[3]. Can
> anyone shed any light on the work outstanding here, and the process to
> land these patches?

Hi!  So, I'm the one who did the most recent rebase of wip-elisp.

Last I heard, Wingo was interested in merging, but looked over the
branch and saw that it wasn't a trivial merge... the branch added some
non-trivial changes to the compiler, and I think Wingo wanted to review
those before merging.  (I'm not really qualified to help, there.)

> Relatedly, what's the process for getting commit access to Guile?

I leave this one to someone else to reply to, but you'd definitely need
to do copyright assignment for Guile.  I think other "establishing
yourself in the community" things apply, informally(??)

Re: Experimental JIT for Guile (???!)

[Christopher Allan Webber]

Oh great to hear this extra backstory!

Atsuro, welcome to the Guile community... I'd love to see collaboration
between you and Guile core!

Nala Ginrut writes:

> Well, there's interesting story behind it.
> Few days ago this new JIT backend named Nash was introduced in ICFP2016 
> (scheme
> workshop), I and Ludo are lecturers too. When I first saw the topic, I looked
> back to Ludo and thought "oh you guys are making a new weapon secretly huh?"
> I thought this work must be encouraged and helped by Ludo or Wingo or Mark, 
> but
> finally I realized that I was wrong, the authorAtsuro Hoshino was hacking JIT
> backend for Guile all by himself. Ludo got surprised too.
> I have to say, lone hero pattern is not recommended for a community project, 
> but
> anyway he did it bravely and the result seems good according to the paper.
> After the meeting, I and Ludo tried to convince him to get involved into our
> community to get more help and feedback.
>
> I CC him here, and it depends on him whether/when to introduce more.
>
> I think this project is just amazing, really! Thank you Hoshino! ;-)
>
> Best regards.
>
> On Tue, 2016-09-27 at 12:30 -0500, Christopher Allan Webber wrote:
>> Earlier today, David Thompson pointed to this paper in #guix on
>> freenode:
>> 
>>  https://github.com/8c6794b6/guile-tjit-documentation/raw/master/nash.pdf
>> 
>> And here seems to be the source:
>> 
>>  https://github.com/8c6794b6/guile-tjit
>> 
>> I'm not informed enough to judge this myself, but this seems like a
>> reasonable start-of-implementation of the ideas expressed here:
>> 
>>  http://wingolog.org/archives/2015/11/03/two-paths-one-peak-a-view-from-below
>> -on-high-performance-language-implementations
>> 
>> It mentions hot loops and compiling to native code... that's about as
>> much as I can tell myself about it being on track.But it seems pretty
>> cool, especially for something shooting onto the radar seemingly out of
>> nowhere!
>> 
>> Anyone more informed have thoughts? :)
>> - Chris
>> 

Experimental JIT for Guile (???!)

[Christopher Allan Webber]

Earlier today, David Thompson pointed to this paper in #guix on
freenode:

  https://github.com/8c6794b6/guile-tjit-documentation/raw/master/nash.pdf

And here seems to be the source:

  https://github.com/8c6794b6/guile-tjit

I'm not informed enough to judge this myself, but this seems like a
reasonable start-of-implementation of the ideas expressed here:

  
http://wingolog.org/archives/2015/11/03/two-paths-one-peak-a-view-from-below-on-high-performance-language-implementations

It mentions hot loops and compiling to native code... that's about as
much as I can tell myself about it being on track.  But it seems pretty
cool, especially for something shooting onto the radar seemingly out of
nowhere!

Anyone more informed have thoughts? :)
 - Chris

Re: Building Guilemacs

[Christopher Allan Webber]

If someone wanted to rebase the emacs side of the guile-emacs fork...
I think that would be great :)

Wilfred Hughes writes:

> Digging some more, I think the issue is
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24065 . Emacs trunk is
> depending on undefined C behaviour, which gcc 5 has started optimising,
> breaking the code.
>
> I've updated https://www.emacswiki.org/emacs/GuileEmacsBuild to note this.
> You can work around with -fno-builtin-malloc or simply compile with clang
> (my current solution).
>
> Emacs trunk has fixed this (in 4b1436b702d56eedd27a0777fc7232cdfb7ac4f6
> IIUC), but I believe that commit does not exist in the guile-emacs fork.
>
> Wilfred
>
> On 4 September 2016 at 04:48, Stefan Husmann 
> wrote:
>
>> Wilfred Hughes  writes:
>>
>> > I've had a play with the AUR packages too. The guile4emacs PKGBUILD works
>> > perfectly, but the guile-emacs build process segfaults when
>> bootstrap-emacs
>> > attempts to load calendar/cal-loaddefs.el:
>> >
>> > make[2]: Leaving directory
>> > '/home/wilfred/aur/guile-emacs-git/src/guilemacs/lisp'
>> > if test "yes" = "yes"; then \
>> >   rm -f bootstrap-emacs; \
>> >   ln temacs bootstrap-emacs; \
>> > else \
>> >   ./temacs --batch --load loadup bootstrap || exit 1; \
>> >   test "X" = X ||  -zex emacs; \
>> >   mv -f emacs bootstrap-emacs; \
>> > fi
>> > make -C ../lisp autoloads EMACS="../src/bootstrap-emacs"
>> > make[2]: Entering directory
>> > '/home/wilfred/aur/guile-emacs-git/src/guilemacs/lisp'
>> > EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file
>> > --no-site-lisp -l autoload \
>> >--eval "(setq generate-autoload-cookie \";;;###cal-autoload\")" \
>> >--eval "(setq generated-autoload-file (expand-file-name
>> > (unmsys--file-name \"calendar/cal-loaddefs.el\")))" \
>> >-f batch-update-autoloads ./calendar
>> > make[2]: *** [Makefile:466: calendar/cal-loaddefs.el] Segmentation fault
>> > (core dumped)
>> > make[2]: Leaving directory
>> > '/home/wilfred/aur/guile-emacs-git/src/guilemacs/lisp'
>> > make[1]: *** [Makefile:805: ../lisp/loaddefs.el] Error 2
>> > make[1]: Leaving directory
>> > '/home/wilfred/aur/guile-emacs-git/src/guilemacs/src'
>> > make: *** [Makefile:376: src] Error 2
>> > ==> ERROR: A failure occurred in build().
>> > Aborting...
>> >
>> > Any suggestions?
>> >
>> > On 30 August 2016 at 20:32, Wilfred Hughes  wrote:
>> >
>> Hello,
>>
>> I can confirm this, but currently have no ideas what is going on
>> here. This definitely used to work, but gcc was updated since then.
>>
>> Sorry for inconveniance. I will try to dig deeper into it.
>>
>> Best Regards
>>
>> Stefan
>>

Re: Building Guilemacs

[Christopher Allan Webber]

Wilfred Hughes writes:

> Hi folks
>
> I fancied exploring Guile with Emacs, so I've been trying to build Guile
> and Emacs as documented in https://www.emacswiki.org/emacs/GuileEmacsBuild .

Excellent!  Happy to hear of your interest.

> I've successfully built the wip-elisp branch of the main guile repo. The
> wiki page discusses http://git.hcoop.net/?p=bpt/guile.git and
> http://git.savannah.gnu.org/cgit/guile.git/log/?h=wip-elisp . I understand
> that the latter is a rebase of the hcoop commits on Guile 2.1.
>
> However, I've also noticed there's a wip-bpt-elisp branch in the main guile
> repo. This has the newest commits, but it seems to based on an older guile
> version.
>
> Which branch should I use?

wip-bpt-elisp has commits from years ago; wip-elisp is rebased on top of
relatively-recent master, with updated ChangeLog.  This was based on the
recent work of robin's from
http://git.savannah.gnu.org/cgit/guile.git/log/?h=wip-elisp

So, you want wip-elisp in origin.  That's the most up to date.

> I've also failed to compile the Emacs branch that uses Guile. I'm getting
> the error:
>
> gcc -c  -Demacs  -I. -I. -I../lib -I./../lib  [many includes elided...]
> -I/usr/local/include/guile/2.2   -g3 -O2  vm-limit.c
> In file included from vm-limit.c:21:0:
> lisp.h:33:22: fatal error: libguile.h: No such file or directory
>  #include 
>   ^
> compilation terminated.
>
> This rather looks like I've screwed up with autoconf somehow. I've set the
> environment variables suggested by the wiki page:

I don't have time to look into this part, maybe someone else can help
you out!

Re: Packing software developed in GNU Guile Scheme

[Christopher Allan Webber]

Daniel Pimentel writes:

> Hi Guiles,
>
> I developed a software with GNU Guile Scheme and I'd like packing to 
> distribute it.
>
> How do I do? Using Makefile?

Maybe look at the automake files from Sly or Guix.

> Project dependences: Guile, LSH/SSH.
>
> Thanks and sorry for my newbie's question :)
>
> Note: I'll packing for Guix/GuixSD in soon, but I'd like to distribute 
> to other distribution by tarball for example.

No worries!  Yes, it needs to be better documented.

Re: wip-ports-refactor

[Christopher Allan Webber]

Andy Wingo writes:

> Greets,
>
> On Sun 17 Apr 2016 10:49, Andy Wingo  writes:
>
>>   | baseline | foo| port-line | peek-char
>> --+--++---+--
>> guile 2.0 | 0.269s   | 0.845s | 1.067s| 1.280s
>> guile master  | 0.058s   | 0.224s | 0.225s| 0.433s
>> wip-port-refactor | 0.058s   | 0.220s | 0.226s| 0.375s
>
> So, I have completed the move to port buffers that are exposed to
> Scheme.  I also ported the machinery needed to read characters and bytes
> to Scheme, while keeping the C code around.  The results are a bit
> frustrating.  Here I'm going to use a file that contains only latin1
> characters:
>
>   (with-output-to-file "/tmp/testies.txt" (lambda () (do-times #e1e6 
> (write-char #\a
>
> This is in a UTF-8 locale.  OK.  So we have 10M "a" characters.  I now
> want to test these things:
>
>   1. peek-char, 1e7 times.
>   2. read-char, 1e7 times.
>   3. lookahead-u8, 1e7 times.  (Call it peek-byte.)
>   4. get-u8, 1e7 times.  (Call it read-byte.)
>
>| peek-char | read-char | peek-byte | read-byte
>   -+---+---+---+--
>   2.0  | 0.811s| 0.711s| 0.619s| 0.623s
>   master   | 0.410s| 0.331s| 0.428s| 0.411s
>   port-refactor C  | 0.333s| 0.358s| 0.265s| 0.245s
>   port-refactor Scheme | 1.041s| 1.820s| 0.682s| 0.727s
>
> Again, measurements on my i7-5600U, best of three, --no-debug.
>
> Conclusions:
>
>   1. In Guile master and 2.0, reading is faster than peeking, because it
>  does a read then a putback.  In wip-port-refactor, the reverse is
>  true: peeking fills the buffer, and reading advances the buffer
>  pointers.
>
>   2. Scheme appears to be about 3-4 times slower than C in
>  port-refactor.  It's slower than 2.0, unfortunately.  I am certain
>  that we will get the difference back when we get native compilation
>  but I don't know when that would be.
>
>   3. There are some compiler improvements that could help Scheme
>  performance too.  For example the bit that updates the port
>  positions is not optimal.  We could expose it from C of course.
>
> Note that this Scheme implementation passes ports.test, so there
> shouldn't be any hidden surprises.
>
> I am not sure what to do, to be honest.  I think I would switch to
> Scheme if it let me throw away the C code, but I don't see the path
> forward on that right now due to bootstrap reasons.  I think if I could
> golf `read-char' down to 1.100s or so it would become more palatable.
>
> Andy

Happily at least, none of these benchmarks are *that much* slower than
Guile 2.0.  So most "present day" users won't be noticing a slowdown in
IO if this slipped into the next release.

You're probably right (is my vague and uninformed suspicion) that native
compilation would speed it up.

My thoughts are: if this refactor could bring us closer to more useful
code for everyday users, a small slowdown over 2.0 is not so bad.  Eg,
if we could get SSL support, and buffered reads with prompts,
etc... those are good features.  So if you had my vote I'd say: forge
ahead on adding those, and if they come out well, then I think this
merge is worth it anyway, despite a small slowdown in IO over 2.0.
Hopefully we'll get it back in the future anyway!

 - Chris

Re: rename from q to queue

[Christopher Allan Webber]

Ludovic Courtès writes:

> Mathieu Lirzin  skribis:
>
>> One way to not break backward compatibility, would be to move (ice-9 q)
>> code in (ice-9 queue) and make (ice-9 q) use it and re-export every
>> variable with its old name.
>
> Right.
>
>> Finally we will need a way to give the information that (ice-9 q) module
>> is deprectated to the users.  I am not sure what is the best way to
>> achieve this.  Maybe with:
>>
>>   (eval-when (expand load eval)
>> (display "..." (current-warning-port)))
>>
>> What do others think?
>
> We could do a quick hack to iterate over the exported bindings,
> re-export them, and print a message.
>
> However, I think it would be best to have a mechanism in the module
> abstraction to /declare/ renames.  That would allow both the compiler
> and the run-time support to emit an appropriate warning.
>
> This would require something like adding a special field to ‘module’
> records, such that one can write, say:
>
>   (define-module (ice-9 q)
> #:superseded-by (ice-9 queue))
>
> With that in place, it will become much easier to discuss about ice-9
> renames, IMO.
>
> Thoughts?
>
> Thanks,
> Ludo’.

I think that would be useful.  There's been talk about renaminig things
from (ice-9) into a (guile) namespace.  Likewise aliasing things from
(srfi) to something more human-readable.  Are the rename and aliasing
thing the same step or two different things?

Re: wip-ports-refactor

[Christopher Allan Webber]

Andy Wingo writes:

> Hi!
>
> Summarizing my reply over IRC:
>
> On Thu 07 Apr 2016 06:16, Christopher Allan Webber <cweb...@dustycloud.org> 
> writes:
>
>> So, does this branch replace ethreads, or compliment it?  Where should I
>> be focusing my (currently limited) review / integration attempt energy?
>> I've been hoping to review ethreads this week but now I'm unsure.  Can
>> you explain how the efforts currently relate?
>
> This branch hopes to make the "eports" part of that branch unnecessary.
> However actually implementing user-space threads à la ethreads is out of
> scope, as is the epoll wrapper.
>
>> One other question is if this will help in the "no nice way to do custom
>> binary ports" stuff that was blocking the
>> tls-enabled-ports-in-guile-proper thing...
>
> Was that the blocker?

Ludovic explains here:
  https://lists.gnu.org/archive/html/guile-devel/2015-09/msg00042.html

> Anyway the current branch's ports are vey close to R6RS binary
> ports, so this shouldn't be a difficulty any more.  I haven't
> implemented custom binary I/O ports (we have input-only and
> output-only but not both) yet, but it should be doable.

Yay!  It does sound like that might fix it.  (Admittedly it's a bit
beyond me.)

>> As I've said, I'm not tied to 8sync specifically if doing something more
>> internally makes more sense.  (Even if I have a nice site and logo
>> coming together now ;))
>
> I think keep rolling with 8sync :)  It has a nice brand, it's filling a
> need that probably won't be filled in 2.2.0, it's laying groundwork for
> future Guile features.

Yay!  Ok, I will do so.

> Eventually I would like user-space threads in Guile proper,
> implemented in terms of delimited continuations, and that implies a
> scheduler too.  But that's a bit far off.  My goal is to make it
> possible to add such a thing during the 2.2.x series, probably first
> as a library (8sync) and eventually as a core Guile feature.
>
> Andy

Sounds good.  But here's a question: we haven't accepted many
contributions yet to 8sync.  Would it make sense now to require
copyright assignment for the project?  (We don't, yet.)  That might slow
development a bit but could make future merging into Guile, if such a
thing were to be done, easier.

 - Chris

Re: wip-ports-refactor

[Christopher Allan Webber]

Andy Wingo writes:

> Hi,
>
> I have been working on a refactor to ports.  The goal is to have a
> better concurrency story.  Let me tell that story then get down to the
> details.

Hoo, what an email!  I need to read your code before I can do a full
commentary.  But...

> More appropriate is 8sync, a new project by Chris Webber that is
> designed to be a kind of user-space threading library for Guile.

Hey, thanks!  I hope it's on the right track.

> I did give a try at prototyping such a thing a long time ago,
> "ethreads".  Ethreads are user-space threads, which are really delimited
> continuations with a scheduler.  If the thread -- the dynamic extent of
> a program that runs within a prompt -- if the thread would block on I/O,
> it suspends itself, returning to the scheduler, and then the scheduler
> resumes the thread when I/O can continue.  There's an epoll loop
> underneath.
>
> That hack seemed to work; I even got the web server working on it, and
> ran my web site on it for a while.  The problem was, though, that it
> completely bypassed ports.  It made its own port types and buffers and
> everything.  That's not really Guile -- that's a library.
>
> *  *  *
>
> Which brings us to the port refactor.  Ultimately I see ports as all
> having buffers.  These buffers can be accessed from Scheme.  Normal I/O
> goes to the buffer first.  When the buffers need filling or emptying,
> Scheme code can call Scheme code to do that.  There could be Scheme
> dynamic parameters defining whether filling/emptying blocks -- if it
> doesn't block, then if the read would block it could call out to some
> context to suspend the thread.  Since it's all Scheme code, that
> continuation can be resumed as well -- the delimited continuation does
> not capture a trampoline through C.  The buffer itself is represented as
> a bytevector with a couple of cursors, which gives us some basic
> threadsafety without locks on the Scheme side -- Scheme always checks
> that accesses are within bounds.
>
> But, currently in Guile 2.0 and in master, buffering is handled by the
> port implementation.  That means that there is no buffer to expose to
> Scheme, and no real path towards implementing efficient I/O operators
> that need to grovel in a buffer from Scheme.  It also means that there's
> no easy solution for non-blocking I/O, AFAIU.
>
> The wip-port-refactor branch is a step towards centralizing buffering
> management within the generic ports code.  It thins the interface to
> port implementations, instead assuming that the read/write functions are
> to unbuffered mutable stores, as Guile is the part handling the
> buffering.  I've documented what I can in the branch.

So, does this branch replace ethreads, or compliment it?  Where should I
be focusing my (currently limited) review / integration attempt energy?
I've been hoping to review ethreads this week but now I'm unsure.  Can
you explain how the efforts currently relate?

> The commits before the HEAD are fairly trivial I think; it's the last
> one that's a doozy.  It doesn't yet remove locks; there's still a lot of
> locks, and it's hard to know what we can do without locks given the
> leeway give to modern C compilers.  But it's a step.
>
> Going forward we need to define a Scheme data type for ports, and to
> allow the read/write procedures to be called from Scheme, and to allow
> Scheme implementaitons of those procedures.  We also need to figure out
> how to do non-blocking I/O, both on files and non-files; should we set
> all our FD's to O_NONBLOCK?  How does it affect our internal
> interfaces?  I do not know yet.

One other question is if this will help in the "no nice way to do custom
binary ports" stuff that was blocking the
tls-enabled-ports-in-guile-proper thing...

> There's still space for different schedulers.  I wouldn't want to
> include a scheduler and a thread concept in Guile 2.2.0 I don't think --
> but if we can build it in such a way that it seems natural, on top of
> ports, then it sounds like a good idea.

As I've said, I'm not tied to 8sync specifically if doing something more
internally makes more sense.  (Even if I have a nice site and logo
coming together now ;))

Exciting times!
 - Chris

Re: Elisp branch ready for merge (??)

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Well, I didn't think I'd have time to do this (and in a sense I didn't)
> but:
>   https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip
>
> I've rebased the whole branch against git master and added ChangeLog
> style entries.  "make check" is passing.  It seems to me that it's ready
> for merge.  I did the best I could on the ChangeLog additions, both with
> my limited ChangeLog experience and from my limited Guile internals
> experience.  So, corrections welcome, but otherwise...
>
> I think we really should not delay on this, and we should try to merge
> this as soon as possible.  This already has bitrotted before, and if we
> wait on it, it could bitrot again.  It would be great to get this pulled
> into Guile proper.
>
> Plus it would be a nice bullet point in the next release! :)
>
>  - Chris

I've been given commit access to Guile proper (horray!) and I pushed a
rebased version of this branch to origin at wip-elisp!  Have fun!

 - Chris

Re: What is needed in guildhall to include it in Guile?

[Christopher Allan Webber]

Arne Babenhauserheide writes:

> Arne Babenhauserheide writes:
>
>>> - Add the path to the installed package in guild show  or 
>>> list-packages
>> First experience: It takes quite some time to get into guildhall — much
>> longer than I expected. I spend most of the time finding out how to
>> extract information from its datastructures.
>
> I now filed three pull requests: One for better documentation (25), one
> for showing the destination in show (24) — not yet perfect, but a start
> — and one where I started experimenting with a new command (26).
>
> https://github.com/ijp/guildhall/pulls
> https://github.com/ijp/guildhall/pull/24
> https://github.com/ijp/guildhall/pull/25
> https://github.com/ijp/guildhall/pull/26
>
> I’m not really fast, but it’s moving forward :)
>
> Best wishes,
> Arne

Great!

I am still hopeful that Ian Price might return; I miss his
contributions!  If he doesn't though, maybe the repository needs to be
forked somewhere...

Re: Elisp branch ready for merge (??)

[Christopher Allan Webber]

Taylan Ulrich Bayırlı/Kammer writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Taylan Ulrich Bayırlı/Kammer writes:
>>
>>> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>>>
>>>> Well, I didn't think I'd have time to do this (and in a sense I didn't)
>>>> but:
>>>>   https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip
>>>>
>>>> I've rebased the whole branch against git master and added ChangeLog
>>>> style entries.  "make check" is passing.  It seems to me that it's ready
>>>> for merge.  I did the best I could on the ChangeLog additions, both with
>>>> my limited ChangeLog experience and from my limited Guile internals
>>>> experience.  So, corrections welcome, but otherwise...
>>>
>>> Exciting!
>>>
>>> Small remark: the "title" line of the commit messages should be complete
>>> sentences.
>>
>> Yes, though I didn't write them.  I also don't know in each case what a
>> complete sentence would be.  I did my best job by filling in the
>> ChangeLog style part.  Often times my figuring out the ChangeLog stuff
>> was based on some sentence fragment.
>>
>> Changing the one part that is the original author's writing to something
>> different... I can do it by attempting to guess, but I'm worried about
>> removing that context.
>>
>> One thing I could do is leave in the description: "Original title: foo"
>>
>> What do you think of that?
>
> In some cases it seems the title is already a sentence, just lacking
> capitalization and punctuation.  (It seems Guile doesn't have a strict
> rule about the punctuation though.)
>
> In other cases it seems like titles that should be "Add foobar" are
> shortened to just "foobar", e.g.:
>
> - guile-backtrace function
> - eval-when
> - fset macro
> - defsubst
> - compiler macros
> - elisp @@ macro
>
> All in all it looks like most titles should be straightforward to fix.
> Fixing any would be better than fixing none IMO.

You are probably right.

> And I'd say mentioning the original title is unnecessary for the
> obvious ones, though we should get Robin's sanctioning for what we do.
> Is Robin perhaps available for some basic feedback?

I haven't heard from them, but if they chose to reach out it would be of
course most welcome!

 - Chris

Re: Elisp branch ready for merge (??)

[Christopher Allan Webber]

Taylan Ulrich Bayırlı/Kammer writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Well, I didn't think I'd have time to do this (and in a sense I didn't)
>> but:
>>   https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip
>>
>> I've rebased the whole branch against git master and added ChangeLog
>> style entries.  "make check" is passing.  It seems to me that it's ready
>> for merge.  I did the best I could on the ChangeLog additions, both with
>> my limited ChangeLog experience and from my limited Guile internals
>> experience.  So, corrections welcome, but otherwise...
>
> Exciting!
>
> Small remark: the "title" line of the commit messages should be complete
> sentences.

Yes, though I didn't write them.  I also don't know in each case what a
complete sentence would be.  I did my best job by filling in the
ChangeLog style part.  Often times my figuring out the ChangeLog stuff
was based on some sentence fragment.

Changing the one part that is the original author's writing to something
different... I can do it by attempting to guess, but I'm worried about
removing that context.

One thing I could do is leave in the description: "Original title: foo"

What do you think of that?

>> I think we really should not delay on this, and we should try to merge
>> this as soon as possible.  This already has bitrotted before, and if we
>> wait on it, it could bitrot again.  It would be great to get this pulled
>> into Guile proper.
>>
>> Plus it would be a nice bullet point in the next release! :)
>
> +1
>
> Would also be good that one doesn't need to keep two versions of Guile
> around when hacking on Guile-Emacs.

I agree! :)

Re: Elisp branch ready for merge (??)

[Christopher Allan Webber]

Mathieu Lirzin writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Well, I didn't think I'd have time to do this (and in a sense I didn't)
>> but:
>>   https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip
>>
>> I've rebased the whole branch against git master and added ChangeLog
>> style entries.  "make check" is passing.  It seems to me that it's ready
>> for merge.  I did the best I could on the ChangeLog additions, both with
>> my limited ChangeLog experience and from my limited Guile internals
>> experience.  So, corrections welcome, but otherwise...
>
> Nice!
>
> Small nitpick.  Could you remove the extra spaces at the start of
> indented lines?  :)
>
> I know this practice is/was a common practice in Guile but even if it
> looks prettier with indentation, this is not the proper change log
> format described by GCS and recognized by 'change-log-mode' in Emacs.

I could... but I'm hesitant to do so if it's not the standard ways to do
things in Guile's repository.  I'll leave that question to whoever is
interested in merging it.  If they'd like me to change it, I'll do it,
and since they have commit access, I consider them the authority.

Elisp branch ready for merge (??)

[Christopher Allan Webber]

Well, I didn't think I'd have time to do this (and in a sense I didn't)
but:
  https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip

I've rebased the whole branch against git master and added ChangeLog
style entries.  "make check" is passing.  It seems to me that it's ready
for merge.  I did the best I could on the ChangeLog additions, both with
my limited ChangeLog experience and from my limited Guile internals
experience.  So, corrections welcome, but otherwise...

I think we really should not delay on this, and we should try to merge
this as soon as possible.  This already has bitrotted before, and if we
wait on it, it could bitrot again.  It would be great to get this pulled
into Guile proper.

Plus it would be a nice bullet point in the next release! :)

 - Chris

Re: What's needed to get elisp updates into Guile master?

[Christopher Allan Webber]

Taylan Ulrich Bayırlı/Kammer writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Mathieu Lirzin writes:
>>
>>> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>>>
>>>> Mathieu Lirzin writes:
>>>>>
>>>>> You should bring that on emacs-devel where most people seem to think
>>>>> that a 'git commit -m "..."' with the actual diff is self explanatory.
>>>>> :)
>>>>
>>>> I'm afraid to.  The emacs list has had strong responses moving between
>>>> tremendous enthusiasm for the guile-emacs work to outright hostility.  I
>>>> think there are only a couple of hostile people, but they are vocal.
>>>> I'd prefer to have the next stage of things merged before I reach out
>>>> again.
>>>>
>>>> But maybe I'm being irrational.  I could probably still ask for help.
>>>
>>> I was implicitly referring to a current discussion on emacs-devel:
>>>
>>>   https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00180.html
>>>
>>> Sorry for not giving the context of my sarcasm at first.
>>>
>>> Anyway, thanks for giving guile-emacs some love!  :)
>>
>> I saw that conversation right after your email and I still didn't pick
>> up on it.  Heh!
>>
>> Someone from #emacs might actually help though!  (I'm not going to name
>> them and put them on the spot.)
>
> Don't know if you mean me, but in any case I also have it in my mid-term
> TODO to start working on guile-emacs.  Life-Status report:

Great!  I didn't mean you, so maybe we have overlap.  Really, whoever
steps up to do it, great!

> Expect a few more months of "downtime" on my side.  (Recreational
> purposes, though I'm available for emergency situations like if 'guix
> pull' stops working. :P) After that, I'll have a time-frame of maximally
> one year during which I'll be free from occupational work, in which time
> I have to juggle my time between looking into how to best go on with my
> occupational life (the software industry is awful and I wish to be
> politically active in some way but programming is my only skill), and
> working on various software projects including Guix contributions, Guile
> contributions, RnRS/SRFI contributions, and Guile-Emacs.
>
> The Guix and Guile contributions I have on my TODO aren't very big so
> hopefully Guile-Emacs can get a good amount of time allocated to it.
> OTOH, it will need a ton of tedious learning on my side if I have to
> work on nontrivial C code, and especially the BDW-GC.  All in all don't
> set your expectations too high please.  If more experienced hackers can
> work on Guile-Emacs, that would be ideal.

Great!  guile-emacs could definitely use the help.  Thanks for your
interest, and if you do have time, your time! :)

 - Chris

Re: What's needed to get elisp updates into Guile master?

[Christopher Allan Webber]

Mathieu Lirzin writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Mathieu Lirzin writes:
>>>
>>> You should bring that on emacs-devel where most people seem to think
>>> that a 'git commit -m "..."' with the actual diff is self explanatory.
>>> :)
>>
>> I'm afraid to.  The emacs list has had strong responses moving between
>> tremendous enthusiasm for the guile-emacs work to outright hostility.  I
>> think there are only a couple of hostile people, but they are vocal.
>> I'd prefer to have the next stage of things merged before I reach out
>> again.
>>
>> But maybe I'm being irrational.  I could probably still ask for help.
>
> I was implicitly referring to a current discussion on emacs-devel:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00180.html
>
> Sorry for not giving the context of my sarcasm at first.
>
> Anyway, thanks for giving guile-emacs some love!  :)

I saw that conversation right after your email and I still didn't pick
up on it.  Heh!

Someone from #emacs might actually help though!  (I'm not going to name
them and put them on the spot.)

 - Chris

Re: What's needed to get elisp updates into Guile master?

[Christopher Allan Webber]

Mathieu Lirzin writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Christopher Allan Webber writes:
>>
>>> So even better news: I've successfully rebased BT Templeton's wip elisp
>>> branch on top of guile master... you can get it here:
>>>
>>>   https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip
>>>
>>> Maybe I should update the Guix package to make use of that?
>>
>> So I'd like to get this merged in before it bitrots too heavily.
>> It would be a shame to not get it in, after how far it's come!
>
> Agreed.
>
>> Now that it's rebased and running on master, there are two issues I see
>> that would need to be addressed:
>>
>>  - The most recent commit disables three tests (specific to elisp).
>>They should probably be re-enabled, or at worst at least commented
>>out with a warning rather than being switched to expect-fail as they
>>are right now.
>>
>>I'd prefer re-enabling them with fixes; for now, commenting out with
>>clear explaination still seems better than bitrot though.  (The other
>>tests all do pass.)
>
> I don't understand why you would want to do that. 'expect-fail' is the
> correct semantic for a test that should pass but is known to currently
> fail, like in TDD.  Do I miss something?

Ah, okay!  Now I understand.  So I think leaving these as-is is fine in
merging this.

>>  - The commits are not in ChangeLog style format and are fairly terse in
>>their message.  I imagine it's a requirement to fix that.  This seems
>>like the bigger challenge; if anyone but the original author were to
>>do it, there would have to be some careful reading to figure out what
>>the purpose of each commit and change meant.  That seems like some
>>work.  I'm not sure I have the time for it... are there any
>>volunteers?
>
> You should bring that on emacs-devel where most people seem to think
> that a 'git commit -m "..."' with the actual diff is self explanatory.
> :)

I'm afraid to.  The emacs list has had strong responses moving between
tremendous enthusiasm for the guile-emacs work to outright hostility.  I
think there are only a couple of hostile people, but they are vocal.
I'd prefer to have the next stage of things merged before I reach out
again.

But maybe I'm being irrational.  I could probably still ask for help.

> Indeed, it seems a tedious job to add those change logs since it
> requires to understand the purpose of the commit.  The volunteer should
> be familiar with Elisp in general and the limitations of
> current/previous Guile implementation.

Yeah it's tough.

>> Is there general agreement that if we can get these two things done,
>> that it would be worth merging this?
>
> I think it would be great.

Great!

What's needed to get elisp updates into Guile master?

[Christopher Allan Webber]

Christopher Allan Webber writes:

> So even better news: I've successfully rebased BT Templeton's wip elisp
> branch on top of guile master... you can get it here:
>
>   https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip
>
> Maybe I should update the Guix package to make use of that?

So I'd like to get this merged in before it bitrots too heavily.
It would be a shame to not get it in, after how far it's come!

Now that it's rebased and running on master, there are two issues I see
that would need to be addressed:

 - The most recent commit disables three tests (specific to elisp).
   They should probably be re-enabled, or at worst at least commented
   out with a warning rather than being switched to expect-fail as they
   are right now.

   I'd prefer re-enabling them with fixes; for now, commenting out with
   clear explaination still seems better than bitrot though.  (The other
   tests all do pass.)

 - The commits are not in ChangeLog style format and are fairly terse in
   their message.  I imagine it's a requirement to fix that.  This seems
   like the bigger challenge; if anyone but the original author were to
   do it, there would have to be some careful reading to figure out what
   the purpose of each commit and change meant.  That seems like some
   work.  I'm not sure I have the time for it... are there any
   volunteers?

Is there general agreement that if we can get these two things done,
that it would be worth merging this?

 - Chris

Re: Guile & Emacs chat at emacs hackathon/bug-crush SF

[Christopher Allan Webber]

So even better news: I've successfully rebased BT Templeton's wip elisp
branch on top of guile master... you can get it here:

  https://gitlab.com/dustyweb/guile/tree/merge-bipt-elisp-wip

Maybe I should update the Guix package to make use of that?

Guile & Emacs chat at emacs hackathon/bug-crush SF

[Christopher Allan Webber]

Heya everyone,

I was at the Emacs hackathon / bug crushing event and I gave a couple
demos that were Guile related, one showing off guile-emacs, and one
showing off Guix's Emacs integration.  So the good news is: the talk
went super, super well (on both, but especially guile-emacs), and
enthusiasm was high!  When I showed guile-emacs live, there were some
amazed expressions to see oh hey... this is *really* working!

I also had a conversation with John Wiegley, current maintainer of
emacs, and he said several things:

 - He thinks it would be *great* to have Emacs running on Scheme, a
   clear win, assuming it's integrated and runs fast and works well.

 - However, Guile would have to be able to make a promise: once Emacs
   ran on top of Guile, Emacs would have to be able to have say over
   anything that could end up changing actual semantics in Emacs
   (mainly, anything that would break Emacs user's source code).

   (I think there's an easy answer to this: guile-emacs is already
   aiming for heavy backwards compatibility and should just preserve
   that at this level.)

 - If we could prove that performance was better in guile-emacs, that's
   an easy way to win enthusiasm.

 - A good goal to work towards: all of emacs' tests should pass using
   guile-emacs.

So that's all a ways off, but I'm feeling enthusiastic that it's
possible!

 - Chris

PS: I'd like to see bipt's elisp branch merged with master.  I might try
to help... I'm trying to learn enough to do so.  However I don't have a
lot of time, and especially not a lot of experience with compilers..

Re: What is needed in guildhall to include it in Guile?

[Christopher Allan Webber]

Arne Babenhauserheide writes:

> Hi,
>
> In january there was a thread here about Guildhall with the notion
>
>> I encourage you to hack on Guildhall to make it more usable for your
>> needs.
>
> I finished my PhD last month, so I have some freed-up time — and I would
> like to use some of it to hack on Guildhall and make it ready for
> inclusion in Guile.
>
> However there’s one stumbling block: I don’t see what’s actually missing
> from it. So I want to be bold and request something:
>
>
> Please tell me what’s missing in Guildhall, so I can implement it.
>
>
> Best wishes,
> Arne
>
> [1]: https://lists.gnu.org/archive/html/guile-devel/2016-01/msg00010.html
>
> PS: I write now, because I just stumbled over a situation again, where
> I’d have loved to just have guildhall at hand by default. I wanted
> to test uptop.scm which needs guile-ssh, so I just wanted to quickly
> get guile-ssh and run the script. Yes, guix can do that. But the
> guix daemons weren’t running.

I've never run it so it's hard for me to know.  The "obvious" idea might
be to get a Guix package of Guildhall so more of us might play with it? ;)

Re: Towards De-icing ice-9 modules.

[Christopher Allan Webber]

Chad Albers writes:

> Hi,
>
> In my attempt to assist the guile project, I thought I would share a
> document on a plan to migrate some of the ice-9 modules into a more
> intuitive, yet to be decided, namespace.  Before I proposed a technological
> plan, I have begun really an audit of what ice-9 modules are available (and
> undocumented), and other modules that guile ships with. (there are some
> secrets down there).
>
> Here's the document so far:
> https://docs.google.com/spreadsheets/d/1hHAz1xRPlH2K6w_olWE6qej3lL7i4-zKdYAV1xbYHO8/edit#gid=0
>
> (I apologize that it's a google document.  I couldn't find a
> comparable,free software online collaborative alternative.  suggestions are
> welcome).

Try etherpad:
  https://public.etherpad-mozilla.org/

> It is by no means final, and in some cases, inaccurate.  Let me know if I
> should continue along this path.

Re: Contributions to Guile

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Chad Albers writes:
>
> [...]
>
>>> b. More robust documentation system - texinfo is not the greatest. And
>>> it's non-trivial to generate any documentation (including texinfo) for
>>> modules.
>>
>> Texinfo is pretty nice to use if you're an emacs user... in fact, if
>> you're an emacs user, it's the best documentation reading system in the
>> world.  But not everyone's an emacs user.
>
> I think Texinfo is OK even if you’re not an Emacs user, no?  Especially
> with the just-release 6.1 where menus can (finally!) be automatically
> generated.
>
>> If the html export was nicely themed
>
> It can be nicely themed.  FWIW, Gnulib’s gendocs.sh, which is what most
> projects use to export their HTML to gnu.org, now includes a CSS by
> default (see <https://gnu.org/s/guix/manual> as an example.)  We can
> change this CSS anytime, and in fact, I would love it if someone
> talented would come up with improvements!

Relatedly, see this very conversation happening about theming in
emacs-land right now!

  https://lists.gnu.org/archive/html/emacs-devel/2016-02/msg00361.html

Re: Contributions to Guile

[Christopher Allan Webber]

Chad Albers writes:

> Hi Ludo,
>
> Thanks for getting back to me.  I am most interested in remedying the
> pain points that I have encountered while developing guile code.
>
> The pain points I experienced are the following:
>
> a. Simple reference guide.  The guile manual is more of a guide than a
> reference...the best way to find information is by grepping it or
> google search. I'm thinking API documentation with perhaps examples.

Funny, I find kind of the opposite... the manual is an amazing reference
guide but was difficult as a tutorial for me.  (So I've been spending
some time on a tutorial...)

> b. More robust documentation system - texinfo is not the greatest. And
> it's non-trivial to generate any documentation (including texinfo) for
> modules.

Texinfo is pretty nice to use if you're an emacs user... in fact, if
you're an emacs user, it's the best documentation reading system in the
world.  But not everyone's an emacs user.

If the html export was nicely themed, that could help a lot.  Maybe
other things could also be done, I'm not sure.  I've been having fun
with Skribilo which has info export, though it's not always very nice
looking.  I think porting the main manual would be a lot of work
though...

> c. A real packaging system...includes specification, package
> retrieval, and package hosting, package search.  Finding and including
> third-party guile code is difficult at best.

Have you looked at Guildhall?  Sadly nobody is working on it really.
This comes up a lot though...

I've been working on porting packages to Guix when I need them, but
admittedly Guix has not won enough world mindshare to be "the answer"
here.

> d. An easier build system. I see most projects using autoconf and
> make. Using build tools designed for the C language presents a higher
> barrier to those that want to contribute libraries to the guile
> community.

If it were clear *how* to use autoconf for my Guile project, or how to
get started properly, I wouldn't mind so much.  However for every Guile
project I've started I've copy-pasta'ed someone else's autotools config
files and mashed until they worked.  But I've never understood *why*
they've worked.

I'm not super excited dealing with all the m4-expanded shell, I'd still
love a Guile based autotools replacement that implements the ./configure
&& make interface still.  I'm not sure it'll happen.  Maybe someone will
pick it up?

In the meanwhile we've had some conversations about this, see David
Thompson's RFC on a Guile project generator:

  https://lists.gnu.org/archive/html/guile-user/2015-09/msg00042.html

> e. Refactors - I have a _long_ list in my head, but here's one: the
> "ice-9" namespace is cute but confusing to the beginner I once was.

Yes, I encountered frustration here too.  It's faded to the background
now, but I think ice-9 is confusing for newcomers, so is srfi-*.  As
much as I love robots, these robot names are kind of jarring.  We've
talked about some ice-9 stuff being factored into (guile foo) instead of
(ice-9 foo) but I think Mark Weaver said he'd want a lot of careful
cleanup going through that process (could be misremembering).  Some talk
of aliases for srfi modules was made as well..

> Please don't take these of criticisms of the project per se.  These
> are simply the pain points I encountered when I move from other Lisps
> (schemes and clojure) into guile. I'm inclined to take on the more
> technical/coding tasks like c, d, e.
>
> I'm not sure any of these tasks are a priority for the guile project.
> Most of the technical task match my use case - using guile as a
> full-fledged scheme interpreter rather than as an extension language.
> I'm throwing them out there to determine if any of them are priorities
> that would be welcome contributions.

I agree with most of your points, at least within some kind of scope,
clarified above!  I'd love to see movement on them.

If you really want to work on (c) and (d) there are probably two routes:
 - Implement David's RFC for a Guile project generator
 - or, keep going with Guildhall and the guild package format stuff.  We
   could then even try to build tooling in Guix to incorporate this
   stuff.

Anyway, thanks for the thoughtful comments!

 - Chris

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Mike Gerwitz writes:

> On Wed, Dec 16, 2015 at 08:33:30 -0600, Christopher Allan Webber wrote:
>> What you'll notice is that each new sentence starts on its own line.
>> Sentences which have characters which continue beyond line 79 "wrap",
>> but are indented to be clear that they align with a previous sentence.
>
> I'm actually quite fond of this; good idea.  This will clean up my diffs
> substantially (I refill every time I modify a paragraph).
>
> Thanks for sharing that.

Glad someone else will find it helpful :)

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Alex Sassmannshausen writes:

> That's looking pretty good.  Personally I'm not sure about the
> positioning of the bit about text editors — it feels like it is a little
> tangential to getting Guile up and running.  It feels like perhaps it
> should be mentioned later (e.g. when you actually mention storing stuff
> in a .scm file?
>
> (also, it kind of acts as a mental barrier to just firing up Guile and
> having a go — which is, I think, the playful feeling you want to instil
> in your readership?)

I think you're right probably.  I tried to make it an "optional"
section.  On the other hand, realizing why Guile is really nice to work
in requires having a pretty decent setup, so it might be worth the time
investment to mention it there.  I'm not sure yet... I think maybe once
more content has come after it, we can re-evaluate and refactor.  Would
that make sense?

For now, I've broken up the editor and readline part into its own
clearly marked as "optional" section.

> I definitely still think this is a really cool way to go.  I'd probably
> think that you'll want to walk a fine line between playfulness and over
> the top childishness — and that line will be different for different
> people I guess! 8-|

Yeah it will.  I'll be tuning towards my own personal preferences, and
hopefully that aligns with enough people.

> But I think it's a cool initiative, that would have really helped me if
> it had been available on the Guile website when I first started learning
> Guile.

Yay!

> As an aside, what's the best way to pass you "editorial" feedback (typos
> and such) — as git patches or as inline corrections?

Either works; but git patches are nice.  I tried to set up the format of
documentation so that patching it should be easy while keeping the
source readable.  To demonstrate, here's an example:

 (p [You also might build up some fun toys while running through this
   tutorial.
 You might want to play with them and re-use them without having
   to type them in all over again.
 This is where your text editor comes into play!
 Try opening a new file, we'll call it "sandbox.scm".
 When you build something in this tutorial you'd like to use
   over and over again without retyping it between REPL sessions,
   you can put it here.
 Let's try putting something there now:])

What you'll notice is that each new sentence starts on its own line.
Sentences which have characters which continue beyond line 79 "wrap",
but are indented to be clear that they align with a previous sentence.

This is an unusual convention, but I think a sane one: my usual
temptation is to use emacs' fill-paragraph technique to keep things
looking nice in plaintext, but that can entirely rearrange paragraphs,
and in my experience makes merging changes hard.  I heard the
recommendation a while ago that keeping a sentence on its own line is a
better way to go for version-controlled documentation, but usually that
ends up flying off beyond column 80, and I hate that.  So the above
approach merges the two.

That seems like useful information to document, so I'll do that now. :)

Patches welcome.  Please include an update to the copyright line
including your name if you do so, and acknowledge that you're okay with
it being dual licensed under the LGPL and GFDL!

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Christopher Allan Webber writes:
>
>> Hello!
>>
>> So I've been thinking a bit about what a friendly "intro" tutorial would
>> look like that could fit with the direction the site is heading.  I came
>> up with some ideas I wanted to capture before I totally lost them.
>
> Well I started implementing this.
>
> Here's the code:
>   https://notabug.org/cwebber/guile-tutorial

Oh right, and a preview:
  http://dustycloud.org/misc/guile-tutorial.html

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Hello!
>
> So I've been thinking a bit about what a friendly "intro" tutorial would
> look like that could fit with the direction the site is heading.  I came
> up with some ideas I wanted to capture before I totally lost them.

Well I started implementing this.

Here's the code:
  https://notabug.org/cwebber/guile-tutorial

It's using the excellent Skribilo.  (I had *no idea* just how excellent until
I started this... amazing stuff!)

So:

> I think we can keep with the kids playing with robot toys idea and
> stretch that a bit.  Here's a brief outline:
>
>
>  - Intro
>  - Getting up and running
>(picture of one of those robots with a wind-up-toy-key on its back?)
>+ How to start guile from the command line, add readline support
>+ Editor setup, simple
>
>  Details how to write some scheme with any editor, maybe makes a
>  free software editor recommendation of something simple that's not
>  too hard to get going with Scheme.  Would GEdit work?
>
>  Shows how to write a file and then import it at the REPL,
>  then reload it as you add things.
>
>  Teaches the basic idea of writing code in a file + playing at the REPL.
>
>+ Editor setup, advanced: Emacs + Geiser
>
>  Explains that this is the advanced, but recommended version.
>  It takes some time to get started with if you are not already an
>  emacs user, but you may want to come back to it later.  Explains
>  how to set things up.

I've gotten this far.  The rest of the stuff below still needs to be
done.

I'm still interested in mixing this with sirgazil/Lusis's drawings if he
is.

I haven't gotten into the fun part of the tutorial yet, but I'm kind of
optimistic.  The remaining parts are below.  What do people think, is
this worth spending the time on?  And might we want to put it on the
Guile website officially at some point...?

>  - First steps
>
>Much like The Little Schemer uses food as variable names, I think
>it's a good idea to stick with abstract fun concepts.  Here, I think
>it would be great to continue along with the "Guile is a playground,
>come play!" idea by using toys as variable names, and defining
>procedures that evoke nostalgia for older programmers and sound
>playful for younger ones.
>
>Some ideas:
>  + could use building lists as putting toys in and out of a toy
>chest
>
>(define toy-chest '(robot teddy-bear doll-with-comb toy-soldier))
>
>  + could have a simple-bake-oven set of procedures that takes
>arguments like flavor and dessert-type:
>
>  #> (define (simple-bake-oven flavor dessert-type)
>   (format #f "Yum!  You made a tasty ~a flavored ~a!"
>   flavor dessert-type))
>  #> (simple-bake-oven "banana" "cake")
>  $20 = "Yum!  You made a tasty banana flavored cake!"
>
>and then we can increase the advanced features a bit:
>
>  #> (define* (fancy-bake-oven flavor dessert-type
>  #:optional topping)
>   (if topping
>   (format #f "Yum!  You made a tasty ~a flavored ~a covered 
> in ~a!"
>   flavor dessert-type topping)
>   (format #f "Yum!  You made a tasty ~a flavored ~a!"
>   flavor dessert-type)))
>  #> (fancy-bake-oven "mint" "ice cream" "chocolate fudge")
>  $21 = "Yum!  You made a tasty mint flavored ice cream covered in 
> chocolate fudge!"
>
>Yes... the fancy bake oven version is so fancy it can even bake
>ice cream! ;)
>
>  + Introduce modules as extensions for our robots.
>
> I'm sure there are other things!  But I think a tutorial in this style
> might be fun, and would fit the site well.  And the desire for a good
> tutorial has been expressed many times.
>
> What do others think?

Re: Announcing 8sync: an asynchronous programming language for Guile

[Christopher Allan Webber]

William ML Leslie writes:

> On 6 December 2015 at 01:58, Christopher Allan Webber
> <cweb...@dustycloud.org> wrote:
>> Amirouche Boubekki writes:
>>> 8sync has two types of async-request:
>>>
>>> ** run-requests, which implements kind of a *coroutine* behavior.
>>>
>>> It pause the execution of the current procedure and schedule
>>> the provided lambda to be run soonish; This doesn't exists in
>>> async.scm. The only thing useful this can do, is break the callstack
>>> to allow deeper recursion.
>
> An aside on concurrency models: being able to do this - pause
> execution of one turn and allow another to run - creates a strictly
> more powerful model, but with very different properties for the
> purposes of analysis and auditing.  It means that any time you call a
> function, you need to ensure that if there are properties that a turn
> must maintain - such as there being a constant amount of money in the
> system, for example - that this property is maintained /before/ the
> call.
>
> This feature is known as 'stale stack frames' in the literature
> (particularly in E circles), and in an ideal async world, we'd never
> need to do this.  There's a really accessible post at
> https://glyph.twistedmatrix.com/2014/02/unyielding.html about what can
> happen as codebases grow in a system with the potential for stale
> stack frames.  I also think
> https://wingolog.org/archives/2012/02/16/unexpected-concurrency is
> good to review when discussing this, even though it's dealing with a
> slightly different cause.
>
> I'm not saying that it's a bad idea to expose it in 8sync, btw.  One
> great thing about Guile is that it gives you options about how you
> want to do concurrency.  But libraries that use this feature (or may
> decide to use it in the future) should come with a large, flashing
> warning - calling these functions may allow other events to run!

You're right that these are challenging problems... there are a number
of ways to mitigate them, like using the actor model, etc (which of
course still doesn't give you a perfect model of concurrency) or build a
purely funcitonal system on top where you *expect* that a function is
only computing based off of the information that came into the system,
or many other approaches.

But I think it is possible, and best, to leave answering this question
out of the low-level part of 8sync's scheduler.  Instead, systems can be
built on top of it, both bundled with 8sync, and outside of it as
separate packages.  I think that's the best approach... I already intend
to build an actor model system on top of 8sync and attempt to experiment
with some other systems too.  I'd like 8sync's core to be generic enough
where multiple approaches can be built *on top* of it.

I think the current path 8sync is going down makes that possible.

Anyway, good things to keep in mind! :)

Re: Announcing 8sync: an asynchronous programming language for Guile

[Christopher Allan Webber]

Amirouche Boubekki writes:

>> Le 2015-12-04 03:47, Amirouche Boubekki a écrit :
>> 
>>>Le 2015-12-04 03:15, Amirouche Boubekki a écrit :
>>> 
>>>```
>>>(define (read/ sock)
>>>  (abort-to-prompt 'loop (lambda (cc)
>>>   (loop-add-reader sock (lambda () (cc 
>>> (read
>>>sock)))
>>>```
>>> 
>> 
>>This is a mistake, it *must* be a macro
>> 
>
> This will only help with debugging but not catching error and
> dealing with them at runtime ie. it doesn't propagate the exception
> but it can be done and requires a form similar to 
> propagate-%async-exceptions.
>
> TBH the code still seems complex maybe complicated. FWIW here is
> what I think right now. Be warned that everything should be taken
> with a grain of salt.
>
> * Agenda
>
> ** The canonical callback based event loop API is not visible enough
>
> It should be obvious coming from outside Guile world what/where
> is the event loop. As such, agenda doesn't seem like a good name.

"Agenda" is not a unique name for this.  That's what SICP uses, that's
what Sly uses.

> ** Agenda has both `schedule` and `queue`
>
> For a proof of concept, queue/schedule is not useful to demonstrate
> the purpose of eightsync as it's an optimization.

It's no mere optimization.  "schedule" is future events that haven't
been queued; it's only for time-delayed events.  The queue is for things
that must be done immediately.

This separation is very intentional.

> ** `(current-agenda-prompt)`
>
> No need to throw, there should always be a current agenda.

That's not true, if no agenda has been started, %current-agenda will be
#f, so there will be no way to retrieve the current agenda's prompt tag.

> * 
>
> Schedule should be in its own file. Also, in place of:
>
> ```
> (define-record-type 
>(make-schedule-intern segments)
>schedule?
>(segments schedule-segments set-schedule-segments!))
> ```
>
> I prefer:
>
> ```
> (define-record-type 
>(%make-schedule segments)
>schedule?
>(segments schedule-segments-ref schedule-segments-set!))
> ```

%make-schedule might be better.

> Use of `*-ref` and `*-set!`.

-ref is extra typing.  Doing just schedule-segments fits within
conventions shown elsewhere in the manual.

I don't really want to encourage mutation; getters are the default.

> Or better: `(define-record-type*  segments)` (actually I
> think this macro should be in Guile, since it much easier to
> introduce  to new devs).
>
> ** `time` procedures have equivalent in Guile

Yeah I might change that.

> *** `tdelta` is not useful.

tdelta is used so the the agenda knows to put it a future time from the
current execution of the agenda.

> * ``
>
> ** `run-it`
>
> It's sound like it's a `(loop-call-soon callback #:optional (delay 0))`
> procedure.
>
> Instead it's a `(make-run-request proc #:optional time)`.
>
> It's never used.
>
> I'd rather to use a `delay`, and let the devs compute the correct delay,
> when they wants to run something at an absolute time to limit the
> proliferation of sugar procedure in the library.

You're right, it's never really used at present.  There is a difference
between (run (foo)) and (run-it foo) which isn't just the optional #:when,
which can indeed be handled by using run-at or run-delay instead.

Here's the difference: run uses (wrap) to do a friendly
wrap-in-a-thunk.  So you can do to this:


  (define (my-handler)
(let ((foo (bar)))
   (run (mork foo

and behind the scenes, it constructs (lambda () (mork foo)) there,
preserving lexically scoped varibles.

If you already have a thunk though, you can just pass it in.  In such a
case you'd normally need to give another layer of indirection with
(run (my-thunk)).

It's an optimization that might not be necessary.  I might remove
it... I'm going to wait and see.

> *** `(wrap e ...)` and `(wrap-apply body)`
>
> I'm pretty sure that `wrap` can be written using `wrap-apply`. This
> looks suspicious for several reasons.

There's a difference between them:

 - (run) uses (wrap) to wrap all body contents in a thunk, preserving
   the illusion that everything looks the same as running code inline,
   as described above.  For most code, that's what you want.  (wrap)
   basically makes making inline thunks a little cleaner, and hands that
   off to other macros which make thunks too.

   This is nice because you can even do something like:

 (run (cond (foo bar)
(baz basil)))

 - However, things that handle ports aren't thunks... they're callbacks
   that take an argument of whatever port it is.  Hence, making a thunk
   doesn't make sense.  But I still want an easy way to provide the
   clean appearance of inline code.  So (wrap-apply) passes along
   whatever arguments.

   You can't pass certain syntactic forms to (apply), for example cond
   above, so it's not as desirable generally.

> First it looks like a delayed call, so why not use force/delay.

It might be 

Re: Announcing 8sync: an asynchronous programming language for Guile

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Christopher Allan Webber writes:
>>
>>> ... and the complicated-number-code will launch asynchronously, but wake
>>> back up the appropriate function appropriately.  You can also set timed
>>> events, read and write to ports asynchronously, and etc.  It has a nice
>>> non-blocking loop, and personally I've found it to be a delight to use.
>>
>> There's one potential long-term caveat to this (maybe?), which is that
>> the way this achieves nice asynchronous communication is by working hand
>> in hand with guile's (select) statement, which seems to only work for
>> socket type ports and file type ports I think?
>
> Yes, ‘select’ is only for “file ports”, which means ports backed by a
> file descriptor.  I guess 8sync will have to provide a higher-level
> interface on top of that?
>
> Ludo’.

Yes, it's actually already kind of extensible (and polling is already an
option in the loop)... though I'm not sure, what kind of things is
(select) insufficient for?  Databases usually expose some sort of port
interface, though Squee doesn't use it, for instance.

I guess maybe stuff like interfacing with GTK might be insufficient, but
I don't know.

Anyway, it seems like a lot can be done with (select).  The main concern
is, if other things aside from (select) are needed, can they be
composed?

For now, I'm forging ahead with (select)... luckily in Unix, many things
are backed by a file descriptor, including sockets... :)

Announcing 8sync: an asynchronous programming language for Guile

[Christopher Allan Webber]

Hello all,

Some of you may remember the thread on this list about Guile needing an
asynchronous programming library, with a writeup from a conversation
David Thompson, Mark Weaver, guest star Andrew Engelbrecht, and I had at
LibrePlanet:

  https://lists.gnu.org/archive/html/guile-devel/2015-10/msg00015.html

Well! I've begun work on an asynchronous programming language called
"8sync" for Guile (pronounced "eight-sync", as in "eight-synchronous
communication" :))!  You can find it here:

  https://notabug.org/cwebber/8sync

A nice little logo here:

  http://dustycloud.org/gfx/goodies/8sync-protologo2.png

Hm, I need a nice website to go with that logo! :)

It makes significant use of delimited continuations.  You can do things
like this:

  (define (my-number-talker)
(format #t
  "Oh boy, looks like we got back ~a!\n"
  (%sync (%run (+ 1 2 3 (complicated-number-code 38))

... and the complicated-number-code will launch asynchronously, but wake
back up the appropriate function appropriately.  You can also set timed
events, read and write to ports asynchronously, and etc.  It has a nice
non-blocking loop, and personally I've found it to be a delight to use.

(Also should I be calling that thing (%sync) or (8sync)?  The jury is
out, so in the meanwhile, it's an alias!)

Anyway, there's a lot to be done... I'm not at a 0.1 release quite yet,
but am rapidly approaching it.

If you're interested in giving it a whirl, there's an IRC bot that ships
with it already in the demos/ directory.  Oh yeah, did I mention that
this makes use of the cooperative REPL system in Guile?  This means you
can *live hack your asynchronous code*.  Pretty cool!  (As such, we've
got a little guest in #guile right now named "syncbot"...)

More to come soon, including documentation, web programming examples, an
actor model implementation layered on top, and more functional
programming utilities, including maybe an Elm-like time traveling
debugger.  (There's currently some mutation under the hood, but all of
it can be refactored to be purely functional... with the exception of a
parameter that's used to find the current agenda.  Even that though,
I'll be writing code so you can schedule explicitly to the agenda
shortly if you really are upset by that one impurity.  Sure makes doing
%sync / 8sync calls nice though...)

Anyway, I'm having a ton of fun with this project.  Give it a
try... maybe you will too!

 - Chris

Re: Announcing 8sync: an asynchronous programming language for Guile

[Christopher Allan Webber]

Christopher Allan Webber writes:

> ... and the complicated-number-code will launch asynchronously, but wake
> back up the appropriate function appropriately.  You can also set timed
> events, read and write to ports asynchronously, and etc.  It has a nice
> non-blocking loop, and personally I've found it to be a delight to use.

There's one potential long-term caveat to this (maybe?), which is that
the way this achieves nice asynchronous communication is by working hand
in hand with guile's (select) statement, which seems to only work for
socket type ports and file type ports I think?  Mark Weaver said he'd
think if there were other options.  I've briefly scanned the relevant C
code but I'm not really sure if there's any way we can expose a more
flexible system or not.

Anyway, socket and file ports cover quite a lot of async stuff, so
that's a great step already!

 - Chris

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Ludovic Courtès writes:

> Howdy!
>
> I like all the ideas, being a fan of the simplicity and welcoming
> aspects of the “Little” books.
>
> So, who’s in?  :-)
>
> Thanks,
> Ludo’.

I'm glad you like it.  I would *like* to be able to do this, but I
officially don't have the time right at the moment.  Unofficially, I
might do it anyway when procrastinating on something "more important" ;)

But that's definitely a "no promises" statement.  If someone else took
the initiative I'd be happy to jump in and help with that!

 - Chris

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Mathieu Lirzin writes:

> Hi Christopher,
>
> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>>  - Intro
>>  - Getting up and running
>>(picture of one of those robots with a wind-up-toy-key on its back?)
>>+ How to start guile from the command line, add readline support
>>+ Editor setup, simple
>>
>>  Details how to write some scheme with any editor, maybe makes a
>>  free software editor recommendation of something simple that's not
>>  too hard to get going with Scheme.  Would GEdit work?
>
> As much as I appreciate my Emacs environment, I think GEdit + basic REPL
> in a terminal is the right choice for presenting Guile.  I think it
> would be better not explaining how to setup Geiser+Emacs in this
> tutorial and just mentioning it by giving a link to separate resource
> (which don't exist yet...) for those who are already familiar with
> Emacs.

That's roughly what I was going for!  I figured there would be a very
minimal introduction on how to set up a hacking environment with *any*
editor, then those who are ready to set up geiser + emacs can figure out
how to do it, after being pointed in this direction anyway.

>> I'm sure there are other things!  But I think a tutorial in this style
>> might be fun, and would fit the site well.  And the desire for a good
>> tutorial has been expressed many times.
>
> This is definitely a great initiative!
>
> Thanks,

Great, thank you!

(I hope I or someone else have time to do it soon...!)
 - Chris

Re: Asynchronous event loop brainstorm at FSF 30

[Christopher Allan Webber]

Mikael Djurfeldt writes:

> Den 4 okt 2015 02:30 skrev "Christopher Allan Webber" <
> cweb...@dustycloud.org>:
>>  - This would be like asyncio or node.js, asynchronous but *not* OS
>>thread based (it's too much work to make much of Guile fit around
>>that for now)
>
> Why is this (too much work for threads)?

Threads bring a lot of risky problems.  I really don't want to deal with
that much locking.  A lot of Guile's code isn't thread-safe... if we
wanted to go to the "oh yeah super safe with threads!" direction,
it might require something like Clojure's software transactional
memory.  I talked to Mark Weaver about this; it's very expensive to do,
super hard to implement (I don't think we have any guile devs
interested), and makes things slower whenever you *aren't* using
threads.

The asyncio / node.js style of things can solve IO bound problems.  As
for CPU bound, we can use message passing between threads or processes.

It's beneficial to focus on message passing for CPU bound issues anyway,
because this means that our code will be able to span across machines,
if said messages are serializable.

Does that make sense?
 - Chris

Re: Asynchronous event loop brainstorm at FSF 30

[Christopher Allan Webber]

Chris Vine writes:

> On Tue, 17 Nov 2015 11:46:24 -0600
> Christopher Allan Webber <cweb...@dustycloud.org> wrote:
> [snip]
>> This sounds very interesting... is the source available?  Could you
>> point to it?
>> 
>> Thanks!
>>  - Chris
>
> No it's not.  I'll email you something.
>
> Chris

Thanks!  I'm reading the code you sent me.  Before I had received your
code, I had started on something yesterday... I'm happy to see we have
some similar ideas.  I might borrow some from you.

I'll be posting what I have shortly, as soon as I have enough little
demos locally to prove whether the thing I wrote makes sense or not.

 - Chris

Re: Asynchronous event loop brainstorm at FSF 30

[Christopher Allan Webber]

Chris Vine writes:

> It is certainly the case that mixing threads with coroutines is usually
> best avoided, otherwise it becomes very difficult to know what code
> ends up running in which particular thread and thread safety becomes a
> nightmare.  However, it would be good to allow a worker thread to post
> an event to the event loop safely, whereby the handler for the posted
> event would run in the event loop thread.  asyncio allows this.
>
> Although not particularly pertinent to this proposal, which looks
> great, I use coroutines implemented with guile's delimited
> continuations for a minimalist "await" wrapper over glib's event loop
> as provided by guile-gnome (the whole thing is about 20 lines of code),
> which appears (to the user) to serialize the GUI or other events posted
> to the event loop.  When I don't want to use guile-gnome, which is most
> of the time, I have my own (also minimalist) thread-safe event loop
> using guile's POSIX wrapper for select.
>
> My uses of guile are pretty undemanding so as I say these are
> minimalist.  Something like asyncio for guile would be very nice indeed.
>
> Chris

This sounds very interesting... is the source available?  Could you
point to it?

Thanks!
 - Chris

[PATCH] gnu: guile: Add guile-irregex.

[Christopher Allan Webber]

Irregex, packaged for Guile!  Horray!

On a side note, it would really be nice to have a
r6rs-build-system... starting to result in a lot of similar spaghetti in
a few of these packages.

I don't know how that works though, so consider that a research project
for another day.

>From 38edbac8ab42e1518424fd2da4d53ece2c96094c Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Thu, 5 Nov 2015 12:20:03 -0600
Subject: [PATCH] gnu: guile: Add guile-irregex.

* gnu/packages/guile.scm (guile-irregex): New variable.
---
 gnu/packages/guile.scm | 79 ++
 1 file changed, 79 insertions(+)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index da75add..5f8ad73 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -478,6 +478,85 @@ slightly from miniKanren mainline.
 See http://minikanren.org/ for more on miniKanren generally.")
 (license expat)))
 
+(define-public guile-irregex
+  (package
+(name "guile-irregex")
+(version "0.9.3")
+(source (origin
+  (method url-fetch)
+  (uri (string-append
+"http://synthcode.com/scheme/irregex/irregex-;
+version
+".tar.gz"))
+  (sha256
+   (base32
+"1b8jl7bycyl2ssp6sb1j24pp9hvqyxm85ki9bmwd50glyyjs5zay"
+(build-system gnu-build-system)
+(arguments
+ `(#:modules ((guix build utils)
+  (ice-9 match)
+  ,@%gnu-build-system-modules)
+   #:phases
+   (modify-phases %standard-phases
+ (delete 'check)
+ (delete 'configure)
+ (delete 'build)
+ (delete 'check)
+ (replace 'install
+   (lambda* (#:key inputs outputs #:allow-other-keys)
+ (begin
+   (use-modules (guix build utils)
+(ice-9 match))
+   (let* ((out (assoc-ref outputs "out"))
+  (module-dir (string-append out "/share/guile/site/2.0"))
+  (source (assoc-ref inputs "source"))
+  (doc (string-append out "/share/doc/guile-irregex/"))
+  (guild (string-append (assoc-ref %build-inputs "guile")
+"/bin/guild")))
+ ;; Make installation directories.
+ (mkdir-p (string-append module-dir "/rx/source"))
+ (mkdir-p doc)
+
+ ;; Compile .scm files and install.
+ (setenv "GUILE_AUTO_COMPILE" "0")
+
+ (for-each (lambda (copy-info)
+ (match copy-info
+   ((src-file dest-file-basis)
+(let* ((dest-file (string-append
+   module-dir dest-file-basis
+   ".scm"))
+   (go-file (string-append
+ module-dir dest-file-basis
+ ".go")))
+  ;; Install source module.
+  (copy-file src-file
+ dest-file)
+  ;; Install compiled module.
+  (unless (zero? (system* guild "compile"
+  "-L" (getcwd)
+  "-o" go-file
+  src-file))
+(error (format #f "Failed to compile ~s to ~s!"
+   src-file dest-file)))
+   '(("irregex-guile.scm" "/rx/irregex")
+ ("irregex.scm" "/rx/source/irregex")
+ ;; Not really reachable via guile's packaging system,
+ ;; but nice to have around
+ ("irregex-utils.scm" "/rx/source/irregex-utils")))
+
+ ;; Also copy over the README.
+ (install-file "irregex.html" doc)
+ #t)))
+(inputs
+ `(("guile" ,guile-2.0)))
+(home-page "http://synthcode.com/scheme/irregex;)
+(synopsis "S-expression based regular expressions, packaged for Guile")
+(description
+ "Irregex is an s-expression based alternative to your classic
+string-based regular expressions.  It implements SRFI 115 and is deeply
+inspired by the SCSH regular expression system.")
+(license bsd-3)))
 
 ;; There are two guile-gdbm packages, one using the FFI and one with
 ;; direct C bindings, hence the verbose name.
-- 
2.1.4

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Luis Felipe López Acevedo writes:

> On 2015-10-20 17:45, BCG wrote:
>> On 10/19/2015 12:29 PM, Christopher Allan Webber wrote:
>>> Amirouche Boubekki writes:
>>> 
>>>> To the contrary I think it's not a good idea to start upfront the
>>>> tutorial with which editor should be used is a good way to loose half 
>>>> of
>>>> the readers, because they will feel more uncomfortable and not up to 
>>>> the
>>>> task. To be useful emacs requires one 'Getting Started' tutorial in
>>>> itself. Or anykind of setup for that matter. The tutorial should go 
>>>> as
>>>> quickly as possible to the matter and start with coding.
>>>> 
>>>> I started a tutorial at http://hypermove.net/. I don't introduce
>>>> readline until the user knows what the REPL is. Part 1 is all done in
>>>> REPL. I subtly introduce emacs as a good choice for an editor in part 
>>>> 2
>>>> which is done in an editor. But doesn't enforce it.
>>> Okay, sorry if I wasn't being clear... my goal in that section would 
>>> be
>>> to encourage everyone who *isn't already an emacs user* to pick up a
>>> simple editor and know how to use that with Guile, but them give them 
>>> a
>>> brief tip that they want to look into Emacs + Geiser once they've 
>>> dived
>>> in a bit more deeply.
>>> 
>>> I don't think this section needs to be too long.  I agree it should 
>>> not
>>> be overwhelming.
>>> 
>> Personally I would appreciate a not-so-brief tip.  Most resources on
>> the web about coding in scheme seem to claim that emacs is the best
>> environment for it, but rarely go into the reasoning about why or
>> provide a focused way to get started with it.  I'd be happy to try out
>> those tools, but it hardly is worth it to me to slog through the emacs
>> tutorial just to see if I like whatever advantages it may have for
>> coding Scheme.
>
> I think a video tutorial to get the user started with Guile + Emacs 
> would be great for that. First video in a "Learn GNU Guile" series 
> hosted in a MediaGoblin instance :)

That would be great, yes!

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

BCG writes:

> On 10/19/2015 12:29 PM, Christopher Allan Webber wrote:
>> Amirouche Boubekki writes:
>>
>>> To the contrary I think it's not a good idea to start upfront the
>>> tutorial with which editor should be used is a good way to loose half of
>>> the readers, because they will feel more uncomfortable and not up to the
>>> task. To be useful emacs requires one 'Getting Started' tutorial in
>>> itself. Or anykind of setup for that matter. The tutorial should go as
>>> quickly as possible to the matter and start with coding.
>>>
>>> I started a tutorial at http://hypermove.net/. I don't introduce
>>> readline until the user knows what the REPL is. Part 1 is all done in
>>> REPL. I subtly introduce emacs as a good choice for an editor in part 2
>>> which is done in an editor. But doesn't enforce it.
>> Okay, sorry if I wasn't being clear... my goal in that section would be
>> to encourage everyone who *isn't already an emacs user* to pick up a
>> simple editor and know how to use that with Guile, but them give them a
>> brief tip that they want to look into Emacs + Geiser once they've dived
>> in a bit more deeply.
>>
>> I don't think this section needs to be too long.  I agree it should not
>> be overwhelming.
>>
> Personally I would appreciate a not-so-brief tip.  Most resources on the 
> web about coding in scheme seem to claim that emacs is the best 
> environment for it, but rarely go into the reasoning about why or 
> provide a focused way to get started with it.  I'd be happy to try out 
> those tools, but it hardly is worth it to me to slog through the emacs 
> tutorial just to see if I like whatever advantages it may have for 
> coding Scheme.
>
> After so many years of vi muscle memory I just can't find justification 
> to slow down my productivity and learn emacs... but I wouldn't mind 
> doing it just for Guile coding where I'm not really productive yet 
> anyways - especially if there was getting started guide focused on that 
> with pointers to other resources for a deeper dive if I like it.
>
> Even if I didn't end up switching to emacs for everything, I may find 
> that I want to use it just for Scheme... just like I fire up Eclipse 
> when I want to work on Java, but I don't bother using Eclipse for 
> anything else
>
> It doesn't seem like a suggestion for a development environment should 
> distract too much from a tutorial, but maybe that is too much to expect 
> when your audience includes hackers. ;)
>
> Just my opinion, which is worth what you paid for it.
>
> -- Ben

Thanks for the input!  Maybe we can go both ways... keep it brief in the
tutorial, but have a link to "for more on why this is, see this link"
which goes to the manual, or some blogpost, or other resource?

Re: The Guile community we could have (or: solving Worse is More Accessible)

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> To put it another way, many here have probably read the "Worse is
>> Better" essays.  That paradox of "better" I think sticks in the craw of
>> many lispers; something both seems wrong and yet frustratingly right
>> about it.  Maybe another way to look at it: "Worse is more accessible",
>> or "worse has the path of least resistance", or also importantly, "worse
>> is where I felt welcomed."  Given that, no wonder many people feel safe
>> and better rather than what lispers believe is "the right thing".
>>
>> But maybe we could have both.  I believe it requires effort, and perhaps
>> a rebranding of imagery.  The wizard towers are still there if you want
>> them... I doubt any lisper or schemer will give up on Guile or Scheme
>> because of some fun and inviting drawings.  The fun drawings don't take
>> anything away!  But they add something: they make Guile feel like
>> somplace, even as a newcomer, you might belong.
>
> I’ve had your message and thoughts in the back of my mind for some time
> and yes, your characterization of wizardry and the message it conveys to
> newcomers, suggesting both a place where “magical” things happen and an
> intimidating community, makes a lot of sense to me.
>
> The new web site seems like a way to improve that image, indeed!
>
> Ludo’.

\o/

Re: Gambda Logo

[Christopher Allan Webber]

Thorsten Wilms writes:

> Hi!
>
> As mentioned, the next iteration with custom type. The Gambda lost the 
> hard bend and varies in line width:
> http://i.imgur.com/awkMTzX.png

Wow, big improvements!

Re: Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Amirouche Boubekki writes:

> To the contrary I think it's not a good idea to start upfront the 
> tutorial with which editor should be used is a good way to loose half of 
> the readers, because they will feel more uncomfortable and not up to the 
> task. To be useful emacs requires one 'Getting Started' tutorial in 
> itself. Or anykind of setup for that matter. The tutorial should go as 
> quickly as possible to the matter and start with coding.
>
> I started a tutorial at http://hypermove.net/. I don't introduce 
> readline until the user knows what the REPL is. Part 1 is all done in 
> REPL. I subtly introduce emacs as a good choice for an editor in part 2 
> which is done in an editor. But doesn't enforce it.

Okay, sorry if I wasn't being clear... my goal in that section would be
to encourage everyone who *isn't already an emacs user* to pick up a
simple editor and know how to use that with Guile, but them give them a
brief tip that they want to look into Emacs + Geiser once they've dived
in a bit more deeply.

I don't think this section needs to be too long.  I agree it should not
be overwhelming.

>> The "Getting up and running" section looks like a good part to start 
>> materializing the tutorial.
>
> I sent a previous mail to the ML describing my work. Again feel free to 
> comment.

We discussed this on IRC, but I think your tutorial and the one I
proposed above target different audiences, which is great!  Having
multiple tutorials is very helpful.

Ideas for a Guile tutorial to go with the new site

[Christopher Allan Webber]

Hello!

So I've been thinking a bit about what a friendly "intro" tutorial would
look like that could fit with the direction the site is heading.  I came
up with some ideas I wanted to capture before I totally lost them.

I think we can keep with the kids playing with robot toys idea and
stretch that a bit.  Here's a brief outline:


 - Intro
 - Getting up and running
   (picture of one of those robots with a wind-up-toy-key on its back?)
   + How to start guile from the command line, add readline support
   + Editor setup, simple

 Details how to write some scheme with any editor, maybe makes a
 free software editor recommendation of something simple that's not
 too hard to get going with Scheme.  Would GEdit work?

 Shows how to write a file and then import it at the REPL,
 then reload it as you add things.

 Teaches the basic idea of writing code in a file + playing at the REPL.

   + Editor setup, advanced: Emacs + Geiser
   
 Explains that this is the advanced, but recommended version.
 It takes some time to get started with if you are not already an
 emacs user, but you may want to come back to it later.  Explains
 how to set things up.

 - First steps

   Much like The Little Schemer uses food as variable names, I think
   it's a good idea to stick with abstract fun concepts.  Here, I think
   it would be great to continue along with the "Guile is a playground,
   come play!" idea by using toys as variable names, and defining
   procedures that evoke nostalgia for older programmers and sound
   playful for younger ones.

   Some ideas:
 + could use building lists as putting toys in and out of a toy
   chest

   (define toy-chest '(robot teddy-bear doll-with-comb toy-soldier))

 + could have a simple-bake-oven set of procedures that takes
   arguments like flavor and dessert-type:

 #> (define (simple-bake-oven flavor dessert-type)
  (format #f "Yum!  You made a tasty ~a flavored ~a!"
  flavor dessert-type))
 #> (simple-bake-oven "banana" "cake")
 $20 = "Yum!  You made a tasty banana flavored cake!"

   and then we can increase the advanced features a bit:

 #> (define* (fancy-bake-oven flavor dessert-type
 #:optional topping)
  (if topping
  (format #f "Yum!  You made a tasty ~a flavored ~a covered in 
~a!"
  flavor dessert-type topping)
  (format #f "Yum!  You made a tasty ~a flavored ~a!"
  flavor dessert-type)))
 #> (fancy-bake-oven "mint" "ice cream" "chocolate fudge")
 $21 = "Yum!  You made a tasty mint flavored ice cream covered in 
chocolate fudge!"

   Yes... the fancy bake oven version is so fancy it can even bake
   ice cream! ;)

 + Introduce modules as extensions for our robots.

I'm sure there are other things!  But I think a tutorial in this style
might be fun, and would fit the site well.  And the desire for a good
tutorial has been expressed many times.

What do others think?

Re: Gambda Logo

[Christopher Allan Webber]

Thorsten Wilms writes:

> Hi!
>
> I guess the silence (with the one exception, thank you Amirouche!) means 
> I should stop and go away, but I couldn't help it ... here's a 
> comparison using Luis's work and a small refinement:
> http://i.imgur.com/qM0k0Xq.png

It looks really nice!  I think I still prefer the original logo for its
minimalism.  However, it would kind of be a shame to let the Gambda,
which is probably the nicest Lambda-including logo I've ever seen, go to
waste.

I wonder even if it doesn't get used for Guile (and maybe it will?) if
it could be used for some other Guile using project that's up and
coming?  Hell, "Gambda" would be a great name for a library.

Re: New logo and website design proposal

[Christopher Allan Webber]

Excellent!  I can't wait to see this go live! \o/

Thanks for all your hard work Luis / sirgzil :)

Asynchronous event loop brainstorm at FSF 30

[Christopher Allan Webber]

So David Thompson, Mark Weaver, Andrew Engelbrecht and I sat down to
talk over how we might go about an asynchronous event loop in Guile that
might be fairly extensible.  Here are some of what we discussed, in
bullet points:

 - General idea is to do something coroutine based.

 - This would be like asyncio or node.js, asynchronous but *not* OS
   thread based (it's too much work to make much of Guile fit around
   that for now)

 - If you really need to maximize your multiple cores, you can do
   multiple processes with message passing anyway

 - Initially, this would probably providing a general API for
   coroutines.  Mark thinks delimited continuations would not be as
   efficient as he'd like, but we think it's okay because we could
   provide a nice abstraction where maybe something nicer could be
   swapped out later, so delimited continuations could at least be a
   starting point.

 - So what we really need is a nice API for how to do coroutines, write
   asynchronous code, and work with some event loop with a scheduler

 - On top of this, "fancier" high level systems like an actor model or
   something like Sly's functional reactive programming system could be
   done.

 - Probably a good way to start on this would be to use libuv (or is it
   libev?) and prototype this.  It's not clear if that's a good long
   term approach (eg, I think it doesn't work on the HURD for those who
   care about that, and Guix certainly does)

 - Binary/custom ports could be a nice means of abstraction for this

So, that's our thoughts, maybe someone or one of us or maybe even *you*
will be inspired to start from here?

To invoke Ludo, WDYT?
 - Not Ludo

Re: Request for feedback on SRFI-126

[Christopher Allan Webber]

Taylan Ulrich Bayırlı/Kammer writes:

> I will probably work on a delimited continuations SRFI, heavily inspired
> by Guile's call-with-prompt, since I find it *immensely* more easy to
> grok than shift/reset and prompt/control because those mingle together
> the "stack slice" and the "handler" code whereas call-with-prompt
> clearly separates them.

This could be interesting!  I wonder how hard it will be to do?  IIRC
from reading wingolog, delimited continuations need to be planned in at
a pretty low level, and I wonder how hard it will be for many
implementations to bring them in who don't have them at present?  Will
be interesting to see!

Anyway, interesting to hear, good luck!
 - Chris

The Guile community we could have (or: solving Worse is More Accessible)

[Christopher Allan Webber]

Luis Felipe López Acevedo writes:

>> Really the design is really well made, no problem with that, I just
>> want to be sure that we won't disappoint the main audience and
>> attract a lot of newsbies which i think we are not ready for.
>> 
>> If I can vote, I will +0.
>
> I get you point about the illustrations, but I disagree :)
>
> My Idea was to put "newbies" first without leaving out the experts. 
> Maybe with this change we can push ourselves a bit to make the project 
> be ready for people getting started in programming (it is not far from 
> that, I think).

I think this is absolutely the right goal, and the right call.  I am
100% behind the visual design, partly for this very reason!  I like the
design, it is the right level of modern, crisp and professional design,
but with just the right touch of playfulness.  That playfulness is
critical, I think.  Consider the community we have, and the community we
would like to bring in?

>From what I've gathered, the traditional and public view of lisp and
scheme is of some ancient wizards (perhaps not so far from the cover of
SICP) casting spells from towers or potentially labs, maybe AI
labs... I love wizards, and lisp's history has a lot to thank of fairly
wizardly characters, but while wizards evoke a sense of magic, it is a
kind of unapproachable magic.

The new design appeals to me: it retains a kind of magic, but it is the
kind of magic of childhood, and when Guile is at its best, this is
true.  Guile is a magical, wonderful playground, and one whose limits is
your imagination and your patience.  Perhaps you will indeed become a
powerful wizard, but maybe you will become a magician of some other
type.  But no matter what, magic is under your fingertips from the very
beginning.  Come play along!

If you'll permit me to ramble a bit, I think Guile's community is at an
opportune time.  It's not visible to the outside world so much yet, but
Guile's community is a deeply welcoming one... something that was
surprising to me when I entered.  I've heard from plenty of others in
the community that one of the reasons this or that person stayed is
because of the level of patience and care given to questions, and this
is a great sign.

We aren't a diverse community... look at the gender distribution on
these mailing lists, and that's telling enough.  But Guile has the right
environment to become a better.  Guile's three maintainers, Ludo, Andy
and Mark, have all expressed interest in becoming an even more welcoming
and diverse place.  But that requires hard work.  There are lots of
steps to be made, but how we present ourselves is foundational.

To put it another way, many here have probably read the "Worse is
Better" essays.  That paradox of "better" I think sticks in the craw of
many lispers; something both seems wrong and yet frustratingly right
about it.  Maybe another way to look at it: "Worse is more accessible",
or "worse has the path of least resistance", or also importantly, "worse
is where I felt welcomed."  Given that, no wonder many people feel safe
and better rather than what lispers believe is "the right thing".

But maybe we could have both.  I believe it requires effort, and perhaps
a rebranding of imagery.  The wizard towers are still there if you want
them... I doubt any lisper or schemer will give up on Guile or Scheme
because of some fun and inviting drawings.  The fun drawings don't take
anything away!  But they add something: they make Guile feel like
somplace, even as a newcomer, you might belong.

There's plenty more to do (and to be said) to make that true, but I
think this redesign is a great step in that direction.

 - Chris

PS: As for the "it's not professional enough", most companies follow
  where developers want to go these days anyway, and those who are so
  entrenched in professionalism are probably "enterprise software"
  shops.  Those companies are already so deeply entrenched in Java or
  .NET that I wouldn't worry about them at this point. :)

Adding https support

[Christopher Allan Webber]

Hello!

So, Guile currently lacks https support, which I think is... strange in
the present day!  I've currently hit a limit with what I can do in
implementing federation tools using Guile without https support.
Luckily, I was pointed to Guix's guix/build/download.scm containing
https support.  Ludovic said he'd be fine with having his
https-supporting code in Guile core and under the LGPL, and I offered to
port it.  The good news: I have it working locally separated from Guix.
Attached is a patch with the current state of things.  It isn't done
though!

There are remaining issues:
 - The tls file descriptor leak bug from Guix has been carried over here
   http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20145
   but I don't really know enough to know what I'm supposed to fix.
   Pointers?
 - open-socket-for-uri and open-connection-for-uri should be merged
   together.
 - needs a better commit message, I'll get to it!
 - I probably need to sign papers... I've signed them for other GNU
   projects but I think I haven't signed any kind of across-the-board
   GNU copyright assignment thing.

Thoughts?
 - Chris

>From 5df084b42bf6633af8107d6c994f7171afb04a84 Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Thu, 17 Sep 2015 15:14:54 -0500
Subject: [PATCH] Preliminary but mostly-working addition of https support to
 guile

---
 module/web/client.scm | 108 +-
 1 file changed, 107 insertions(+), 1 deletion(-)

diff --git a/module/web/client.scm b/module/web/client.scm
index 070b0c3..4159f73 100644
--- a/module/web/client.scm
+++ b/module/web/client.scm
@@ -1,6 +1,6 @@
 ;;; Web client
 
-;; Copyright (C) 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
+;; Copyright (C) 2011, 2012, 2013, 2014, 2015 Free Software Foundation, Inc.
 
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -45,6 +45,7 @@
   #:use-module (srfi srfi-9 gnu)
   #:export (current-http-proxy
 open-socket-for-uri
+open-connection-for-uri
 http-get
 http-get*
 http-head
@@ -54,11 +55,116 @@
 http-trace
 http-options))
 
+(define %http-receive-buffer-size
+  ;; Size of the HTTP receive buffer.
+  65536)
+
+;; Provide access to the gnutls-module, but fail gracefully if not available.
+;; Why take this route and not just straight up import the module?
+;; Guile can't depend on gnutls because gnutls includes Guile as a dependency.
+;; There's some risk of dependency cycles, so lazily resolving things only
+;; once needed helps!
+
+(define gnutls-module
+  (delay
+(catch 'misc-error
+  (lambda ()
+(resolve-interface '(gnutls)))
+  (lambda _
+(format (current-error-port)
+"warning: (gnutls) module not available\n")
+#f
+
+(define (ensure-gnutls)
+  (if (not (force gnutls-module))
+  (error "(gnutls) module not available")))
+
+(define (gnutls-ref symbol)
+  "Fetch method-symbol from the gnutls module"
+  (ensure-gnutls)
+  (module-ref (force gnutls-module) symbol))
+
 (define current-http-proxy
   (make-parameter (let ((proxy (getenv "http_proxy")))
 (and (not (equal? proxy ""))
  proxy
 
+(define add-weak-reference
+  (let ((table (make-weak-key-hash-table)))
+(lambda (from to)
+  "Hold a weak reference from FROM to TO."
+  (hashq-set! table from to
+
+(define (tls-wrap port server)
+  "Return PORT wrapped in a TLS connection to SERVER.  SERVER must be a DNS
+host name without trailing dot."
+  (define (log level str)
+(format (current-error-port)
+"gnutls: [~a|~a] ~a" (getpid) level str))
+
+  (ensure-gnutls)
+
+  (let ((session ((gnutls-ref 'make-session)
+  (gnutls-ref 'connection-end/client
+
+;; Some servers such as 'cloud.github.com' require the client to support
+;; the 'SERVER NAME' extension.  However, 'set-session-server-name!' is
+;; not available in older GnuTLS releases.  See
+;; <http://bugs.gnu.org/18526> for details.
+(if (module-defined? (force gnutls-module)
+ 'set-session-server-name!)
+((gnutls-ref 'set-session-server-name!)
+ session (gnutls-ref 'server-name-type/dns) server)
+(format (current-error-port)
+"warning: TLS 'SERVER NAME' extension not supported~%"))
+
+((gnutls-ref 'set-session-transport-fd!) session (fileno port))
+((gnutls-ref 'set-session-default-priority!) session)
+((gnutls-ref 'set-session-credentials!) session
+ ((gnutls-ref 'make-certificate-credentials)))
+
+;; Uncomment the following lines in case of debugging emergency.
+;;(set-log-level! 10)
+;;(set-log-procedure! log)
+
+((gnutls-ref

Re: New logo and website design proposal

[Christopher Allan Webber]

Luis Felipe López Acevedo writes:

> Hello list,
>
> As part of the Guile project I imagine, I'd like to propose these visual 
> modifications for the logo and the website.
>
> Logo
> https://multimedialib.files.wordpress.com/2015/09/guile-logo-proposal-2015-06-16.png
>
> Website mockup (some rough sketches here)
> https://multimedialib.files.wordpress.com/2015/09/website-mockup-2015-09-15.png
>
> My intention with this proposal is, hopefully, to modernize the image of 
> the project a bit and make it more attractive to potential users.
>
>
> Your comments are very welcomed,

First of all, I think this looks *amazing*!  Will it keep these kind of
nice black and white drawings, or will the final result have a more
refined look?  Either way, I like the childrens' book type
illustrations.  (My request would be that if they do become colored, the
two children-programmers whose bodies are somewhat visible have
different skin tones for diversity reasons.  I think they look gender
ambiguous enough because of the suits.)  Honestly I think the rough
black and white versions look very nice as-is though, and I am not sure
how to color them without a kind of watercolor look.  Very "Where the
Wild Things Are" meets robots.  A good combination!

Second, a friend of mine pointed out a couple of things:

 - The "Guile is an extension language platform" section gives the
   impression maybe overly that it's an extension language far and above
   being an independent language.  Guile is certainly optimized for
   this, especially for legacy reasons, but more and more Guile
   applications seem to be written entirely within Guile itself.  Maybe
   the text could be something like:

 "Guile contains a neat and highly optimized virtual machine.
 It can be used out of the box to write programs in Scheme, or
 can easily be integrated with C and C++ programs.

 Guile is the GNU Ubiquitous Intelligent Language for Extensions,
 and the official extension language of the GNU project."
   
   That's a bit more verbose than the present text; I wonder if it can
   be cleaned up?

 - It may be nice to show code examples.  Lisp is often interpreted as
   being hard to read; I have found that a nice theme and
   rainbow-parentheses has reduced that fear for some users.  Here's
   some examples of screenshots I have taken of my own emacs with
   various lisps... none of them are perfect, and this isn't scheme
   code, but to get the general idea:

 http://dustycloud.org/tmp/emacs_lisp_setup.png
   ^- Probably could do without the highlighting?

 http://dustycloud.org/tmp/emacs-rainbow-all-the-things.png
   ^- Probably could do without the rainbow-delimeters?

 http://dustycloud.org/tmp/emacs_wombat.png
   ^- Something different, but also a nice theme

   I'm not 100% confident on this; showing "real code" might actually
   clutter things or make things more intimidating.  The present
   design I think is fairly intimidation-free.

Anyway, what do you think?

I love the designs... keep on rocking!  I can't wait to see this
deployed!

 - Chris

Re: [PATCH] ice-9: Add JSON module.

[Christopher Allan Webber]

I tested this, including with the fixes specified in the email.
(ice-9 json) seems to be working great... I'd love to see it
merged!

Re: Musings on performance, 2.2 and GOOPS (found factor 30)

[Christopher Allan Webber]

Jan Nieuwenhuizen writes:

 Jan Nieuwenhuizen writes:

 I have an application of about 10.000 LOC using Guile/GOOPS with
 performance problems.  I found a way (thanks Ludo!) to run it with
 and without GOOPS, and gained a factor of 30 in performance.  See
 details below.

 I just found out about Andy's wip-goops-refactor branch.  Using that
 gives me almost the same benefit (only 15-20% slower), without any of
 the modifications that I made over the past two weeks.

 Greetings, Jan

That's very cool to hear!
 - Chris