Re: [racket-users] Multiple namespaces in Racket

[Anthony Carrico]

> https://lexi-lambda.github.io/blog/2017/10/27/a-space-of-their-own-adding-a-type-namespace-to-hackett/

Nice summary. Whoever writes the great macro programming book in the sky
would do well to review this post. Good work as usual Alexis.

-- 
Anthony Carrico

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

This is now merged, and I ended up managing to simplify the
implementation somewhat, though the bulk of the changes were still
necessary. I’ve written up the details of how it works in a blog post
here:

https://lexi-lambda.github.io/blog/2017/10/27/a-space-of-their-own-adding-a-type-namespace-to-hackett/

And with that, I’ll stop bumping this thread if nobody else replies to
it — I just wanted to record the details of my solution to the problem
for posterity. :)

Thanks to everyone who helped,
Alexis

> On Oct 25, 2017, at 10:33 AM, Alexis King 
> wrote:
> 
> I finally did this and got everything working, mostly. I have opened
> a pull request with the current state of the namespaced
> implementation:
> 
>https://github.com/lexi-lambda/hackett/pull/49
> 
> Comments are welcome.
> 
> Alexis

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

> On Oct 18, 2017, at 1:08 PM, Matthias Felleisen 
> wrote:
> 
> What I imagined was that you’d define @h-examples, which would
> translate the former into the latter. — Matthias

I finally did this and got everything working, mostly. I have opened a
pull request with the current state of the namespaced implementation:

https://github.com/lexi-lambda/hackett/pull/49

Comments are welcome.

I ended up needing to inline the definition of scribble/manual’s
`defform` into Hackett’s codebase and tweak it slightly to allow
displaying documented identifiers with different symbolic names. The
resulting forms require documentation authors to prefix all types with
`t:` in Scribble documentation, but it seems essentially invisible to
readers of the documentation, which is good.

All told, this change currently adds a net total of 988 LOC, which is
rather large. However, 497 of those are from Racket libraries I needed
to partially inline into Hackett’s implementation, which means the
weight of this feature is currently about 491 LOC. However, that number
does *not* include creating new versions of `only-in`, `rename-in`, and
friends that respect multiple namespaces, so there is still more work
to do.

The initial modules imports problem is still unsolved. I don’t currently
believe it is solvable without changes to Racket, but I’d like to be
proven wrong. The current solution to Scribble documentation is also
subpar, and I don’t think it provides a nice enough interface for users
documenting their own libraries written in `#lang hackett`, but that
seems possibly fixable. There’s also a lot of code cleanup I need to do
before I feel comfortable merging these changes, but I think they are
close to being functionally finished.

Feedback would be appreciated, though I imagine a lot of these changes
are too far down in the weeds for most potential reviewers. Still, even
small suggestions would likely be helpful here.

Alexis

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Matthias Felleisen]

> On Oct 18, 2017, at 4:04 PM, Alexis King  wrote:
> 
> 
>@(examples
>   (: (Tuple 42 "hello") (#,t:Tuple Integer String)))
> 
> That’s because the `examples` form doesn’t allow escaping, which makes
> sense, since there would be no way to know what `#,t:Tuple` should mean
> when performing sandboxed evaluation. It’s theoretically possible to use
> eval:alts to manually provide a version of the program for typesetting
> and a version for evaluation:
> 
>@(examples
>   (eval:alts (: (Tuple 42 "hello") (#,t:Tuple Integer String))
>  (: (Tuple 42 "hello") (Tuple Integer String
> 
> ...but this sort of defeats the purpose of having examples that are
> automatically typeset.


What I imagined was that you’d define @h-examples, which would 
translate the former into the latter. — Matthias

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

> On Oct 18, 2017, at 8:09 AM, Matthias Felleisen 
> wrote:
> 
> Wouldn’t something like this work for Hackett?

I don’t think so. Perhaps it would be better illustrated with an
example.

First, consider the Hackett definition of, say, the `Maybe` type:

(data (Maybe a) (Just a) Nothing)

This defines two runtime bindings with the following types:

Just : (forall [a] {a -> (Maybe a)})
Nothing : (forall [a] (Maybe a))

It also defines a type-level binding, `Maybe`, which is in the type
namespace.

Now consider the Hackett definition of the 2-tuple type:

(data (Tuple a b) (Tuple a b))

This defines two bindings, one in the value namespace and one in the
type namespace. Troublingly, they are both named `Tuple`.

To work around this duplication, these two bindings are exported as
`Tuple` and `#%hackett-type:Tuple`. When imported, the latter is
unprefixed and injected into the type namespace. But what about
Scribble? I might like to write the following:

@(examples
   (: (Tuple 42 "hello") (Tuple Integer String)))

This will evaluate properly, and it will typeset like this:

> (: (Tuple 42 "hello") (Tuple Integer String)))
: (Tuple Integer String)
(Tuple 42 "hello")

But what about the links? The first `Tuple` should link to the value
constructor (which is actually called `Tuple`), and the second `Tuple`
should link to the type constructor (actually called
`#%hackett-type:Tuple`). It won’t, however, because Scribble will link
them both to the value constructor.

I could do what you describe and create separate bindings for the two:

@(module values-for-label racket
   (require (for-label (only-in hackett Tuple))
scribble/manual)
   (provide Tuple)
   (define Tuple @racket[Tuple]))

@(module types-for-label racket
   (require (for-label (only-in hackett
[#%hackett-type:Tuple Tuple]))
scribble/manual)
   (provide t:Tuple)
   (define t:Tuple @racket[Tuple]))

Now I can use these two different bindings in a paragraph by writing
`@Tuple` or `@t:Tuple`, and they will both typeset as “Tuple”. But this
still doesn’t solve the problem of scribble/example, since the following
doesn’t work:

@(examples
   (: (Tuple 42 "hello") (#,t:Tuple Integer String)))

That’s because the `examples` form doesn’t allow escaping, which makes
sense, since there would be no way to know what `#,t:Tuple` should mean
when performing sandboxed evaluation. It’s theoretically possible to use
eval:alts to manually provide a version of the program for typesetting
and a version for evaluation:

@(examples
   (eval:alts (: (Tuple 42 "hello") (#,t:Tuple Integer String))
  (: (Tuple 42 "hello") (Tuple Integer String

...but this sort of defeats the purpose of having examples that are
automatically typeset.

A solution I alluded to in my previous email would be to provide a
notion of “eval transformers” that work like the existing “element
transformers” that scribble/racket provides. That way, `t:Tuple` could
be both an element transformer and an eval transformer that do the right
thing in each context. Otherwise, I’m not sure if there’s a different
solution (at least without dramatically changing how the documentation
is put together).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Matthias Felleisen]

> On Oct 17, 2017, at 11:02 PM, Alexis King  wrote:
> 
>It’s obvious that  should be in the value namespace and
>  should be in the type namespace. When expanding code, this
> makes things easy, but Scribble doesn’t expand its code examples,
> it merely looks at the lexical information of the source syntax.
> If a `Tuple` type has a data constructor that is also named
> `Tuple`, there is no way for Scribble to know which binding to use.


In HtDP/2e, I need different references to the same identifier in one 
paragraph. 
Here is the fully spelled out mechanism: 

> @(module racket-ormap racket/base 
>(provide racket-ormap)
>(require (for-label racket) (only-in scribble/manual racket))
>(define (racket-ormap) @racket[ormap]))
> 
> @(module isl-ormap racket/base
>(provide isl-ormap)
>(require (for-label lang/htdp-intermediate) (only-in scribble/manual 
> racket))
>(define (isl-ormap) @racket[ormap]))
> 
> @(require (submod "." racket-ormap) (submod "." isl-ormap))

Wouldn’t something like this work for Hackett? 

I understand that Type might just be defined in the preceding 
evaluator example. But that’s a different issue, namely, that 
we often need to accumulate the definitions across several different 
example and interaction items. If you combined those two issues with 
something like the above trick, would you be able to get the bindings
done properly? 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

> On Oct 17, 2017, at 20:11, Philip McGrath 
> wrote:
> 
> It wouldn't solve the problem with shadowing `require`d identifiers,
> but would having `#%module-begin` introduce the `require` (with proper
> lexical context information), instead of doing it with the reader,
> solve at least this part of the problem?

Theoretically, yes. It still doesn’t solve the shadowing problem, but
that seems essentially impossible to do in a totally transparent way.
The main unfortunate thing, however, is that `#%module-begin` doesn’t
know what its module language is. One could theoretically export a
different `#%module-begin` from each of hackett/private/kernel,
hackett/base, and hackett (the three module languages Hackett provides),
each of which introduces a different unhygienic `require`. The downside
to this is that a user extending `#lang hackett` to make a new `#lang`
would likely reuse Hackett’s `#%module-begin`, and they’d be in for a
real surprise when they discovered their types weren’t available in
their new `#lang`!

(It will also probably do strange things when interacting with module*
submodules with #f for the module path, but that can be hacked around —
the implementation of namespaces already does plenty of that.)

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Philip McGrath]

On Tue, Oct 17, 2017 at 10:02 PM, Alexis King  wrote:
>
>
>   1. The issue with module language imports still seems hopeless.


It wouldn't solve the problem with shadowing `require`d identifiers, but
would having `#%module-begin` introduce the `require` (with proper lexical
context information), instead of doing it with the reader, solve at least
this part of the problem?

Furthermore, making this change in the reader causes serious issues when
> using `hackett` as a module language explicitly, not as a `#lang`. This
> is not common when writing most modules, since modern Racket style is to
> always use `#lang`, but module languages still crop up when defining
> submodules. This means a user defining a submodule with `hackett` as the 
> module
> language with have to manually insert the `require`, since there is no
> reader control to introduce it implicitly. This is a wart I have not yet
> been able to resolve.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

I’ve been continuing to work on this for the past two days, and I’ve
managed to get a lot more working. I figured I’d write a (somewhat
shorter) summary of what I have and haven’t solved.

  1. The issue with module language imports still seems hopeless.

  2. I managed to solve the submodule issue! The solution involved a
 change to splicing-syntax-parameterize[1] that improves how it
 handles submodules, plus using make-syntax-delta-introducer instead
 of make-syntax-introducer to preserve the scope across module
 instantiations (thanks to Matthew for that).

  3. Scribble is going to be hard. Hackett builds and the tests pass
 (though the code isn’t the prettiest), and I can even get the docs
 to render, but all the cross-references are broken for types.

 This is hard, because in Hackett, there is a syntactic distinction
 between which pieces of syntax need to be in which namespace.
 Consider expanding the type annotation form:

   (:  )

 It’s obvious that  should be in the value namespace and
  should be in the type namespace. When expanding code, this
 makes things easy, but Scribble doesn’t expand its code examples,
 it merely looks at the lexical information of the source syntax.
 If a `Tuple` type has a data constructor that is also named
 `Tuple`, there is no way for Scribble to know which binding to use.

 I don’t know that this is fixable, not without actually performing
 expansion as part of rendering documentation, and that isn’t
 possible in general because not every piece of typeset code is a
 self-contained program. It seems that the simple solution, to avoid
 needing to reinvent half of Scribble right now, is to annotate
 which bindings are types.

 I’ve managed to get *almost* there by prefixing all the type
 imports with `t:` and binding them to an element transformer
 produced by `make-element-id-transformer` that typesets them
 without the prefix. The trouble is that this doesn’t work with
 scribble/example, since, as far as I can tell, there are not any
 “eval/example transformers” that allow me to tell scribble/example
 to use a symbol without the prefix for evaluation. I’m not sure if
 there’s a better way, and if there isn’t, I’m not sure how hard
 that would be to add.

Those issues aside, multi-namespace Hackett works decently. Again, there
are some unsolved subtleties around the user interface for
hiding/renaming types, but I think those are entirely solvable with
custom require transformers. Check Syntax has no trouble understanding
the more complex binding structure of multi-namespace programs. Interop
with Racket is awkward, since Racket requires flattening the two
namespaces, but this is also solvable by prefixing types when importing
them into a Racket module (and I imagine this will be much less of a
problem once Hackett has a real FFI, so less needs to be done in Racket
to wire the two sides together).

Those interested in the guts of how this works can check out the
“namespaces” branch of the Hackett repo[2]. Of course, that branch will
eventually go away, so that link will eventually break, but my hope is
that it will go away because it has been be merged into Hackett, not
because I’ve given up. In the meantime, any thoughts are appreciated,
especially if you have any suggestions on solving the Scribble problem.

Alexis


[1]: 
https://github.com/racket/racket/commit/1e38918aa9719d212534d2027aa1a3a8b46462dd
[2]: https://github.com/lexi-lambda/hackett/tree/namespaces

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Matthias Felleisen]

> On Oct 16, 2017, at 2:00 PM, Alexis King  wrote:
> 
> Apologies for the very long email, but this issue has ended up being much 
> more nuanced than I had hoped


Thank you for taking the time to write this up. Fascinating — Matthias


-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

I ended up spending the majority of my weekend working on this, and even
after two days, it still doesn’t work quite right. My verdict is that
it seems impossible to do perfectly seamlessly, but it seems possible
(but maybe hard) to get 85% there. Here’s a (rather long) overview of
what I tried and the problems I encountered — some people might have
some suggestions.

I implemented two namespaces, and each namespace corresponds to a
(use-site) syntax introducer. The introducers are stored in syntax
parameters that are parameterized by #%module-begin. I implemented a
~type pattern-expander that uses the type introducer to attach the type
scope to a piece of syntax before parsing it, which provides a nice,
declarative way to annotate which pieces of forms belong in which
namespace. This part actually works great — for single-module programs,
values and types can cleanly use the same symbolic name without problems
— but the devil is in the details.

I ran into two main problems when implementing this, neither of which I
have come up with completely satisfactory solutions to.

  1. Importing and exporting types is complicated.

 My instinct is that Matthew’s suggestion of using submodules for
 namespace management is a good one, but as Alex pointed out, I’m
 not sure how to actually implement it in practice. I tried the
 simpler name mangling scheme, and I managed to get it mostly
 working. Exporting types is done with an explicit `type-out` form
 (though most of the time it isn’t necessary, since `type-out` is
 implicit when providing types with `data` or `class`), which
 applies the type scope and mangles the names by prepending
 `#%hackett-type:` to the beginning. Similarly, on the importing
 side, Hackett actually provides a modified version of `require`
 that wraps the entire set of import specs with a require
 transformer that unmangles the names and injects them into the
 proper scope.

 This works okay, but it has some problems:

   a. Forms like rename-in, only-in, and except-in don’t work on 
  types. I think this is largely unavoidable, since they
  fundamentally don’t understand that there could be multiple
  bindings with the same name, so the solution is to reimplement
  rename-in, only-in, and except-in with versions that allow
  some sort of annotation that a binding being controlled should
  operate on type bindings. For example, a user could write
  something like the following:

(rename-in hackett [(type String) Text])

  This is a little bit of unfortunate extra work, but it doesn’t
  seem fundamentally difficult.

   b. The bigger problem is that this scheme doesn’t work at all for
  types provided by the module language.

  What do I mean by that? Well, note that when a user authors a
  module with `#lang hackett` at the top, the reader eventually
  produces a module form like the following:

(module mod-name hackett
  )

  Types that are built-in to Hackett are provided by that module
  language specification. This is a problem, because they won’t
  be properly unmangled since module languages do not allow any
  customization of how bindings are introduced (unlike
  `require`). This is a real problem, and there doesn’t seem to
  be any good solution available.

  The only real option appears to be having the reader insert a
  `require` in the resulting program, so the resulting module
  looks like this:

(module mod-name hackett
  (require hackett)
  )

   However, this doesn’t really work, because while the bindings
   *are* properly introduced, they are now introduced by a
   `require`, not the module language. This distinction seems
   irrelevant, but it isn’t — bindings brought into scope by
   `require` cannot conflict with other bindings from other
   `require`s. This means that it is now impossible for users
   to shadow names from `#lang hackett` with their own bindings.
   In other languages, this might be okay, but in Racket, it
   isn’t acceptable.

   My current compromise is to at least mitigate the damage by
   only making the inserted `require` import types, so core
   forms and functions can still be shadowed by other imports,
   but types cannot. This is more manageable, but it’s still
   unpleasant and confusing when it causes problems.

   Furthermore, making this change in the reader causes serious
   issues when using `hackett` as a module language explicitly,
   not as a `#lang`. This is not common when writing most
   modules, since modern Racket style is to always use `#lang`,
   but module languages 

Re: [racket-users] Multiple namespaces in Racket

[Hendrik Boom]

On Sun, Oct 15, 2017 at 03:43:42PM -0400, David Christiansen wrote:
> > It would probably cause trouble iif we ever were to generalize to
> > dependent types.
> 
> Sure, but those same problems will be there anyway if the compile-time
> and run-time bindings live in separate namespaces and are allowed to
> bind the same identifier to different things, as is done in languages
> like Haskell.

They aren't namespace issues; they are binding time issues.

-- hendrik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[David Christiansen]

> Ultimately, though, that would mean that it would separate the runtime
> value and type namespaces, but now types and users’ phase 1 bindings
> would share the same namespace, which is still probably confusing and
> unintuitive. Maybe that’s okay? Again, I feel like I’d need to
> understand better why we idiomatically use phase 0 transformer bindings
> for these things — I don’t really get it.

The LCF-style tactic engine that Sam and I have running in the macro
expander uses a transformer binding to _invoke_ the tactic engine, but
all of the individual tactics live in phase 1 bindings. Though there's
no call to syntax-local-eval - they're just used directly.

I would think that, similarly, type ascription could be a macro that
associates entirely phase-1 type values with run-time expressions. But
there's surely aspects of this that I don't see :-)

/David

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[David Christiansen]

> It would probably cause trouble iif we ever were to generalize to
> dependent types.

Sure, but those same problems will be there anyway if the compile-time
and run-time bindings live in separate namespaces and are allowed to
bind the same identifier to different things, as is done in languages
like Haskell.

/David

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Hendrik Boom]

On Sun, Oct 15, 2017 at 11:59:47AM -0400, David Christiansen wrote:
> Hi Alexis,
> 
> What about keeping type bindings separate from program bindings as a
> matter of phase? This seems to me to fit in with the Hindley-Milner
> program, where types exist only at compile time only, and programs
> exist at run time only.

It would probably cause trouble iif we ever were to generalize to 
dependent types.

-- hendrik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

> On Oct 15, 2017, at 08:59, David Christiansen
>  wrote:
> 
> What about keeping type bindings separate from program bindings as a
> matter of phase? This seems to me to fit in with the Hindley-Milner
> program, where types exist only at compile time only, and programs
> exist at run time only.

I thought about this, too, but I’m not sure it’s the right thing to do.
I admit I’ve never really tried replacing phase 0 transformer bindings
with phase 1 bindings, but everyone seems to use the former instead of
the latter, and I’m not even entirely sure why. What is the reason we
use define-syntax + syntax-local-value instead of define-for-syntax +
syntax-local-eval? I’m not sure. Maybe someone can explain that to me.

Ultimately, though, that would mean that it would separate the runtime
value and type namespaces, but now types and users’ phase 1 bindings
would share the same namespace, which is still probably confusing and
unintuitive. Maybe that’s okay? Again, I feel like I’d need to
understand better why we idiomatically use phase 0 transformer bindings
for these things — I don’t really get it.

On a different note, I started working to apply the name mangling scheme
to Hackett, and it mostly works alright. There’s just one major problem
I’ve run into: types provided by #langs. I can provide a custom version
of `require` that unmangles all type identifiers, but as far as I can
tell, I have no way to hook into identifiers provided by a module
language.

This means that I can write (require hackett) and have names be
unmangled and placed into the correct scope, but I can’t write (module
m hackett ...) and have the types be properly imported. I’m not sure
there’s a way around this with any existing technique (save potentially
David’s suggestion of using the phase system). Fundamentally, even if I
found a way to inject a `require` that imports identifiers from the
module language, this is problematic, since additional `require`s cannot
shadow those bindings, and users cannot explicitly hide them.

Furthermore, while I managed to make #lang hackett insert a require that
injects types into the proper scope, it doesn’t work if I use hackett
as a module language instead of as a #lang (because the hack in the
reader that inserts the additional require cannot work in a module
language). I think that means this technique is a bit of a dead-end,
unless I’m missing something significant.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[David Christiansen]

Hi Alexis,

What about keeping type bindings separate from program bindings as a
matter of phase? This seems to me to fit in with the Hindley-Milner
program, where types exist only at compile time only, and programs
exist at run time only.

/David

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

> On Oct 13, 2017, at 18:52, Geoffrey Knauth  wrote:
> 
> Maybe do undisputedly?-evil name-mangling now to get things working,
> then become a saint with a better solution down the road.  Sometimes
> I don't realize how to do something the "good" way until I've traveled
> down the path of doing it some less than good way, then the "good" way
> reveals itself early one morning or late one night, when I least
> expect it.  30 years ago C++ had ugly name mangling, until that ugly
> part of C++ went away 5-10 years later.  The non-mangling way did not
> reveal itself at first.

I decided to take your advice and give this a try. I came up with a
solution using name mangling that seems to work at first blush, though
I’ve only thrown an extremely simple problem at it. The example and its
implementation are available here:

  https://gist.github.com/lexi-lambda/dbd9371f1268f72f9a0b9f8e09e1b46c

I’m not sure if this technique will hold up under additional stress, but
it seems to work alright, so I’m going to try applying it to Hackett
itself.

Additionally, in the process of creating this example, I discovered that
the documentation for splicing-syntax-parameterize appears to be a lie:
splicing-syntax-parameterize works quite well with require transformers.
Perhaps that caveat no longer applies with the set-of-scopes expander?
I hope I’m not relying on intentionally undefined behavior.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Geoffrey Knauth]

On Friday, October 13, 2017 at 1:06:15 PM UTC-4, Alexis King wrote:
>
> At this point, though it is indisputably evil, it seems more feasible 
> to use some name mangling scheme than to expand to a submodule. That 
> would be, of course, deeply unsatisfying, so I would very much like to 
> have a better solution. 
>

Maybe do undisputedly?-evil name-mangling now to get things working, then 
become a saint with a better solution down the road.  Sometimes I don't 
realize how to do something the "good" way until I've traveled down the 
path of doing it some less than good way, then the "good" way reveals 
itself early one morning or late one night, when I least expect it.  30 
years ago C++ had ugly name mangling, until that ugly part of C++ went away 
5-10 years later.  The non-mangling way did not reveal itself at first.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

This is a cool demo! Unfortunately, I have been thinking about the
problem you describe since last night, and I am still totally stumped.
This is something that seems difficult or impossible to paper over with
more macros because #%require and #%provide are, ultimately, given
special treatment by the macroexpander (at least that is my
understanding), and the `expand` subform of #%provide is not something
that can be emulated as a derived concept.

At this point, though it is indisputably evil, it seems more feasible
to use some name mangling scheme than to expand to a submodule. That
would be, of course, deeply unsatisfying, so I would very much like to
have a better solution.

> On Oct 12, 2017, at 6:07 PM, Alex Knauth  wrote:
> 
> I just implemented this, and this is a problem, but the `all-from-out`
> form in general is an ever bigger problem, it's completely broken.
> 
> The first strategy is to just lift the provide form into a submodule,
> but `all-from-out` expects there to be a require in the *same* module,
> `all-from-out` always yells at you with the error, all-from-out: no
> corresponding require.
> 
> The second strategy is to try to resolve the `all-from-out` references
> in the pre-transformer using `expand-export`. However this is still
> broken for `all-from-out` because
> `syntax-local-module-required-identifiers` complains about "not
> currently transforming module provides."
> (syntax-local-transforming-module-provides? is false). All-from-out
> uses syntax-local-module-required-identifiers to determine what
> identifiers should be in it, and that needs to be called within a
> provide-transformer, not a provide pre-transformer.
> 
> To create a new lifted module I need to use a provide-pre-transformer,
> but to resolve all-from-out's properly I need to use a
> provide-transformer. Is there any way around this?
> 
> What I have so far:
> https://gist.github.com/AlexKnauth/b7d9f2e0af1c5b8e2186d6581b1f7e4d
> 
> Alex Knauth

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alex Knauth]

> On Oct 12, 2017, at 6:31 PM, Alexis King  wrote:
> 
>> On Oct 12, 2017, at 1:46 PM, Matthew Flatt  wrote:
>> 
>> You could put all the type exports in a submodule. Then, you do need
>> your own variant of `require`, but that variant can mostly just check
>> for the presence of a type submodule, much the way that TR or
>> `plai-typed` do. If the submodule is available, you can import it with
>> the extra scope for types.
> 
> This makes a lot of sense to me, but I’m not completely sure how to
> implement the providing side. I imagine that users may very well want
> to use Racket’s namespace management features with types, leading to
> pathological declarations like this:
> 
>(provide (except-out (all-from-out foo/bar)
> (type-out ExcludedType)))
> 
> If type-out is a provide transformer that doesn’t actually provide
> anything, it merely expands to a submodule (perhaps using
> `syntax-local-lift-module-end-declaration`), then it can’t possibly
> understand that it should be doing something different when used in
> “excluding” position. Is the unfortunate answer here that I can’t get
> around needing to reimplement my own version of `provide` as well?

I just implemented this, and this is a problem, but the `all-from-out` form in 
general is an ever bigger problem, it's completely broken. 

The first strategy is to just lift the provide form into a submodule, but 
`all-from-out` expects there to be a require in the *same* module, 
`all-from-out` always yells at you with the error, all-from-out: no 
corresponding require.

The second strategy is to try to resolve the `all-from-out` references in the 
pre-transformer using `expand-export`. However this is still broken for 
`all-from-out` because `syntax-local-module-required-identifiers` complains 
about "not currently transforming module provides." 
(syntax-local-transforming-module-provides? is false). All-from-out uses 
syntax-local-module-required-identifiers to determine what identifiers should 
be in it, and that needs to be called within a provide-transformer, not a 
provide pre-transformer.

To create a new lifted module I need to use a provide-pre-transformer, but to 
resolve all-from-out's properly I need to use a provide-transformer. Is there 
any way around this?

What I have so far:
https://gist.github.com/AlexKnauth/b7d9f2e0af1c5b8e2186d6581b1f7e4d 


Alex Knauth

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Alexis King]

> On Oct 12, 2017, at 1:46 PM, Matthew Flatt  wrote:
> 
> You could put all the type exports in a submodule. Then, you do need
> your own variant of `require`, but that variant can mostly just check
> for the presence of a type submodule, much the way that TR or
> `plai-typed` do. If the submodule is available, you can import it with
> the extra scope for types.

This makes a lot of sense to me, but I’m not completely sure how to
implement the providing side. I imagine that users may very well want
to use Racket’s namespace management features with types, leading to
pathological declarations like this:

(provide (except-out (all-from-out foo/bar)
 (type-out ExcludedType)))

If type-out is a provide transformer that doesn’t actually provide
anything, it merely expands to a submodule (perhaps using
`syntax-local-lift-module-end-declaration`), then it can’t possibly
understand that it should be doing something different when used in
“excluding” position. Is the unfortunate answer here that I can’t get
around needing to reimplement my own version of `provide` as well?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Multiple namespaces in Racket

[Matthew Flatt]

At Thu, 12 Oct 2017 12:59:17 -0700, Alexis King wrote:
> An alternative approach that
> I have considered (and discussed briefly with some people) is to
> introduce a “value” scope using make-syntax-introducer in
> #%module-begin, then use a separate “type” scope when parsing types to
> keep the two bindings completely separate.

That sounds right to me.

> Specifically, the main question occurs when considering code that spans
> more than a single module. What do require and provide do? For the
> provide case, it doesn’t seem too hard to have a (type-out ) provide
> transformer, but what happens when a user requires the module? Since my
> understanding is that modules essentially only provide a set of
> *symbols* per phase, not identifiers, would I need to mangle names in
> order to allow a module to export a type with the same name as a value?
> Would I need to write my own version of require to handle this without
> wrapping all imports in a require transformer? What about interop with
> other Racket languages?

You could put all the type exports in a submodule. Then, you do need
your own variant of `require`, but that variant can mostly just check
for the presence of a type submodule, much the way that TR or
`plai-typed` do. If the submodule is available, you can import it with
the extra scope for types.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.