On Wednesday, April 29, 2015 at 11:36:15 PM UTC-4, MA Laforge wrote: > > *Scott and Michael:* > I am pretty certain I understand what you are saying, but I find your > examples/descriptions a bit confusing. I think (hope) I know why Stafan is > confused. > > *Stefan:* > I think Scott has a valid point but I disagree that "exported functions > from a module must reference types defined in that module". It is my > strong belief that Scott is merely focusing on the symptom instead of the > cause. >
I've never advocated that! What I've said, a number of times now, is that *if* the compiler determines that a method in a module is using a type defined in that module, which should mean that it is unambiguous, and it is exported, that if somebody does "using xxx", it should be merged. I actually now think that it really needs to be explicit... it is not very clear in Julia what the intent of the programmer was. You have me confused with Michael here... I disagree with that part of what he was saying, and agree totally with Stefan that it would be way to limiting. I also think that the approach of just postponing the check for ambiguity to run-time would be a very bad thing... I think it's hard enough now to know if your module is correct, it seems a lot of stuff isn't caught until run-time. I also think that your description of the problem is very good, but I don't think your solution does enough to make things better... I would propose adding some new syntax to indicate if a function name in a module is meant to extend a particular function, instead of that happening implicitly because it was previously imported, as well as syntax to indicate that a function name is meant to be a new "concept", that others can extend, and can unambiguously be used in the global namespace (i.e. it uses one (or more) of its own type(s) as part of its signature... module baz function foo(args) export extends Base # i.e. extends Base.foo, is exported function silly(args) export extends Database # i.e. extends Database.silly, is exported function bar(abc::MyType) export generic # Creates a new concept "bar", which is exported, and can be extended by other modules explicity with the extends syntax function ugh(args) extends Base # extends Base.ugh in the module, but is not visible outside the module (have to use baz.ugh to call) function myfunc(args) # makes a local definition, will hide other definitions from Base or used in the parent module, but can be use as baz.myfunc function myownfunc(args) private # makes a local definition, as above, but is not visible at all in parent module I'm not sure if this is possible, but, what if you wanted to extend a particular function, but *only* in the context of your module? Currently, the extension is made public, even if you didn't export the function, which seems a bit wrong to me... To me, it seems that whether a function is meant to be a new generic, or extend something else, is orthogonal to whether you want it to be able to be used directly in code that does "using". It would also be good to be able to make methods (even ones that are generic within the module) (or types, for that matter) private from the rest of the world... so that outside the module, they cannot be used... I see a big problem with Julia, in that it seems that one cannot prevent users from directly accessing your implementation, as opposed to be limited to just the abstract interface you made. This was a wonderful thing in CLU... I suppose they no longer teach CLU at MIT? :-( [it would also be wonderful to be able to specify which exceptions a method can throw, and have that strict... if a method specifies its exceptions, then any unhandled exceptions in the method get converted to a special "Unhanded Exception" exception...] Scott > Fortunately, I am certain that this is not the crux of the problem... And > I agree completely with Stefan: Limiting exports to this particular case is > extremely restrictive (and unnecessary). I also agree that, this > restriction *would* keep developers from developing very useful monkey > patches (among other things). So let's look at the problem differently... > > *Problem 1: A module "owns" its verbs.* > See discussion above discussion ( > https://groups.google.com/d/msg/julia-users/sk8Gxq7ws3w/ASFlqZmVwYsJ) if > you are not familiar with the idea. > > *Problem 2: Julia has 2 symbol types (for objects/methods/...)* > Well, this is not really a problem... It is terrific! Julia's > multi-dispatch engine allows us to overload methods in a way I have never > seen before! > > In fact, the multi-dispatch system allows programmers to DO AWAY WITH > namespaces altogether for the new type of symbol (at least to a first > order). > > *So what are the symbol types?* > 1) Conventional Type: Used in most other imperative languages > 2) Multi-Dispatch Type: Symbols of methods whose signature can be uniquely > identified. > > By this definition, if a symbol is not associated with a unique signature, > it is simply a conventional symbol. > > And, as defined, conventional symbols run a high potential for "signature > collisions"... because the signature is insufficient to uniquely identify > whether we are referring to Foo.CommonSymbol(???) or Bar.CommonSymbol(???). > > *So why do we need namespaces?* > Namespaces were created to let programmers use succinct symbol names > without the problem of running into never-ending name collisions. Instead > of dealing with symbol bloat > (Foo.FoosSpecialSymbolThatCannotCollideWithAnybody) - we use scopes to give > symbols a nice hierarchy (namespaces are basically named scopes). When > writing code in the native scope, all symbols are nice and short... and you > can even *import* the symbol names to your own scope to interact with the > module. Now the user gets to use short names - not just the module > developer! > > *So what about the multi-dispatch-ed symbols (type 2)?* > Technically, they could *all* be located at the global scope. You don't > really need to say Foo.run... because the call signature is unique (by > definition) - so there is no ambiguity. > > *Ok, then where is the problem for module developers?* > Simply put: Julia is trying to cram those beautiful multi-dispatchable > methods into a construct (namespaces) that *is not technically needed* for > type 2 symbols. As a consequence, the current Julia implementation > actually makes it *difficult* for module developers to use multi-dispatch. > Of course, everything is just peachy when you work from within Base :). > > Unfortunately, I believe this has caused a little more collateral damage: > Since it is easier to work within Base, it has become this sort of "god > module". I believe Jeff has mentioned developers are getting reluctant to > make it grow any further. > > > > > *The Distributions module: Can it remain elegant?* > The short answer is yes :)! > > Stefan has a valid concern: you either have to keep using full symbol > paths (Distributions.Normal), or > "[...] > you'd have to explicitly import every Distributions type and generic stats > function that you want to use. Instead of being able to write `using > Distributions` and suddenly having all of the stats stuff you might want > available easily, you'd have to keep qualifying everything or explicitly > importing it. Both suck for interactive usage - and frankly even for > non-interactive usage, qualifying or explicitly importing nearly every name > you use is a massive and unnecessary hassle. > " > > I agree. We don't want this... what a pain! Let's not go there. > > *So where do we start?* > In this case, "Normal" is a relatively common name, and I cannot say its > argument list is sufficiently unique to qualify it as a type 2 method. The > signature only involves two "::Number" arguments. > > Good rule of thumb: If an argument list is made up solely from base types > (number/char/string/...) is likely to collide with a function in another > module (say, a Geometry module). > > That was easy!: That means "Normal" is a type 1 symbol. As such, Normal > *should* be "export"-ed by module Distributions... Nothing changes in the > implementation! > > *Now how do we generate numbers with a normal distribution?* > The Distributions package includes a rand() function. Interestingly > enough, the signature for rand makes it a type 2 method - and so it has a > very low probability of signature collisions. > > Here is a simplified definition of the rand function (Not using > UnivariateDistribution, ...): > import Base: [...] rand > rand(x::Normal, n::Int) = ... > Basically: Distributions is extending Base.rand > > With this infrastructure, you can generate random distributions with very > nice code: > using Distributions > X = Normal(0.0, 1.0) > values = rand(X, 3) > > *So what is the problem here?* > Again, it is quite subtle: In Julia, the first module to define a method > essentially becomes the "owner" (in this case "Base"). This is awkward. > > *Then what could be done differently?* > Well, one possible solution would be simply declare type 2 methods as > "global": > global rand(x::Normal, n::Int) = ... > (tentative syntax) > > Due to its unique signature, this type 2 method is unlikely to collide > with the definition from another module anyways... so why not make it > global? > > NOTE: I don't think this is ideal, but it is much better than the current > solution where a module "owns" a given symbol (or verb, if you will). > > *Ok, but does this actually help?* > Surprisingly yes! By not using the same hammer for type 1 & type 2 > symbols, *module developers* get a means to push the safe type 2 symbols to > the global namespace... and *module users* get to decide when/where to > import type 1 symbols from a given library. > > Consequence 1: > It will be very unlikely that two modules will trigger signature > collisions. For example: draw(PlotPackage1.Canvas, ...) does not collide > with draw(PlotPackage2.Canvas, ...). > > Consequence 2: > Even in any other programming language out there: you expect that type 1 > symbol collisions be fatal (because they *are* ambiguous). As usual, you > can only import symbols ("using") from *a single* conflicting module to the > current namespace at a time. The other import *must* be a true "import". > > ...But that's ok... because you still get to use Julia's awesome > multi-dispatch engine on all the type 2 symbols!!! > > >>>And that's the *biggest* problem with Julia's current behavior. You > don't even get to use multi-dispatch on type 2 symbols when you do an > "import". You only get to reap the benefits when "using" succeeds - or if > you import individual type 2 methods.<<< >
