Re: Is package.d a good idea?
On Wednesday, July 04, 2018 11:50:07 Steven Schveighoffer via Digitalmars-d wrote: > The obvious pros to ditching package.d and using a directory and module > with the same name are: 1. no ambiguity in the filesystem (assuming you > don't have weird package/file schemes) and 2. It's more understandable > to someone not used to the current state of affairs. > > To me, it's not worth it for just those things. I'm curious why Rust is > changing. I don't know, but I suspect that they actually require that all of the modules of a package be in the same folder and don't allow the weird renaming stuff that we allow and then basically tell people to never use. I question that even then, inferring the modules in the package is ultimately a good idea, but it would make more sense if the module layout is less flexible. - Jonathan M Davis
Re: Is package.d a good idea?
On 7/4/18 11:29 AM, Jonathan M Davis wrote: On Wednesday, July 04, 2018 11:13:07 Steven Schveighoffer via Digitalmars-d wrote: I think if we went the different route we would have to provide a mechanism to declare inside the module "this is a package". Maybe "package module"? I would point out that the pretty much the only reason that we were able to convince Walter to have package.d in the language was because of how straightforward it is and how it changed nothing about import semantics and required very little to add it. Any other approaches would likely need to provide a significant benefit over what we have now in order to be acceptable. And suggestions like this just seem to be shuffling things around without really changing or improving what you can do. I'm fully in agreement with you. I just wanted to point out that it's not as cut-and-dry as people have been saying -- we would need some new mechanism to replicate the current behavior. The obvious pros to ditching package.d and using a directory and module with the same name are: 1. no ambiguity in the filesystem (assuming you don't have weird package/file schemes) and 2. It's more understandable to someone not used to the current state of affairs. To me, it's not worth it for just those things. I'm curious why Rust is changing. -Steve
Re: Is package.d a good idea?
On Wednesday, July 04, 2018 11:13:07 Steven Schveighoffer via Digitalmars-d wrote: > I think if we went the different route we would have to provide a > mechanism to declare inside the module "this is a package". Maybe > "package module"? I would point out that the pretty much the only reason that we were able to convince Walter to have package.d in the language was because of how straightforward it is and how it changed nothing about import semantics and required very little to add it. Any other approaches would likely need to provide a significant benefit over what we have now in order to be acceptable. And suggestions like this just seem to be shuffling things around without really changing or improving what you can do. - Jonathan M Davis
Re: Is package.d a good idea?
On 7/4/18 11:06 AM, aliak wrote: On Wednesday, 4 July 2018 at 14:54:41 UTC, Steven Schveighoffer wrote: On 7/1/18 7:36 AM, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. How would this affect the package attribute? Currently, anything inside the package.d file that is attributed with package is limited to the package itself, not where the package resides. Likewise, anything attributed with package inside other modules in the package are visible from the package.d file. This makes sense, since package.d is conceptually and physically *inside* the package. When foo.d is outside the foo directory, its package attributed items become visible to the parent directory as well, and it won't have access to the submodule package data. It also doesn't make it clear that foo is a package and not a module. As much as the current scheme is confusing, it makes it more sane when it comes to package protection. It's like a space where you can declare things inside the package that do not belong to any actual modules. It's more explicit in the intent of the package designer. I don't know what the Rust situation is, so I can't really comment on that. The D situation doesn't seem worth changing, even if it helps alleviate some confusion. What I would like to see, however, is an error on having both foo.d and foo/... as this is ambiguous and prone to confusion. I guess it can be the same? If instead of: "if there's a file in this folder called package.d then this folder is a package" it would be: "if there's a file outside of this folder with the same name as this folder then that folder is a package" And the attribute package inside the folder applies to the package and the attribute package in file.d applies to the package that file is in, if any? (actually it seems like being able to make file.d part of whatever package it's in is a bonus here) This means that a module becomes a package or not depending on the existence of other items. This could end up being even more confusing than it is now. Note that packages and modules don't necessarily have to be declared in the same file or directory structure even. See Adam's post. I think if we went the different route we would have to provide a mechanism to declare inside the module "this is a package". Maybe "package module"? -Steve
Re: Is package.d a good idea?
On 7/4/18 10:59 AM, Adam D. Ruppe wrote: On Wednesday, 4 July 2018 at 14:54:41 UTC, Steven Schveighoffer wrote: How would this affect the package attribute? Nothing should change, since packages are determined from the D module declaration, not the filename or directory layout. This is even true with package.d itself, but it is a weird exception because for some inexplicable reason, the compiler ALSO requires a certain filename to accept the declaration (whereas it doesn't care for any other module). Hm.. I tested it out: pack1/pack2/package.d pack1/pack2/foo.d pack1/bar.d package symbols in pack1.pack2 are visible to the package.d and foo.d file, but not the bar.d file. pack1/pack2.d pack1/bar.d package symbols in pack1.pack2 are now visible to the bar.d file. So it does make a difference. And the difference would be bizarre if pack2 was also a package. Note: I did find that I needed the pack1 top-level package for this to demonstrate. When pack2 was a top level package, things did not work out very well. If you were to have a dir layout foo.d whatever/bar.d and foo.d was module ok.awesome.works.for.me; and whatever/bar.d was module ok.awesome.works.for.someone; they are both part of the `ok.awesome.works.for` package, regardless of their filenames. Right, the difference though is that package.d is special in that it is treated like a package, and not a module for purposes of visibility. For example, I don't think you could do: module ok.awesome.works.for; ... module ok.awesome.works.for.me; without using the package.d filename. -Steve
Re: Is package.d a good idea?
On Wednesday, 4 July 2018 at 14:54:41 UTC, Steven Schveighoffer wrote: On 7/1/18 7:36 AM, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. How would this affect the package attribute? Currently, anything inside the package.d file that is attributed with package is limited to the package itself, not where the package resides. Likewise, anything attributed with package inside other modules in the package are visible from the package.d file. This makes sense, since package.d is conceptually and physically *inside* the package. When foo.d is outside the foo directory, its package attributed items become visible to the parent directory as well, and it won't have access to the submodule package data. It also doesn't make it clear that foo is a package and not a module. As much as the current scheme is confusing, it makes it more sane when it comes to package protection. It's like a space where you can declare things inside the package that do not belong to any actual modules. It's more explicit in the intent of the package designer. I don't know what the Rust situation is, so I can't really comment on that. The D situation doesn't seem worth changing, even if it helps alleviate some confusion. What I would like to see, however, is an error on having both foo.d and foo/... as this is ambiguous and prone to confusion. -Steve I guess it can be the same? If instead of: "if there's a file in this folder called package.d then this folder is a package" it would be: "if there's a file outside of this folder with the same name as this folder then that folder is a package" And the attribute package inside the folder applies to the package and the attribute package in file.d applies to the package that file is in, if any? (actually it seems like being able to make file.d part of whatever package it's in is a bonus here) Cheers, - Ali
Re: Is package.d a good idea?
On Wednesday, 4 July 2018 at 14:54:41 UTC, Steven Schveighoffer wrote: How would this affect the package attribute? Nothing should change, since packages are determined from the D module declaration, not the filename or directory layout. This is even true with package.d itself, but it is a weird exception because for some inexplicable reason, the compiler ALSO requires a certain filename to accept the declaration (whereas it doesn't care for any other module). If you were to have a dir layout foo.d whatever/bar.d and foo.d was module ok.awesome.works.for.me; and whatever/bar.d was module ok.awesome.works.for.someone; they are both part of the `ok.awesome.works.for` package, regardless of their filenames.
Re: Is package.d a good idea?
On 7/1/18 7:36 AM, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. How would this affect the package attribute? Currently, anything inside the package.d file that is attributed with package is limited to the package itself, not where the package resides. Likewise, anything attributed with package inside other modules in the package are visible from the package.d file. This makes sense, since package.d is conceptually and physically *inside* the package. When foo.d is outside the foo directory, its package attributed items become visible to the parent directory as well, and it won't have access to the submodule package data. It also doesn't make it clear that foo is a package and not a module. As much as the current scheme is confusing, it makes it more sane when it comes to package protection. It's like a space where you can declare things inside the package that do not belong to any actual modules. It's more explicit in the intent of the package designer. I don't know what the Rust situation is, so I can't really comment on that. The D situation doesn't seem worth changing, even if it helps alleviate some confusion. What I would like to see, however, is an error on having both foo.d and foo/... as this is ambiguous and prone to confusion. -Steve
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 11:36:51 UTC, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. when I started D I was really confused about just this one thing as well! We should definitely get rid of package.d and support just having packages along modules without that weird error (foo.d + foo/something.d which will have modules foo and foo.something)
Re: Is package.d a good idea?
On Tuesday, 3 July 2018 at 12:51:59 UTC, bauss wrote: On Tuesday, 3 July 2018 at 12:51:18 UTC, bauss wrote: On Sunday, 1 July 2018 at 11:36:51 UTC, Yuxuan Shui wrote: [...] I use package.d for more than just submodules. I use it to share modules from different packages into a single package too, as well symbols and more! I should probably have added this to my post. A good example of it is this: https://github.com/DiamondMVC/Diamond/blob/master/database/package.d So that would just go in to master/database.d and your database/ folder would be unchanged then right?
Re: Is package.d a good idea?
On Tuesday, 3 July 2018 at 12:51:18 UTC, bauss wrote: On Sunday, 1 July 2018 at 11:36:51 UTC, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. I use package.d for more than just submodules. I use it to share modules from different packages into a single package too, as well symbols and more! I should probably have added this to my post. A good example of it is this: https://github.com/DiamondMVC/Diamond/blob/master/database/package.d
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 11:36:51 UTC, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. I use package.d for more than just submodules. I use it to share modules from different packages into a single package too, as well symbols and more!
Re: Is package.d a good idea?
On 07/01/2018 10:23 AM, Yuxuan Shui wrote: I was suggesting we do what Rust did. i.e. 'import foo', imports foo.d, which can in turn do 'import foo.bar', which will import foo/bar.d. AIUI, D doesn't support having *both* a module (ie, file 'foo.d') and a package (ie, directory 'foo/') with the same name (ie, 'foo')...for some goofy unknown reason. I agree that suggestion is how it should've been from the start.
Re: Is package.d a good idea?
On Sunday, July 01, 2018 14:23:36 Yuxuan Shui via Digitalmars-d wrote: > On Sunday, 1 July 2018 at 11:55:17 UTC, Jonathan M Davis wrote: > > On Sunday, July 01, 2018 11:36:51 Yuxuan Shui via Digitalmars-d > > > > wrote: > >> [...] > > > > The entire reason that package.d was added as a feature was so > > that modules could be split into packages without breaking > > code, and it's still valuable for that. > > > > [...] > > I was suggesting we do what Rust did. i.e. 'import foo', imports > foo.d, which can in turn do 'import foo.bar', which will import > foo/bar.d. So, basically, we'd implicitly have a package.d for every package where each implicit package.d publicly imported every module in its package (including the implicit package.d from any sub-packages)? Well, for starters, that's going in pretty much the opposite direction of where best practice in D has been going. Local and scoped imports have become best practice, which means writing stuff like import std.conv : to; inside the function that's using std.conv.to rather than import std.conv; at the top of the file, let alone import std; So, the suggestion that you could just import any package and get everything in it recursively, doesn't really fit with where we've been going. Regardless of that though, I'm not even sure that it works with D's import model. As things stand, D only ever imports exactly the modules that you tell it to import. It doesn't even have the concept of importing a package. package.d is a file like any other. It's just that the compiler has the little bit of magic to know that if it's told to import a.b that if a/b/package.d exists and has the module declaration for a.b, then it should be treated as being module a.b. The only reason that package.d then ends up importing the entire package is because the programmer explicitly pubicly imports each module from the package. They could do something else, and in some cases, it makes good sense to do stuff like include modules from elsewhere or to exclude modules from within the package. Your suggestion gives no such control, which could be a serious downside. But aside from the control issue, the fact of the matter is that D currently only imports what it's told to import. It doesn't have to know or care what all of the modules in a package are. It just goes and grabs what it's told to grab, whereas if it has to search recursively, then that probably negatively impacts the performance of imports, and it fundamentally changes how importing works, because then instead of just grabbing what it's told to grab, it has to compile the list based on what it finds - which may or may not be the correct list. If any modules are missing for some reason, or if the compiler's import paths are misconfigured, then you could think that you're importing a module when you're not, which could lead to function hijacking when you try to use the function. In most cases, it would just be a compilation error, because the symbol is missing, but if another symbol which isn't missing happens to match, then instead of getting a symbol conflict, it would call the wrong function. It's the sort of thing that's a lot more likely to happen when installs and upgrades go wrong than during your typical development, but it does make the situation more error-prone than it is now. It also wouldn't work well if anyone did something like split a package across libraries (which is a questionable practice but perfectly legal). When you compiled one library, you'd get one set of modules when importing the package, and when you imported another module, you'd get a different set (whereas right now, you just get the modules that you explicitly imported). What you'd get when using both libraries would then be highly dependent on how you compile your code. The idea that you're going to recursively find all of the modules in a package simply doesn't work well with a separate compilation model. It's not impossible to make work, but it's error-prone. And of course, I have no clue how any of this interacts with modules where the file name doesn't match the module declaration (which is unfortunately very purposefully legal). It's something that I never do (and doesn't tend to work well with some build tools), so I don't understand the details very well, but it's the sort of detail that makes the whole situation much more complicated. If someone wants to write a DIP on this, then they can, but the whole idea seems to fly in the face of current best practice in D, and honestly, the whole idea just seems error-prone. It would probably work a decent percentage of the time, but it looks like it has some nasty corner cases, and it would almost certainly hurt compilation speed. As it is, Walter approved package.d precisely because it added almost nothing. It just used the pre-existing import rules with the change that the compiler would know to check for the existance of package.d. The import rules themselves didn't change at all
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 21:40:05 UTC, Yuxuan Shui wrote: On Sunday, 1 July 2018 at 18:03:41 UTC, Adam D. Ruppe wrote: On Sunday, 1 July 2018 at 14:23:36 UTC, Yuxuan Shui wrote: I was suggesting we do what Rust did. i.e. 'import foo', imports foo.d, which can in turn do 'import foo.bar', which will import foo/bar.d. Yeah, that's the way it should have been done in the first place. Nowhere else in D does it require a specific filename except package.d - it is a pointless inconsistency anyway. Now we just need to wait for someone to write the DIP There nothing prevent you from writing one, is there? -Alexander Heistermann
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 18:03:41 UTC, Adam D. Ruppe wrote: On Sunday, 1 July 2018 at 14:23:36 UTC, Yuxuan Shui wrote: I was suggesting we do what Rust did. i.e. 'import foo', imports foo.d, which can in turn do 'import foo.bar', which will import foo/bar.d. Yeah, that's the way it should have been done in the first place. Nowhere else in D does it require a specific filename except package.d - it is a pointless inconsistency anyway. Now we just need to wait for someone to write the DIP
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 14:23:36 UTC, Yuxuan Shui wrote: On Sunday, 1 July 2018 at 11:55:17 UTC, Jonathan M Davis wrote: On Sunday, July 01, 2018 11:36:51 Yuxuan Shui via Digitalmars-d wrote: [...] The entire reason that package.d was added as a feature was so that modules could be split into packages without breaking code, and it's still valuable for that. [...] I was suggesting we do what Rust did. i.e. 'import foo', imports foo.d, which can in turn do 'import foo.bar', which will import foo/bar.d. You mean that if no package.d is present, the import that *would* match to the missing package is then equivalent to an automatic public import for each module prefixed ? - case 1 : `import foo;` use package.d foo/package.d /bar.d /baz.d - case 2 : `import foo;` make all stuff (bar/baz) public imports automatically if no package.d foo/bar.d /baz.d If this is what you mean then this could indeed be added, as sugar, in addition to the existing system of package. By "could" i mean from a technical P.o.V, because of course this kind of stuff are decided elsewhere.
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 14:23:36 UTC, Yuxuan Shui wrote: I was suggesting we do what Rust did. i.e. 'import foo', imports foo.d, which can in turn do 'import foo.bar', which will import foo/bar.d. Yeah, that's the way it should have been done in the first place. Nowhere else in D does it require a specific filename except package.d - it is a pointless inconsistency anyway.
Re: Is package.d a good idea?
On Sunday, 1 July 2018 at 11:55:17 UTC, Jonathan M Davis wrote: On Sunday, July 01, 2018 11:36:51 Yuxuan Shui via Digitalmars-d wrote: [...] The entire reason that package.d was added as a feature was so that modules could be split into packages without breaking code, and it's still valuable for that. [...] I was suggesting we do what Rust did. i.e. 'import foo', imports foo.d, which can in turn do 'import foo.bar', which will import foo/bar.d.
Re: Is package.d a good idea?
On Sunday, July 01, 2018 11:36:51 Yuxuan Shui via Digitalmars-d wrote: > In Rust, they have something call mod.rs, which is very similar > to package.d. When you use a module 'foo' in Rust, it can either > be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to > be 'foo/mod.rs'. > > Now in the Rust 2018 edition, they are getting rid of mod.rs. So > when you import 'foo', rustc will always look for 'foo.rs', and > if 'foo' has submodules, it can still reside in > 'foo/submodule.rs'. > > This makes me think if package.d is a good idea, and if we should > try to get rid of it as well. The entire reason that package.d was added as a feature was so that modules could be split into packages without breaking code, and it's still valuable for that. Now, other folks have liked the idea of using it to import a lot of code with a single module - potentially even importing entire librares at once - but with the increasing focus on local and scoped imports, best practice really leans towards importing each symbol individually, which is the complete opposite of the spectrum from importing entire packages at once and means that using package.d in order to import a bunch of other modules arguably goes against current best practice. Regardless, I expect that some folks will continue to want to do it - especially for stuff like scripts where you really don't want to bother importing each symbol individually. - Jonathan M Davis
Re: Is package.d a good idea?
On 01/07/2018 11:36 PM, Yuxuan Shui wrote: In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well. We added it a few years ago after 10+ years of people having a module called 'all' which did the same exact job just without the compiler help. So no, we should not get rid of it, because its works pretty well over here :)
Is package.d a good idea?
In Rust, they have something call mod.rs, which is very similar to package.d. When you use a module 'foo' in Rust, it can either be 'foo.rs' or 'foo/mod.rs'. If 'foo' has sub-modules, it has to be 'foo/mod.rs'. Now in the Rust 2018 edition, they are getting rid of mod.rs. So when you import 'foo', rustc will always look for 'foo.rs', and if 'foo' has submodules, it can still reside in 'foo/submodule.rs'. This makes me think if package.d is a good idea, and if we should try to get rid of it as well.
