Re: [racket-dev] should package X imply package X-test?
It seems like a common thing to have a make test that comes with various software packages out the in the wild. Judging from the traffic on the gambit list, their version seems to get used, too. I'm not sure if you'd classify gambit's 'make test' as something that would be in the test package, tho. A conscientious user might run the test suite (if it were written by a conscientious owner who made it easily push-button) before submitting a bug report just to see if there were something interesting to report along with the bug report. I guess I still don't actually have a strong opinion, but if we don't go this direction, it would be nice to have a positive rationale for why we suggest leaving this particular piece out of X. I can clearly see a strong reason to leave parts out of X-lib for minimizing dependencies, but that ship has sailed if we are including X-doc in X. Robby On Tue, Oct 15, 2013 at 11:00 PM, Neil Toronto neil.toro...@gmail.comwrote: On 10/15/2013 06:40 PM, Robby Findler wrote: Actually, on second thought, I think I'm going to buck the trend here and answer yes to the question in the subject line. Two scenarios come to mind: someone hears about some cool new library and then does something like 'raco pkg install math'. Next thing, something goes wrong and they ask about it on a mailing list and get the response Run this code: (require ...) which ends up running something from the test suite of that library [1]. In that case, we'd probably like them to have everything related to the library. Second: someone decides to build a library that depends on the, say, the math library and they have to choose what to put in their info.rkt file. They don't want to put math no matter if it includes the tests or not, since they don't to force their clients to install the docs. So they'll put math-lib. In other words, I don't see a lot of value in a pkg name taken up for the lib+docs subset of some (conceptual) pile of code, but I do see value in the everything pile and the non-docs non-tests pile. So I'd say that the name X (from the subject) should mean everything. ... at least for a library. For an app, I'm less clear. Robby [1] this actually happened recently as you probably recall (altho 'test-floating-point' may not be in the math-test pkg) http://lists.racket-lang.org/**users/archive/2013-October/**059881.htmlhttp://lists.racket-lang.org/users/archive/2013-October/059881.html Right. It's in math-lib (the `math/utils' module) because it's useful outside of automated testing. Users can run it to find out how reliable and repeatable their floating-point results are on a given system. The rest of the math library tests are more or less system-agnostic. (There's a custodian shutdown test for unloading `math/bigfloat' that I think may be good to test on multiple platforms, but that's more of an automation thing.) I can't imagine asking a user to run them, and if I did, I wouldn't mind asking them to install an extra package first. Are there any other circumstances, besides a package owner asking, that a user would want to run all the tests for a package? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/**dev http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
On Wed, Oct 16, 2013 at 7:55 AM, Robby Findler ro...@eecs.northwestern.edu wrote: it would be nice to have a positive rationale for why we suggest leaving this particular piece out of X. Here's my positive rationale: First, why include X-doc? I expect that if I tell someone they should try out Typed Racket, or Slideshow, or Frog, or some other bit of Racket software, I'll suggest that they do 'raco pkg install X', for whichever X we're talking about. This means that 'X' should contain all the stuff they'd want to use. In particular, I assume that almost everyone who's intentionally installing 'X' will want X's documentation. However, I don't think that everyone will want the tests -- if they do want the tests, it will be in a situation like you described, where asking someone to 'raco pkg install X-test' is easy. But why not just install X-test anyway, in case they need it? I think this goes back to why we're not shipping absolutely everything at once. There's a cost to a bigger download, whether that's someone deciding not to download a big file because the file size scares them off (a real phenomenon), or a network error that wouldn't have happened with a smaller download, or even actual monetary cost for someone on a metered network. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
I have no strong opinion, either, but... In most software projects, `make test` is something that you can do with the sources, but not with an installation. I think we agree that the tests for package X should be in the same repository as X-lib, X-doc, etc., but I don't think there's much precedent for including tests in the main distribution. From the perspective of avoiding dependencies, I think there is value in leaving things out of X (i.e., X as a typical user of the library would want to install it), even when X-lib and other packages are available. For example, the main-distribution package includes X for various Xs --- not X-lib plus X-doc plus It makes more sense to me to put the decision of what is normally included as X with X itself, instead of with each package that defines a distribution. At Wed, 16 Oct 2013 06:55:21 -0500, Robby Findler wrote: It seems like a common thing to have a make test that comes with various software packages out the in the wild. Judging from the traffic on the gambit list, their version seems to get used, too. I'm not sure if you'd classify gambit's 'make test' as something that would be in the test package, tho. A conscientious user might run the test suite (if it were written by a conscientious owner who made it easily push-button) before submitting a bug report just to see if there were something interesting to report along with the bug report. I guess I still don't actually have a strong opinion, but if we don't go this direction, it would be nice to have a positive rationale for why we suggest leaving this particular piece out of X. I can clearly see a strong reason to leave parts out of X-lib for minimizing dependencies, but that ship has sailed if we are including X-doc in X. Robby On Tue, Oct 15, 2013 at 11:00 PM, Neil Toronto neil.toro...@gmail.comwrote: On 10/15/2013 06:40 PM, Robby Findler wrote: Actually, on second thought, I think I'm going to buck the trend here and answer yes to the question in the subject line. Two scenarios come to mind: someone hears about some cool new library and then does something like 'raco pkg install math'. Next thing, something goes wrong and they ask about it on a mailing list and get the response Run this code: (require ...) which ends up running something from the test suite of that library [1]. In that case, we'd probably like them to have everything related to the library. Second: someone decides to build a library that depends on the, say, the math library and they have to choose what to put in their info.rkt file. They don't want to put math no matter if it includes the tests or not, since they don't to force their clients to install the docs. So they'll put math-lib. In other words, I don't see a lot of value in a pkg name taken up for the lib+docs subset of some (conceptual) pile of code, but I do see value in the everything pile and the non-docs non-tests pile. So I'd say that the name X (from the subject) should mean everything. ... at least for a library. For an app, I'm less clear. Robby [1] this actually happened recently as you probably recall (altho 'test-floating-point' may not be in the math-test pkg) http://lists.racket-lang.org/**users/archive/2013-October/**059881.htmlhttp:// lists.racket-lang.org/users/archive/2013-October/059881.html Right. It's in math-lib (the `math/utils' module) because it's useful outside of automated testing. Users can run it to find out how reliable and repeatable their floating-point results are on a given system. The rest of the math library tests are more or less system-agnostic. (There's a custodian shutdown test for unloading `math/bigfloat' that I think may be good to test on multiple platforms, but that's more of an automation thing.) I can't imagine asking a user to run them, and if I did, I wouldn't mind asking them to install an extra package first. Are there any other circumstances, besides a package owner asking, that a user would want to run all the tests for a package? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/**dev http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
I'm speaking of third-party packages only; I don't know about the packages that make up core Racket. (But perhaps whatever is done for any special needs of core Racket can avoid complicating things for third-party packages.) For third-party packages, for the audience of technical users of Racket, I think that getting documentation, source code, and tests by default is a good thing. Not only do technical users often want those things, but IMHO we should be encouraging people to use those things. This could be the simple case. For use or installation of third-party packages on very small devices, such as on an OpenWRT router, or for packaging shrinkwrap apps, maybe there should be a directive in info.rkt for what files can be excluded in ``tiny'' installations, in addition to not including docs. This seems like an unusual case, but seems like there is a simple solution, while also keeping the usual case simple. For third-party package authors who want to avoid some big dependency in the tests (e.g., huge data set, or dependencies on some other packages), maybe the solution is the same as ``tiny'' above, maybe they need to separate out some of their tests into a separate package, or maybe the dependency is not all that necessary. This too seems like an unusual case to me, although I do have one package myself that kinda runs into it (non-small test files in http://www.neilvandyke.org/racket-mediafile/;). Again, it seems to me like the solution for this unusual case is simple and keeps the usual case simple. Neil V. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
I agree. I think the normal case is just that there's a package X, and it's only when clients of X request finer-grained distribution that it's worth doing anything --- like the request that made Neil T. split math. The tiny mode you describe sounds like binary mode, and there are indeed info.rkt fields to control what is included and excluded when a package is pruned to binary form. At Wed, 16 Oct 2013 11:30:27 -0400, Neil Van Dyke wrote: I'm speaking of third-party packages only; I don't know about the packages that make up core Racket. (But perhaps whatever is done for any special needs of core Racket can avoid complicating things for third-party packages.) For third-party packages, for the audience of technical users of Racket, I think that getting documentation, source code, and tests by default is a good thing. Not only do technical users often want those things, but IMHO we should be encouraging people to use those things. This could be the simple case. For use or installation of third-party packages on very small devices, such as on an OpenWRT router, or for packaging shrinkwrap apps, maybe there should be a directive in info.rkt for what files can be excluded in ``tiny'' installations, in addition to not including docs. This seems like an unusual case, but seems like there is a simple solution, while also keeping the usual case simple. For third-party package authors who want to avoid some big dependency in the tests (e.g., huge data set, or dependencies on some other packages), maybe the solution is the same as ``tiny'' above, maybe they need to separate out some of their tests into a separate package, or maybe the dependency is not all that necessary. This too seems like an unusual case to me, although I do have one package myself that kinda runs into it (non-small test files in http://www.neilvandyke.org/racket-mediafile/;). Again, it seems to me like the solution for this unusual case is simple and keeps the usual case simple. Neil V. _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
Another scenario is tests that are expensive and/or require a third party account, so that a user is very unlikely to run them. Libs that are wrappers for web services are examples I have, like AWS or echonest or Google api discovery. Can't test interacting with the service without an account configured, and the tests may have nontrivial I/O or even $ costs. Having said that I'm not sure in my cases the cost to download/install the tests is such a big deal, as opposed to the cost or manual setup to run them. On Oct 16, 2013 10:09 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Wed, Oct 16, 2013 at 7:55 AM, Robby Findler ro...@eecs.northwestern.edu wrote: it would be nice to have a positive rationale for why we suggest leaving this particular piece out of X. Here's my positive rationale: First, why include X-doc? I expect that if I tell someone they should try out Typed Racket, or Slideshow, or Frog, or some other bit of Racket software, I'll suggest that they do 'raco pkg install X', for whichever X we're talking about. This means that 'X' should contain all the stuff they'd want to use. In particular, I assume that almost everyone who's intentionally installing 'X' will want X's documentation. However, I don't think that everyone will want the tests -- if they do want the tests, it will be in a situation like you described, where asking someone to 'raco pkg install X-test' is easy. But why not just install X-test anyway, in case they need it? I think this goes back to why we're not shipping absolutely everything at once. There's a cost to a bigger download, whether that's someone deciding not to download a big file because the file size scares them off (a real phenomenon), or a network error that wouldn't have happened with a smaller download, or even actual monetary cost for someone on a metered network. Sam _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] should package X imply package X-test?
We have several packages X that imply packages X-lib and X-doc --- and that seems good to me. Some Xs also imply X-test, while other Xs do not imply X-test (even though X-test exists). We should change one of those sets to be consistent with the other. It's important that X-lib and X-test end up in the same source repository, but I don't think that X clients necessarily need tests for X. So, I suggest that X should not imply X-test. Other opinions? _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
I agree with this. In particular, I like to be able to use libraries in testing code that the rest of the library doesn't depend on, and I'd like to not make users install those extra libraries. Sam On Oct 15, 2013 4:47 PM, Matthew Flatt mfl...@cs.utah.edu wrote: We have several packages X that imply packages X-lib and X-doc --- and that seems good to me. Some Xs also imply X-test, while other Xs do not imply X-test (even though X-test exists). We should change one of those sets to be consistent with the other. It's important that X-lib and X-test end up in the same source repository, but I don't think that X clients necessarily need tests for X. So, I suggest that X should not imply X-test. Other opinions? _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
Well, that's already available via X-lib (plus possibly X-doc). I don't have a strong opinion on this, but I'm not sure that's the right rationale. Robby On Tue, Oct 15, 2013 at 4:08 PM, Sam Tobin-Hochstadt sa...@cs.indiana.eduwrote: I agree with this. In particular, I like to be able to use libraries in testing code that the rest of the library doesn't depend on, and I'd like to not make users install those extra libraries. Sam On Oct 15, 2013 4:47 PM, Matthew Flatt mfl...@cs.utah.edu wrote: We have several packages X that imply packages X-lib and X-doc --- and that seems good to me. Some Xs also imply X-test, while other Xs do not imply X-test (even though X-test exists). We should change one of those sets to be consistent with the other. It's important that X-lib and X-test end up in the same source repository, but I don't think that X clients necessarily need tests for X. So, I suggest that X should not imply X-test. Other opinions? _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
Yes, I think that was the point of the original message: to figure out what consistent thing we think it should be. Robby On Tue, Oct 15, 2013 at 9:34 PM, Eli Barzilay e...@barzilay.org wrote: It would be nice if it implied some consistent default, depending on what I generally want. Ie, I can be someone who just want the minimum -libs, or someone who always wants everything including the -tests. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
At Tue, 15 Oct 2013 22:40:54 -0400, Eli Barzilay wrote: Just now, Robby Findler wrote: Yes, I think that was the point of the original message: to figure out what consistent thing we think it should be. My point was in the depending on what *I* generally want. I'm saying that it much better to leave it consistent per some user choice, since there are different types of users; in contrast to finding some global default that would work for everyone. (Since I believe there isn't such a default.) I agree with that goal, but I don't see a way to get there in the near future. On Tue, Oct 15, 2013 at 9:34 PM, Eli Barzilay e...@barzilay.org wrote: It would be nice if it implied some consistent default, depending on what I generally want. Ie, I can be someone who just want the minimum -libs, or someone who always wants everything including the -tests. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] should package X imply package X-test?
On 10/15/2013 06:40 PM, Robby Findler wrote: Actually, on second thought, I think I'm going to buck the trend here and answer yes to the question in the subject line. Two scenarios come to mind: someone hears about some cool new library and then does something like 'raco pkg install math'. Next thing, something goes wrong and they ask about it on a mailing list and get the response Run this code: (require ...) which ends up running something from the test suite of that library [1]. In that case, we'd probably like them to have everything related to the library. Second: someone decides to build a library that depends on the, say, the math library and they have to choose what to put in their info.rkt file. They don't want to put math no matter if it includes the tests or not, since they don't to force their clients to install the docs. So they'll put math-lib. In other words, I don't see a lot of value in a pkg name taken up for the lib+docs subset of some (conceptual) pile of code, but I do see value in the everything pile and the non-docs non-tests pile. So I'd say that the name X (from the subject) should mean everything. ... at least for a library. For an app, I'm less clear. Robby [1] this actually happened recently as you probably recall (altho 'test-floating-point' may not be in the math-test pkg) http://lists.racket-lang.org/users/archive/2013-October/059881.html Right. It's in math-lib (the `math/utils' module) because it's useful outside of automated testing. Users can run it to find out how reliable and repeatable their floating-point results are on a given system. The rest of the math library tests are more or less system-agnostic. (There's a custodian shutdown test for unloading `math/bigfloat' that I think may be good to test on multiple platforms, but that's more of an automation thing.) I can't imagine asking a user to run them, and if I did, I wouldn't mind asking them to install an extra package first. Are there any other circumstances, besides a package owner asking, that a user would want to run all the tests for a package? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/dev
