Re: [racket-dev] [racket] tests/eli-tester feedback (Was: Racket unit testing)
Ryan, thank you very much for refocusing the discussion about testing on the essence and away from the superficial syntactic issues. Syntax, as we all know, is just a macro [which some people may get backwards and conclude that testing is just a macro. Nothing could be further from the truth, however]. Find someone to implement this idea. -- Matthias On Feb 17, 2011, at 8:21 PM, Ryan Culpepper wrote: On 02/15/2011 07:28 AM, Eli Barzilay wrote: And finaly, there's the litmus test for existing code: * Ryan: is something like this enough to implement the GUI layer? Not well, I think. The Test-Result type in Noel's racktest code is too simple and inflexible. It represents the minimal essence of testing, but it would be awkward to extend to richer testing sytems. Here's my counter-proposal for representing the results of tests: TestResult = (test-result header execution status) where header is a TestHeader, execution is a TestExecution, and status is a TestStatus A TestResult combines a description of the test, a record of its behavior during execution, and its final status. (This fixes the awkward distribution of information in racktest's Test-Result, with every status variant including a name field---except for exn:fail, which is anonymous or perhaps stashes it in the continuation marks.) TestHeader = (test-header name suite info) where name is (U string #f), suite is (Listof string) and info is a dictionary. A TestHeader identifies the test that has been run via its name and its enclosing test suite. (Suites can be nested.) It also accommodates additional information that may be provided by some frameworks but not others. Examples include the location of the test expression and the mode of the test (expected to fail, test not yet implemented, etc). TestExecution = dictionary TestExecution contains information about the execution of the test that is unrelated to its success or failure. For example, the rackunit gui reports the execution time and any custodian-managed resources that were left around when the test finished. If output is captured during execution, it would also belong here. TestStatus is one of - (test-success) - (test-failure info) where info is a dictionary - (test-error raised-value) TestStatus represents the status of the test. For the error case, note that any value can be raised, not just exn structures. Common info-dict keys include 'actual and 'expected, and perhaps 'comparison, but those keys don't apply to all failures. If the testing framework views failure as something that occurs during execution of code (as in rackunit, as opposed to check-expect), then the info-dict would also contain the continuation marks. Perhaps there should be another variant for skipped/pending tests, or perhaps that should be represented by a successful TestStatus and an annotation on the TestHeader. And that's not quite the end of it. The rackunit gui creates an entry for a test case as soon as it starts running, so the user can see what test case is hanging and interrupt it if they choose. That requires additional communication between test execution and test display. Ryan _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] tests/eli-tester feedback (Was: Racket unit testing)
Three minutes ago, Matthias Felleisen wrote: Ryan, thank you very much for refocusing the discussion about testing on the essence and away from the superficial syntactic issues. These are two completely unrelated issues, very concretely: one is the representation layer that would allow different tools to cooperate, the other is what the user of a particular test suite uses. Neither of them is superficial. The eli-tester feedback subject is unfitting for this discussion. Syntax, as we all know, is just a macro [which some people may get backwards and conclude that testing is just a macro. Nothing could be further from the truth, however]. *sigh* The syntax is important, and this was never said seriously. Now that the two subjects are clear (hopefully), the original point was more like this: * You need some representation for data related to testing. * You need some syntax (more generically, some api) for writing tests. The difficulty of the first is in designing this representation, the implementation is trivial (it'll end up as a bunch of structs and some parameter/s). The difficulty of the second is in both design (for example, making tests catch too many errors), and syntax (making them convenient). In both cases, if you approach the design in a naive way, then testing becomes that trivial little bit of code, and some api that is easy to do. But since the design is not as straightforward, this is wrong, which is why it was never serious. (Otherwise I'd never keep my thing into a hidden unpublic tool.) So can we finally get over this just a syntax joke now? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] [racket] tests/eli-tester feedback (Was: Racket unit testing)
50 minutes ago, Ryan Culpepper wrote: On 02/15/2011 07:28 AM, Eli Barzilay wrote: And finaly, there's the litmus test for existing code: * Ryan: is something like this enough to implement the GUI layer? Not well, I think. The Test-Result type in Noel's racktest code is too simple and inflexible. It represents the minimal essence of testing, but it would be awkward to extend to richer testing sytems. Here's my counter-proposal for representing the results of tests: [...] I can't make sense of it, besides a vague wy to heavy feeling for something that should be core-ishly minimalistic. In an attempt to follow it, I did this: TestResult = header execution status but your TestHeader is used only there, so it could be folded in: TestResult = name (U String #f) suite (Listof String) info Dictionary execution status TestExecution is also used only once so it can also be folded in -- but since it's just a generic dictionary, it can be dropped. TestResult = name (U String #f) suite (Listof String) info Dictionary status Now, status is one of three options, the failure one has a dictionary so it can be removed (folded into the above). So overall, it looks like a simple struct with a name, a suite (kind of, defined indirectly by a hierarchy of string lists), a generic dictionary for stuff, and a status. This is all modulo some questions/issues that are unclear to me: * It looks like it tries to break away too many pieces into a formal description. For example, it looks like an overkill to have fields with the actual value and expected value and worse -- the comparison. For example, What happens when the comparison is parameterized (eg, close within dx to some number)? * What happens when there's no specific expected value to compare? For example, run some two pieces of code 10 times each and check that the average runtime of the first is below the runtime of the second. This could be phrased in terms of an expected value, but in a superficial way, and will prevent useful information from being expressed (since the information would have to be reduced to two numbers). * What if you don't want to hold on to the value? (For example, free some related resource.) * This solidifies the list-of-strings as a representation of the test hierarchy. But perhaps there is no way to avoid this -- if it's made into a proper hierarchy of objects it will probably complicate things in a way that requires the listener to get update events that tells it how the structure changed. * I'm not sure about the error result. It seems to me that this is a meta issue that you're dealing with when you develop the test suite, and as such it should be something that you'd deal with in the usual ways = throw an exception. It's the tools that should be in charge of catching such an exception and deal with it -- which means that - in my tester's case, it'll defer to racket as usual, meaning that you'd just get an error. - in rackunit's case you'd probably get some report listing the erroneous tests, instead of propagating the error. - and in your gui case you'd catch exceptions and show them as error results. Also: And that's not quite the end of it. The rackunit gui creates an entry for a test case as soon as it starts running, so the user can see what test case is hanging and interrupt it if they choose. That requires additional communication between test execution and test display. Yes, that would e part of the protocol for the listener -- and it makes sense to allow tests to invoke it to let it know that a test has started. I'll try to synthesize all of the above, with Noel's suggestion, into something more concrete later today. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Copyright issues
To anyone who commits code to the plt repository: please be careful when you commit code from elsewhere into the tree, to avoid future problems wrt copyrights. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Latex files
There are problematic latex files in collects/scribble/jfp/jfp.cls and sigplan/sigplanconf.cls. The latter has a copyright line but no license specified, and the first is much worse, since it explicitly states: %% This software may only be used in the preparation of journal %% articles or books or parts of books to be published by Cambridge %% University Press. Any other use constitutes an infringement of %% copyright. Does anyone know of some compatible but open files? (If not, then it sounds like it'll be a major PITA.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Latex files
Can we pull the Wine trick of having a script that runs during compilation that downloads the file from a well-known location? Jay 2011/2/18 Eli Barzilay e...@barzilay.org: There are problematic latex files in collects/scribble/jfp/jfp.cls and sigplan/sigplanconf.cls. The latter has a copyright line but no license specified, and the first is much worse, since it explicitly states: %% This software may only be used in the preparation of journal %% articles or books or parts of books to be published by Cambridge %% University Press. Any other use constitutes an infringement of %% copyright. Does anyone know of some compatible but open files? (If not, then it sounds like it'll be a major PITA.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Latex files
About a minute ago, Jay McCarthy wrote: Can we pull the Wine trick of having a script that runs during compilation that downloads the file from a well-known location? That sounds like a headache... and won't help with distribution (which will have the compiled file). Probably easier to do it at runtime, getting the file into some /tmp, then using it from there -- ? But then the problem is that running scribble on the file should probably ask you if you want to get it from there... (IOW, still sounds like a pain.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] [racket] tests/eli-tester feedback (Was: Racket unit testing)
On 02/18/2011 07:30 AM, Eli Barzilay wrote: 50 minutes ago, Ryan Culpepper wrote: On 02/15/2011 07:28 AM, Eli Barzilay wrote: And finaly, there's the litmus test for existing code: * Ryan: is something like this enough to implement the GUI layer? Not well, I think. The Test-Result type in Noel's racktest code is too simple and inflexible. It represents the minimal essence of testing, but it would be awkward to extend to richer testing sytems. Here's my counter-proposal for representing the results of tests: [...] I can't make sense of it, besides a vague wy to heavy feeling for something that should be core-ishly minimalistic. Simplicity is no good if it gets in the way of representing information that needs to be represented. In an attempt to follow it, I did this: TestResult = header execution status but your TestHeader is used only there, Not necessarily. A testing framework that distinguishes test construction from test creation might create the header when the test is constructed. SchemeUnit used to work that way, and RackUnit is able to, although less gracefully than before. (See also my final remark, about test started notifications.) so it could be folded in: TestResult = name (U String #f) suite (Listof String) info Dictionary execution status TestExecution is also used only once so it can also be folded in -- but since it's just a generic dictionary, it can be dropped. I think it's a bad idea to collapse the two dictionaries, because they represent different information. Especially since the set of keys is open-ended, it is helpful to separate information about the test from information about its execution. TestResult = name (U String #f) suite (Listof String) info Dictionary status Now, status is one of three options, the failure one has a dictionary so it can be removed (folded into the above). I object again to the conflation of unrelated dictionaries. So overall, it looks like a simple struct with a name, a suite (kind of, defined indirectly by a hierarchy of string lists), a generic dictionary for stuff, and a status. This is all modulo some questions/issues that are unclear to me: * It looks like it tries to break away too many pieces into a formal description. For example, it looks like an overkill to have fields with the actual value and expected value and worse -- the comparison. For example, What happens when the comparison is parameterized (eg, close within dx to some number)? Not every testing framework might use 'actual and 'expected, and even frameworks that do might not use them all the time. For example, in rackunit: (check-equal? 'apple 'orange) would result in the following failure attributes: 'actual = 'apple 'expected = 'orange 'comparison = 'equal? (I think rackunit currently reports the check name, 'check-equal?, rather than the comparison name. It could work either way, or maybe it could include both.) On the other hand, something like check-not-false might only report an actual value. And a check parameterized over a tolerance could just include the tolerance as an extra attribute. * What happens when there's no specific expected value to compare? For example, run some two pieces of code 10 times each and check that the average runtime of the first is below the runtime of the second. This could be phrased in terms of an expected value, but in a superficial way, and will prevent useful information from being expressed (since the information would have to be reduced to two numbers). You can include whatever information you want. That's why it's a dictionary, rather than a fixed set of fields. The real question is how a test result displayer will know how to interpret the fields correctly. I think a useful default is to show all attributes with keys that are interned symbols or strings. Custom attributes would only work for test result displayers that know about them. * What if you don't want to hold on to the value? (For example, free some related resource.) Then convert it to a string and keep the string. (This is what the rackunit gui does to report custodian-managed leftovers.) * This solidifies the list-of-strings as a representation of the test hierarchy. But perhaps there is no way to avoid this -- if it's made into a proper hierarchy of objects it will probably complicate things in a way that requires the listener to get update events that tells it how the structure changed. I was actually going to propose something more complicated for the hierarchy, but I figured it was better to leave that for later. I'm certainly open to changing this part. * I'm not sure about the error result. It seems to me that this is a meta issue that you're
[racket-dev] Packaging
You may recall from the meeting over the summer that I promised a packaging Christmas present to Racket. I'm over a month late, but here it is: http://faculty.cs.byu.edu/~jay/tmp/pkgs/spec.html I lay out some goals for the new packaging system and make a concrete proposal. Please share your feedback so I can direct my efforts better. Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Packaging
Minor comment: why encourage names like libgtk and libgtk2 instead of a major and minor version number (ala PLaneT)? Don't we want those two libraries to be associated somehow (at least loosely)? Also, it also isn't clear which of the complaints with PLaneT you're actually dealing with. I don't see anything about security for example or the discovery issue. And one fairly abstract worry: can I end up in a situation where I had a working system with dependencies between packages being obvious but install a new package and fall out of the obvious realm in some potentially confusing way? Robby On Friday, February 18, 2011, Jay McCarthy wrote: You may recall from the meeting over the summer that I promised a packaging Christmas present to Racket. I'm over a month late, but here it is: http://faculty.cs.byu.edu/~jay/tmp/pkgs/spec.html I lay out some goals for the new packaging system and make a concrete proposal. Please share your feedback so I can direct my efforts better. Jay -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Packaging
I read your contribution with great interest. One problem that is not addressed, as far as I have seen, is that any idiot, like me, can install his/her contributions (modules/collections/packages or whatever) For a simple windows 7 user as I it is rather difficult to use command line instructions. I plead for an easy to use gui for making contributions. I also plead for an option to delete contributions that have never been dowloaded by anyone else than the contributor (would that be possible?) I once contributed a module wrongly. I am sure it has never been dowloaded, but I have no option to correct my silly mistake. All I could do is to upload an empty version, but the former version is still there. As for security, would it be possible to sandbox a download with some restrictions/options of permissions (opening files, creating new files, deleting files, renaming files, calling foreign applications, enter internet or email, and so on)??? Jos -Original Message- From: dev-boun...@racket-lang.org [mailto:dev-boun...@racket-lang.org] On Behalf Of Jay McCarthy Sent: 18 February 2011 22:21 To: dev Subject: [racket-dev] Packaging You may recall from the meeting over the summer that I promised a packaging Christmas present to Racket. I'm over a month late, but here it is: http://faculty.cs.byu.edu/~jay/tmp/pkgs/spec.html I lay out some goals for the new packaging system and make a concrete proposal. Please share your feedback so I can direct my efforts better. Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Packaging
Jay - 1. Thanks for having this out - this is a great start and a very important problem to solve 2. Is it correct that *heap* maps to the account name in planet? Such as jaymccarthy, schematics, or bzlib? There is always tension between the naming by capability or author in package systems. Do we have a preference one way or another? 3. For all the modules that are currently in core, it might make sense to simply lump them under a *core* heap to simplify the reorganization, but it's clear that not all packages are strictly required, so it would eventually be great to separate them out: 1. absolute core (scheme racket language, no GUI) 2. GUI 3. other languages 4. etc 4. Agreed that the installation/compilation takes way too long. Would like the ability to turn off doc installation - I believe docs needs to be online, and easily updated, i.e. wiki format 5. Another thing to improve the time of installation is to have pre-compiled binaries. If this is a possibility, then perhaps binary-only package should also be considered. That might not be the open source ideal, but can increase more adoption 6. Planet packages also have problems with dependency breakage when the dependency is a development link. So that will need to be addressed 7. It's hard to understand the Glue section - are you suggesting having the URL embedded into the require spec? I hope I am reading it incorrectly, as such hardcoding should be avoided. 8. The Linux repo (such as apt or yum) should be the preferred approach here - having a repo definition that defines where to search for packages. Such approach allows for having planet mirrors and private repos The above are what I can think of for now. Cheers. yc On Fri, Feb 18, 2011 at 1:21 PM, Jay McCarthy jay.mccar...@gmail.comwrote: You may recall from the meeting over the summer that I promised a packaging Christmas present to Racket. I'm over a month late, but here it is: http://faculty.cs.byu.edu/~jay/tmp/pkgs/spec.html I lay out some goals for the new packaging system and make a concrete proposal. Please share your feedback so I can direct my efforts better. Jay _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Packaging
One more comment: one of PLaneT's design goals what that if you have a working system and you install a new planet package, then you didn't break any of the working parts from before. The new system doesn't seem to have that as a design goal anymore (I noticed automatic upgrades and freezing being an explicit operation but there may be other places). Do you have a rationale for deviating from this seemingly nice property? Robby On Fri, Feb 18, 2011 at 3:21 PM, Jay McCarthy jay.mccar...@gmail.com wrote: You may recall from the meeting over the summer that I promised a packaging Christmas present to Racket. I'm over a month late, but here it is: http://faculty.cs.byu.edu/~jay/tmp/pkgs/spec.html I lay out some goals for the new packaging system and make a concrete proposal. Please share your feedback so I can direct my efforts better. Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Latex files
Sorry. I think we should remove the .cls files and say that users have to put them somewhere that Latex will find them (e.g., set `TEXINPUTS'). _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Latex files
It wouldn't hurt to include a little script that downloads the correct versions of the files from racket-lang.org somewhere and puts them where they are currently, tho, no? Robby On Fri, Feb 18, 2011 at 7:56 PM, Matthew Flatt mfl...@cs.utah.edu wrote: Sorry. I think we should remove the .cls files and say that users have to put them somewhere that Latex will find them (e.g., set `TEXINPUTS'). _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Latex files
On Fri, Feb 18, 2011 at 9:42 PM, Robby Findler ro...@eecs.northwestern.edu wrote: It wouldn't hurt to include a little script that downloads the correct versions of the files from racket-lang.org somewhere and puts them where they are currently, tho, no? At least for the JFP file, it's pretty clearly a copyright violation for us to distribute the file at all -- on the web page or elsewhere. We'd have to download it from the CUP site. For the ACM file, I emailed the contact email for the style file about the license. sam th Robby On Fri, Feb 18, 2011 at 7:56 PM, Matthew Flatt mfl...@cs.utah.edu wrote: Sorry. I think we should remove the .cls files and say that users have to put them somewhere that Latex will find them (e.g., set `TEXINPUTS'). _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev -- sam th sa...@ccs.neu.edu _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Packaging
On Fri, Feb 18, 2011 at 8:17 PM, Robby Findler ro...@eecs.northwestern.edu wrote: One more comment: one of PLaneT's design goals what that if you have a working system and you install a new planet package, then you didn't break any of the working parts from before. The new system doesn't seem to have that as a design goal anymore (I noticed automatic upgrades and freezing being an explicit operation but there may be other places). Do you have a rationale for deviating from this seemingly nice property? I think (from my experience working on this problem in the Fortress context) that this property isn't as nice as it seems to start with. It has a few important problems: 1. It can't possible be true in general, since two packages or (especially) two versions of the same package can contend for the same system resource (a file, a database, a network port, a parameter, a bit of Racket global state, etc). 2. It's in conflict with two other important goals -- upgrading packages (to remove bugs, add features, etc) and reducing resource usage (disk, memory, etc). 3. Often, sharing state in a package/library is very important. For example, generative structs have to be generated only once to be properly shared. With the planet policy, it's all to easy to install two packages, both of which require (planet foo/bar), get different instantiations of foo/bar, and have the two packages be subtly incompatible (it's even worse when the incompatibility is in some shared resource other than generative structs, like shared files). We've had this problem with planet libraries, I believe. I believe the right solution is to allow individual library authors to specify that they want the planet policy, or some other policy, to be applied to particular dependencies, without making it the default. Robby On Fri, Feb 18, 2011 at 3:21 PM, Jay McCarthy jay.mccar...@gmail.com wrote: You may recall from the meeting over the summer that I promised a packaging Christmas present to Racket. I'm over a month late, but here it is: http://faculty.cs.byu.edu/~jay/tmp/pkgs/spec.html I lay out some goals for the new packaging system and make a concrete proposal. Please share your feedback so I can direct my efforts better. Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev -- sam th sa...@ccs.neu.edu _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] dev Digest, Vol 25, Issue 38 (packaging)
If you do this, please at least make installing the full documentation an option. Not everyone can ensure that they will always be online when working with Racket. Sent from my iPad On Feb 18, 2011, at 5:56 PM, dev-requ...@racket-lang.org wrote: 4. Agreed that the installation/compilation takes way too long. Would like the ability to turn off doc installation - I believe docs needs to be online, and easily updated, i.e. wiki format _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
