Re: [racket-users] Re: Pretty display of tabular data?
On Thursday, March 14, 2019 at 12:26:39 AM UTC-5, Alex Harsanyi wrote: > > There are now several projects announced on this list, all of them deal > with > data analysis on one way or the other. Would it be possible to join forces > and merge these projects so that we end up with one library that servers > multiple purposes equally well? Something where the final product is > greater > than the sum of its parts... > > Or perhaps these libraries have aims that are so different from each other > that the only thing they share is a very abstract concept of "table"? > I think my project "plisqin" is one of those you are thinking of. Matt's "tbl" is also one. I'm also keeping an eye on Ryan's "sql". Are there any more you were thinking of? Regarding joining forces/merging these projects, this is a good question that I think warrants discussion. So I'll share my thoughts. Obviously I can't speak for all of us, but right not I only see the "very abstract concept of "table"" as potential shared code. (Also, learning about snip% earlier in this thread was awesome. I'd love to use something like that in my project.) I think the differences between plisqin and tbl are fairly obvious - plisqin is an alternative to SQL while tbl is an alternative to "Python/NumPy/SciPy, or R/Tidyverse (or, horrors, plain R)" Now comparing Ryan's sql to plisqin is a different story. These projects are both alternatives to SQL. But I think there is enough difference between our approaches and scope to warrant separate projects, at least for now. 1) sql seems to be mostly implemented as macros. plisqin is mostly implemented as procedures. 2) plisqin has some design decisions that some might consider "too much magic", namely inline joins and "inject-able aggregates" (need better name) as documented here: https://docs.racket-lang.org/plisqin/intro.html. Whereas sql-the-package seems to more closely mirror SQL-the-language - it would be difficult to surprise yourself with the SQL you generate. 3) I am trying to design #lang plisqin so that people with no Lisp experience can use it. (Whether I will succeed is another matter...) I apologize to Ryan C if I have mischaracterized sql. I'd like to have a longer conversation about this, but maybe this list is not the right place. (Also, Ryan, if you think our goals are more similar than I do, I'd be happy to work with you. You're definitely a more experienced Racketeer and it would surely boost my code quality.) - Ryan Kramer -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Pretty display of tabular data?
On Thu, Mar 14, 2019 at 11:51 AM wrote: > > 3. Which brings me to to the biggest obstacle. > > — beginners make mistakes > — languages must explain mistakes > — beginners better understand these explanations > > Yes to all three of your points. I was thinking about your last point in my first exploration, and error handling/reporting is something that is at the front of my mind as I look at things like Pyret's sanitizers, as well as the many/manifest ways that I see students struggle/fail with R. I don't think I'll be able to "solve" this in the first instance, but keeping centered the lived experience of the novice when learning to work with data (which often comes in "from the wild," and is not consistent/reliable/etc.), how to manage that messiness, and how to report back in a learning-centric way when they encounter difficulty is going to be 1) hard and 2) critical. Which is perhaps a reflection/restatement/variation of what your last point. I'm generally familiar with the broad space in which Guillaume's work is situated, and have seen in presented/read it. So, yes. And, thank you for the pointer to the rewriter library. The Racket stack is rich enough at this point that I forget the wealth of tools that are available. Cheers, M -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Pretty display of tabular data?
> On Mar 14, 2019, at 1:26 AM, Alex Harsanyi wrote: > > Would it be possible to join forces > and merge these projects so that we end up with one library that servers > multiple purposes equally well? 1. I have been at this point many times. A big example is the testing libraries that we use for teaching (test-engine/) vs production programming (rackunit/). Initially I wanted the former to compile to the latter, but because we need John’s stepper to work and we focus on DrRacket and we have pedagogic constraints, that just didn't work out. Another example would be drawing and event handling, see pict and images and big-bang vs the GUI tool box. They are even further apart than the testing libraries. 2. So Matt’s injection of two perspectives is a really good one. Still it might make sense to think of the one for devs as a scaled-up version of those for pedagogic setting and vice versa. This can happen at — the conceptual level: let me introduce concepts in CS1 or my high school course that are useful later when you use Real Racket — the implementation level: the pedagogic one several restricts the dev-focused one and improves the error messages severely. 3. Which brings me to to the biggest obstacle. The key difference between the teaching languages and Racket is that the former assume that — beginners make mistakes — languages must explain mistakes — beginners better understand these explanations This is hard to get right because implementations bake in assumptions about who the users are w/o (usually) paying attention to this idea. Whoever works on the pedagogic version, consider linking in the rewriter library so you benefit from Guillaume Marceau’s research: (require lang/private/rewrite-error-message) (get-rewriten-error-message x) — Matthias -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Re: Pretty display of tabular data?
>
> There are now several projects announced on this list, all of them deal
> with
> data analysis on one way or the other. Would it be possible to join forces
> and merge these projects so that we end up with one library that servers
> multiple purposes equally well? Something where the final product is
> greater
> than the sum of its parts...
>
> Or perhaps these libraries have aims that are so different from each other
> that the only thing they share is a very abstract concept of "table"?
>
Yes?
It makes complete sense, from a practical perspective, to not duplicate
work.
Without bikeshedding ("where should the conversation happen?", "what color
should the logo be?"), there are easier and harder design constraints. For
example, I realized that any sufficiently interesting table interface
would, ultimately, embed a copy of LISP... wait... would be a half-assed
reimplementation of SQL. So, in my rethink, I just set things on top of the
sql library, thus providing the "base language" from which I would work. If
SQL can't do it, it's possible I don't need to do it, and it is 100%
certain that a first-year, who has been programming for 6 weeks, will not
have introductory data questions that cannot be handled by my "target
language."
Keeping a non-leaky abstraction (or, as non-leaky as possible) that lets a
student who is early in HtDP do work with data (in a principled way...
another loaded perspective...) is very important to me. I'll trade all the
fancy databases in the world (as well as the full expressivity of SQL, and
performance for datasets beyond 100K rows, and and and...) for an interface
that does a small number of things very well for novices. If we can do our
design work so that there are demarked shells of increasing complexity,
then yes, I'm confident that we could find ways to combine forces.
If nothing else, I'm already eyeing other libraries that I want to "wrap,"
so that they operate on the substrate I'm laying. I want simplified
plotting (with possibly reduced levels of customization from full 'plot',
either enforced through interfaces or simply enforced by reducing the
documentation for the interface), basic tools for summarizing and analyzing
data (I'm thinking of wrapping the "data-science" library that is floating
around, but not packaged)... so, yes. I don't want to reimplement
everything, but I do need a common substrate. At the moment, I've decided
that anywhere Racket runs (and that my students will use it) is powerful
enough to also have SQLite, and for my time, energy, and task, there are
worse choices than just using SQL.
But, back to bike-shedding... at the least, it might be interesting to kick
around a set of requirements/wants/needs/desires, and from there think
about next steps in design. However, blank-whiteboard design phase work is
challenging in distributed/asynchronous modes, so I'm also concerned that a
mailing list amongst people who do not know each-other is a hard way to do
good design on something nuanced... but, that could just be a failing of
mine. Suggestions for "next steps" on collaboration are something I'm
absolutely open to.
At the end of the day, I need tools for next Fall (ideally, sooner, so I
can begin developing course materials); that's a hard, non-optional
design/implementation deadline, no matter how much interest and goodwill
there is to collaborate.
Cheers,
Matt
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
