It seems to me that all we really need is a list of all first generation revdeps on a given package (any version), each specified maybe as (in pseudodata):
type VersionRange = (Version,Version) type RevDep = (PackageName,[VersionRange]) RevDeps = [RevDep] (alt M.Map PackageName [VersionRange]) RevDepStructure = M.Map PackageName RevDeps I can see how things get uglier if we instead do: RevDepStructure = M.Map PackageName (M.Map Version RevDeps) but I think the latter structure is potentially too granular for most uses? --Gershom On Mon, Oct 14, 2013 at 2:44 PM, Duncan Coutts <dun...@well-typed.com> wrote: > On Mon, 2013-10-14 at 12:18 -0400, Carter Schonwald wrote: >> So you're saying we need to add a db proper to hackage2 server, like SQLite >> or Postgres so as to make it more performant for interesting features? >> What's needed to do that? > > I don't think that's needed at all. > > What we need is to look carefully at what we want from a revdeps feature > and then look carefully at what info should be stored (ie cached) and > what should be calculated on demand. > > It just needs someone to spend some time on the feature, there's no need > to panic about the data storage (especially before looking at what is > actually needed). > > Duncan > >> On Monday, October 14, 2013, Matthew Gruen wrote: >> >> > Hey, thanks for taking an interest in this. There is kind of a >> > space-time-featureset tradeoff with revdeps, unfortunately. It seems to me >> > like the challenge doesn't come so much from indirect deps, but rather from >> > the sheer number of versions there are! There are 5600+ packages and 33800+ >> > versions, and the set of dependencies of foo-1.2 vs foo-1.3 can be >> > completely different. Not to mention, when you look at the package page for >> > a particular version, you may want to see the revdeps *for that version*, >> > and at this point there are a billion possible combinations. npm's UI, in >> > contrast, has an "always in HEAD" kind of feel. >> > >> > If you are interested in how the disabled implementation worked, by the >> > way, I have a post here (as gracenotes): >> > https://github.com/haskell/hackage-server/issues/40 >> > >> > So revdeps is great, and there are many different killer uses for it, such >> > as popularity metrics and finding upper version bounds which need updating. >> > In sum, these require keeping around a lot of data. Because there's so >> > much, acid-state is probably not a good place to store it. Not to mention, >> > it's time-consuming to generate it from scratch and space-consuming to keep >> > around data structures to incrementally update it. So there are some tool >> > limitations, but it seems fundamentally tricky to do frugally. >> > >> > Matt >> > >> > >> > >> > On Mon, Oct 14, 2013 at 12:20 AM, Jens Petersen < >> > j...@community.haskell.org> wrote: >> > >> >> Not really complaining :) but one of the things I had been looking >> >> forward to with Hackage2 was the display of reverse-dependencies, but I >> >> gather it was disabled for now (because it loads the server too much when >> >> updating the data iirc?). >> >> >> >> I was thinking perhaps a compromise would be to show only direct >> >> reverse-dependencies rather than including all indirect dependencies too. >> >> Perhaps that would make it scale better? I think direct dependency >> >> information is the most useful anyway (and also what packdeps provides). >> >> >> >> Before doing anything I thought I would ask here first if this makes >> >> sense and should be work? >> >> I for one would still really like to see basic reverse dependencies info >> >> listed on hackage (I noticed yesterday that npm has them too). >> >> >> >> Jens >> >> >> >> _______________________________________________ >> >> cabal-devel mailing list >> >> cabal-devel@haskell.org >> >> http://www.haskell.org/mailman/listinfo/cabal-devel >> >> >> >> >> > >> _______________________________________________ >> cabal-devel mailing list >> cabal-devel@haskell.org >> http://www.haskell.org/mailman/listinfo/cabal-devel > > > -- > Duncan Coutts, Haskell Consultant > Well-Typed LLP, http://www.well-typed.com/ > > _______________________________________________ > cabal-devel mailing list > cabal-devel@haskell.org > http://www.haskell.org/mailman/listinfo/cabal-devel _______________________________________________ cabal-devel mailing list cabal-devel@haskell.org http://www.haskell.org/mailman/listinfo/cabal-devel
