Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
-Original Message- >From: Alexander Johannesen >Sent: Nov 1, 2010 9:33 PM >To: CODE4LIB@LISTSERV.ND.EDU >Subject: Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...] ... >So this is not a *dumb* idea, nor is it one of simply saying "it's the >API, stupid." It does goes deeper than that. Not that you / we should >do it, but that's it a good exercise that should have happened. And, >heck, might even have happened. > Alex explains eloquently the problems and possibilities of a DSL for the library world, and semantic maps generally. This is also in answer to some of Patrick E.'s earlier questions about what I am trying to do. >From the readme ( http://www.avantilibrarysystems.com/README.txt ): "Avanti Nova is a general-purpose semantic mapping system. Although first conceived for use as new kind of library catalog, it can be used in any situation to dynamically map the relationships in a set of objects. Some of its core ideas were first used in Avanti MicroLCS, and these were later generalised and extended in Nova. "Nova is firstly a data structure with a scripting language interface to it. Programs are written in Nova to create and manage semantic maps. It also runs as a client-server system, and can be used from other environments like Unix shell scripts. And since it is written in Java, it naturally provides a Java API. "The core idea in Nova is to represent a semantic map as a very large array. This allows for great flexibility in manipulating relationships in semantic maps." The README also has some actual examples of programs in the Nova language as it was a couple of months ago, and all of this stuff works right now, although there is a long way to go. Simple and crude perhaps, but that's how all things begin. I see much potential value in this thing even though much of it could be done with generic tools. I also believe that the generic tools (or anything for that matter) become their own mind-prisons if we rely too much on them. Soon we're speaking and thinking only in Ruby, Python, Perl, Django, Java, MARC or whatever with all their inherent metaphors and limitations. Pick up a hammer and everything starts looking like nails. It even goes into more general things: "Everything is an API." We become like the blind men and the elephant. There are DSLs that serve other domains quite well. Mathematica and R are good examples. Why not a DSL that speaks the language of semantic maps directly? Semantic mapping can be a very large and rich domain. A DSL that seperates the problem from the API, as Alex points out, can handle many problems much more elegantly and flexibly than APIs or something that's tied to a specific tool's way of looking at things. Another line of thought that's a big innovation killer is: "Something like this has been done before, so further work on the problem is not worthwhile." Carried to an extreme we'd all still be clunking around on wheels chiseled from stone slabs today "because they've always been good enough." MARC is a very heavy stone wheel that has become a millstone around the neck of the library profession. My approach in doing this is to not worry or think too much about what has been done before. Start fresh with the basic concepts and what makes sense and try to make something useful out of it, from the ground up. Worrying about what's been done before leads to copying and a lot of original ideas may get rejected. If some wheels get reinvented, so what? Reinventing the wheel often leads to better wheels. Sometimes you have to build something first even if it doesn't solve a specific problem in order to see its value. I am no expert Nova programmer yet because I'm still building the thing. There's no community around this yet. It may be homegrown by a lone developer right now but that's the way many things start. And besides, as far as doing it goes, what the heck? Peter Schlumpf www.avantilibrarysystems.com
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
On Tue, Nov 2, 2010 at 5:03 AM, Jonathan Rochkind wrote:
> I would be very unlikely to use someone's homegrown library specific
> scripting language. However, if you want to make a library for an existing
> popular scripting
> language that handles your specific domain well, I'd be quite likely to use
> that if I had a problem with your domain and I was comfortable with the
> existing popular scripting language, i'd use it for sure.
Hmm. The balance between the old and tried, and the new and
experimental will, forever, cause these kinds of discussions. Now, I
agree with the basic sentiment of what you're saying, but ...
> Odds are your
> domain is not really "libraries" (that's not really a software problem
> domain), but perhaps as Patrick suggests "dealing with relationships among
> semantic objects", and then odds are libraries are not the only people
> interested in this problem domain.
I've worked in the three basic tiers of library development world; the
plain vanilla programming world, the semantic web world, and the dark
dungeons of the Cult of MARC. Is the domain of library IT solved by
the generic technologies used? No.
There's nothing bad about a DSL, in fact, I encourage it. If you want
to get away from MARC, say, then having a DSL that approaches meta
data on the programmatic level directly is a wonderful abstraction.
But yes, we have to separate API from language. And API is, mostly
these days, simply a function/method call on top of an abstraction,
and it processes your request with your input. A language, on the
other hand, will let you deal directly with that problem. Most DSLs
are functional abstraction pre-compiled.
The line between a library and a language perhaps these days are more
blurred than ever before, however there are certain things that I
think justifies a library DSL ;
* focus on identity management
* mergability on entities
* large distributed sets
* more defined line between data and meta data
* controlled vocabularies and structures
There's generic tools for all of these, however no one central thing
that binds them all together in a seamless way, elegant or otherwise.
No platform binds these together in an easy nor elegant way, and
perhaps such a thing would be beneficial to the library community, to
create a language that tries to create a bridge between computer
programming and what you learn in library school.
But even if we all concede that a library DSL perhaps is not a
practical solution, I'd still like to see us work on it, for nothing
more than sussing out our actual needs and wants in the process. Don't
underestimate the process of doing something that will never
eventuate, even knowingly.
> Some people like ruby because of it's support for creating what they call
> "domain specific languages", which I think is a silly phrase, which really
> just means "a libraryAPI at the right level of abstraction for the tasks at
> hand, so you can accomplish the tasks at hand concisely and without repeated
> code."
Depends on the language. Perhaps this doesn't make sense in Ruby, but
it certainly does in Scala, Haskell, and perhaps more than any, Rebol.
Even Lisp and derivatives, who can create custom structures on the
fly, are well suited to create actual languages that redefine the
language's original syntax and structure. You can redefine the hell
out of C to create any language imaginable, too, even when you
shouldn't.
A well-defined API is not a bad thing, though, but an API are
basically semantic entities in a language to parse structures.
However, a language redefines the syntax used by that language. Sure
you can create a word "record" in an API that mimics, say, a MARC
record, but the interesting part is when you redefine the syntax to
work *with* that semantic concept, like ;
external_repository {
baseURI: 'http://example.com/',
type: OAI-PHM
}
my_repository {
baseURI: 'http://example.com/',
type: RIF-CS
}
some_vocabulary {
baseURI: 'http://example.com/vocab'
type: thesauri
}
foreach record in external_repository [without tag 850] {
inject into my_repository {
with: exploded words ( tag 245 )
when: match words in some_vocabulary ( NT > 2 )
merge into: tag 850
}
}
Creating classes that deal with record merging based on identity
management and various standards would be trivial to script together
super-fast, because the underlying concepts for us is rather
well-known. Hacking this together in Java or otherwise is a test on
patience and sanity, because they are generic tools, even when known
library-type APIs are used. Of course lots of stuff is assumed in the
example, but these are well-understood assumptions (about merging
subject headings (like multiple tags handling, LCSH lookup, etc.),
about identity control, about word lookup (for example, I'm assuming
some form of stemming before matching), and on and on. A language that
half text manipulation and looku
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
DSL's are simple to create in ruby. Becuase what people call a DSL in ruby is almost always just plain ruby methods -- it's in that context that I don't even like the phrase "DSL". In ruby, developers very seldomly find a need to create actually another language with it's own grammar and parser and such -- people accomplish the same thing just by creating the right ruby methods. The way that ruby supports the concise supplying of code blocks as arguments to methods is usually the key thing people rely on to do this. Will Kurt wrote: This is one of my favorite passage from SICP: "It is no exaggeration to regard this as the most fundamental idea in programming: The evaluator, which determines the meaning of expressions in a programming language, is just another program. To appreciate this point is to change our images of ourselves as programmers. We come to see ourselves as designers of languages, rather than only users of languages designed by others." In general I think there is too much fear of using language as just another means of abstraction. While I certainly agree that creating an entire language from scratch is a bad idea, I don't think it would be insane to create a dsl to solve a common set of problems on top of an existing runtime. I actually think this would be particularly useful in the library world since there is such a range of programming talent, a dsl that simplified some common library related tasks could certainly be useful, especially if there was full language underneath. Of course there is the problem that even DSLs are not simple to create, the number of library programmers with experience in parsers and language design is probably very, very small. But the ease of creating dsl is increasing and I think their use will get more popular over time (hopefully). From: Jonathan Rochkind mailto:rochk...@jhu.edu>> Date: November 1, 2010 11:03:13 AM PDT To: "CODE4LIB@LISTSERV.ND.EDU<mailto:CODE4LIB@LISTSERV.ND.EDU>" < CODE4LIB@LISTSERV.ND.EDU<mailto:CODE4LIB@LISTSERV.ND.EDU>> Subject: Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...] Reply-To: Code for Libraries > I would be very unlikely to use someone's homegrown library specific scripting language. However, if you want to make a library for an existing popular scripting language that handles your specific domain well, I'd be quite likely to use that if I had a problem with your domain and I was comfortable with the existing popular scripting language, i'd use it for sure. Odds are your domain is not really "libraries" (that's not really a software problem domain), but perhaps as Patrick suggests "dealing with relationships among semantic objects", and then odds are libraries are not the only people interested in this problem domain. Some people like ruby because of it's support for creating what they call "domain specific languages", which I think is a silly phrase, which really just means "a libraryAPI at the right level of abstraction for the tasks at hand, so you can accomplish the tasks at hand concisely and without repeated code." Patrick Etienne wrote: Peter - I was bewildered at the notion of needing yet another scripting language, let alone one as "library domain-specific" (that wording alone throws up red flags everywhere), but I'm not here to bash ideas. Instead I looked up your site and read the small blurb about "Nova". It seems that the main objective behind your pursuit is creating a language that provides a specific data type for semantic objects (or relationships). I have to ask, what about semantic maps makes you believe that they require a specific data type rather than just being an object type? Are other scripting languages too slow to suit certain needs such that a new data type is necessitated? I really can't see this being the case. That being said, while it can be an invaluable experience to learn about making scripting languages, if there's to be any community movement toward a particular language (php, ruby, java, scheme or what have you) there has to be some very real and significant benefit. Or more directly, you seem to have specific ideas about a library domain-specific language. What do today's languages not have that you believe is so essential that you'd be willing to write a new scripting language? - Patrick E. On Sat, Oct 30, 2010 at 10:51 AM, Peter Schlumpf mailto:pschlu...@earthlink.net>> wrote: Bill, you hit a nail pretty squarely on the head. I believe this decades long fetish with MARC has to go. It was designed to efficiently store data on magtapes and doesn't make any sense in today's world. It's a huge millstone around the neck of Libraryland and it keeps them stuck in that tiny little ghetto. Anything can be a mind-prison, even P
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
This is one of my favorite passage from SICP: "It is no exaggeration to regard this as the most fundamental idea in programming: The evaluator, which determines the meaning of expressions in a programming language, is just another program. To appreciate this point is to change our images of ourselves as programmers. We come to see ourselves as designers of languages, rather than only users of languages designed by others." In general I think there is too much fear of using language as just another means of abstraction. While I certainly agree that creating an entire language from scratch is a bad idea, I don't think it would be insane to create a dsl to solve a common set of problems on top of an existing runtime. I actually think this would be particularly useful in the library world since there is such a range of programming talent, a dsl that simplified some common library related tasks could certainly be useful, especially if there was full language underneath. Of course there is the problem that even DSLs are not simple to create, the number of library programmers with experience in parsers and language design is probably very, very small. But the ease of creating dsl is increasing and I think their use will get more popular over time (hopefully). > From: Jonathan Rochkind mailto:rochk...@jhu.edu>> > Date: November 1, 2010 11:03:13 AM PDT > To: "CODE4LIB@LISTSERV.ND.EDU<mailto:CODE4LIB@LISTSERV.ND.EDU>" < > CODE4LIB@LISTSERV.ND.EDU<mailto:CODE4LIB@LISTSERV.ND.EDU>> > Subject: Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...] > Reply-To: Code for Libraries CODE4LIB@LISTSERV.ND.EDU>> > > I would be very unlikely to use someone's homegrown library specific > scripting language. > > However, if you want to make a library for an existing popular scripting > language that handles your specific domain well, I'd be quite likely to > use that if I had a problem with your domain and I was comfortable with > the existing popular scripting language, i'd use it for sure. Odds are > your domain is not really "libraries" (that's not really a software > problem domain), but perhaps as Patrick suggests "dealing with > relationships among semantic objects", and then odds are libraries are > not the only people interested in this problem domain. > > Some people like ruby because of it's support for creating what they > call "domain specific languages", which I think is a silly phrase, which > really just means "a libraryAPI at the right level of abstraction for > the tasks at hand, so you can accomplish the tasks at hand concisely and > without repeated code." > > Patrick Etienne wrote: > Peter - > > I was bewildered at the notion of needing yet another scripting > language, let alone one as "library domain-specific" (that wording > alone throws up red flags everywhere), but I'm not here to bash ideas. > Instead I looked up your site and read the small blurb about "Nova". > It seems that the main objective behind your pursuit is creating a > language that provides a specific data type for semantic objects (or > relationships). I have to ask, what about semantic maps makes you > believe that they require a specific data type rather than just being > an object type? Are other scripting languages too slow to suit certain > needs such that a new data type is necessitated? I really can't see > this being the case. That being said, while it can be an invaluable > experience to learn about making scripting languages, if there's to be > any community movement toward a particular language (php, ruby, java, > scheme or what have you) there has to be some very real and > significant benefit. > > Or more directly, you seem to have specific ideas about a library > domain-specific language. What do today's languages not have that you > believe is so essential that you'd be willing to write a new scripting > language? > > - Patrick E. > > On Sat, Oct 30, 2010 at 10:51 AM, Peter Schlumpf > mailto:pschlu...@earthlink.net>> wrote: > > Bill, you hit a nail pretty squarely on the head. I believe this decades > long fetish with MARC has to go. It was designed to efficiently store data > on magtapes and doesn't make any sense in today's world. It's a huge > millstone around the neck of Libraryland and it keeps them stuck in that > tiny little ghetto. Anything can be a mind-prison, even PHP, Python or > Django. They are all arbitrary anyway. > > And you are correct in pointing out that the natural response of librarians > to a problem is to seek consensus in a self-absorbed way. Form committees > and all that nonsense which never goes a
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
I would be very unlikely to use someone's homegrown library specific scripting language. However, if you want to make a library for an existing popular scripting language that handles your specific domain well, I'd be quite likely to use that if I had a problem with your domain and I was comfortable with the existing popular scripting language, i'd use it for sure. Odds are your domain is not really "libraries" (that's not really a software problem domain), but perhaps as Patrick suggests "dealing with relationships among semantic objects", and then odds are libraries are not the only people interested in this problem domain. Some people like ruby because of it's support for creating what they call "domain specific languages", which I think is a silly phrase, which really just means "a libraryAPI at the right level of abstraction for the tasks at hand, so you can accomplish the tasks at hand concisely and without repeated code." Patrick Etienne wrote: Peter - I was bewildered at the notion of needing yet another scripting language, let alone one as "library domain-specific" (that wording alone throws up red flags everywhere), but I'm not here to bash ideas. Instead I looked up your site and read the small blurb about "Nova". It seems that the main objective behind your pursuit is creating a language that provides a specific data type for semantic objects (or relationships). I have to ask, what about semantic maps makes you believe that they require a specific data type rather than just being an object type? Are other scripting languages too slow to suit certain needs such that a new data type is necessitated? I really can't see this being the case. That being said, while it can be an invaluable experience to learn about making scripting languages, if there's to be any community movement toward a particular language (php, ruby, java, scheme or what have you) there has to be some very real and significant benefit. Or more directly, you seem to have specific ideas about a library domain-specific language. What do today's languages not have that you believe is so essential that you'd be willing to write a new scripting language? - Patrick E. On Sat, Oct 30, 2010 at 10:51 AM, Peter Schlumpf wrote: Bill, you hit a nail pretty squarely on the head. I believe this decades long fetish with MARC has to go. It was designed to efficiently store data on magtapes and doesn't make any sense in today's world. It's a huge millstone around the neck of Libraryland and it keeps them stuck in that tiny little ghetto. Anything can be a mind-prison, even PHP, Python or Django. They are all arbitrary anyway. And you are correct in pointing out that the natural response of librarians to a problem is to seek consensus in a self-absorbed way. Form committees and all that nonsense which never goes anywhere. They are happy enough going around in circles, like the Nowhere Man making all his nowhere plans for nobody. My hope is that some among us would just undertake these problems ourselves. Outside of the realm of the libraries and the limiting mindsets many of us work in. We've all got ideas. Fire up vi and get busy and make something happen, like a library domain-specific language. Start fresh. There is nothing wrong with that. What's wrong is how the library community goes about such things. Let's go somewhere. Peter Schlumpf www.avantilibrarysystems.com
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
Peter - I was bewildered at the notion of needing yet another scripting language, let alone one as "library domain-specific" (that wording alone throws up red flags everywhere), but I'm not here to bash ideas. Instead I looked up your site and read the small blurb about "Nova". It seems that the main objective behind your pursuit is creating a language that provides a specific data type for semantic objects (or relationships). I have to ask, what about semantic maps makes you believe that they require a specific data type rather than just being an object type? Are other scripting languages too slow to suit certain needs such that a new data type is necessitated? I really can't see this being the case. That being said, while it can be an invaluable experience to learn about making scripting languages, if there's to be any community movement toward a particular language (php, ruby, java, scheme or what have you) there has to be some very real and significant benefit. Or more directly, you seem to have specific ideas about a library domain-specific language. What do today's languages not have that you believe is so essential that you'd be willing to write a new scripting language? - Patrick E. On Sat, Oct 30, 2010 at 10:51 AM, Peter Schlumpf wrote: > Bill, you hit a nail pretty squarely on the head. I believe this decades > long fetish with MARC has to go. It was designed to efficiently store data > on magtapes and doesn't make any sense in today's world. It's a huge > millstone around the neck of Libraryland and it keeps them stuck in that tiny > little ghetto. Anything can be a mind-prison, even PHP, Python or Django. > They are all arbitrary anyway. > > And you are correct in pointing out that the natural response of librarians > to a problem is to seek consensus in a self-absorbed way. Form committees > and all that nonsense which never goes anywhere. They are happy enough going > around in circles, like the Nowhere Man making all his nowhere plans for > nobody. > > My hope is that some among us would just undertake these problems ourselves. > Outside of the realm of the libraries and the limiting mindsets many of us > work in. We've all got ideas. Fire up vi and get busy and make something > happen, like a library domain-specific language. Start fresh. There is > nothing wrong with that. What's wrong is how the library community goes > about such things. > > Let's go somewhere. > > Peter Schlumpf > www.avantilibrarysystems.com -- Patrick K. Etienne Systems Analyst Georgia Institute of Technology Library & Information Center (404) 385-8121
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
On Oct 30, 2010, at 10:50 AM, Peter Schlumpf wrote: > And you are correct in pointing out that the natural response of librarians > to a problem is to seek consensus in a self-absorbed way. Form committees > and all that nonsense which never goes anywhere. They are happy enough going > around in circles, like the Nowhere Man making all his nowhere plans for > nobody. The above certainly does seem to jive with my experience. Going around in circles. Endless consensus. Librarianship could use a bit more democracy and/or science to their (our) method. At the same time I agree that "E V E R Y T H I N G ! ! !" is wrong with the profession creating our own language. -- EL Morgano
Re: [CODE4LIB] Let's go somewhere [was PHP vs. Python...]
>>> What's wrong with the library world developing its own domain language? >> >> >>EVERYTHING!!! >> >>We're already in a world of pain because we have our own data formats and >>ways of dealing with them, all of which have basically stood idle while 30 >>years of advances computer science and information architecture have whizzed >>by us with a giant WHOOSHing sound. >> >>Having a bunch of non-experts design and implement a language that's >>destined from the outset to be stuck in a tiny little ghetto of the >>programming world is a guaranteed way to live with half- or un-supported >>code, no decent libraries, and yet another legacy of pain we'd have to >>support. >> >> I'm not picking on programming in particular. It's a dumb-ass move EVERY >>time a library is presented with a problem for which there are experts and >>decades of research literature, and it choses to ignore all of that and >>decide to throw a committee of librarians (or whomever else happens to be in >>the building at the time) at it based on the vague idea that librarians are >>just that much smarter (or cheaper) than everyone else (I'm looking at you, >>usability...) then Peter said: My hope is that some among us would just undertake these problems ourselves. > Outside of the realm of the libraries and the limiting mindsets many of us > work in. We've all got ideas. Fire up vi and get busy and make something > happen, like a library domain-specific language. Start fresh. There is > nothing wrong with that. What's wrong is how the library community goes > about such things. > Yes, he hit the nail square on the head. But i think his point is that a huge community of really intelligent people (specifically, not librarians, but generally, not just every isolated pobody with a hair-brain scheme) hit vi and got busy a long time ago. What do you expect to gain from the huge PIA you propose at the cost of sacrificing everything a HUGE community of skilled programmers offers? have fun. -elliot
[CODE4LIB] Let's go somewhere [was PHP vs. Python...]
Bill, you hit a nail pretty squarely on the head. I believe this decades long fetish with MARC has to go. It was designed to efficiently store data on magtapes and doesn't make any sense in today's world. It's a huge millstone around the neck of Libraryland and it keeps them stuck in that tiny little ghetto. Anything can be a mind-prison, even PHP, Python or Django. They are all arbitrary anyway. And you are correct in pointing out that the natural response of librarians to a problem is to seek consensus in a self-absorbed way. Form committees and all that nonsense which never goes anywhere. They are happy enough going around in circles, like the Nowhere Man making all his nowhere plans for nobody. My hope is that some among us would just undertake these problems ourselves. Outside of the realm of the libraries and the limiting mindsets many of us work in. We've all got ideas. Fire up vi and get busy and make something happen, like a library domain-specific language. Start fresh. There is nothing wrong with that. What's wrong is how the library community goes about such things. Let's go somewhere. Peter Schlumpf www.avantilibrarysystems.com -Original Message- >From: Bill Dueber >Sent: Oct 29, 2010 8:18 PM >To: CODE4LIB@LISTSERV.ND.EDU >Subject: Re: [CODE4LIB] PHP vs. Python [was: Re: Django] > >On Fri, Oct 29, 2010 at 6:28 PM, Peter Schlumpf wrote: > >> What's wrong with the library world developing its own domain language? > > >EVERYTHING!!! > >We're already in a world of pain because we have our own data formats and >ways of dealing with them, all of which have basically stood idle while 30 >years of advances computer science and information architecture have whizzed >by us with a giant WHOOSHing sound. > >Having a bunch of non-experts design and implement a language that's >destined from the outset to be stuck in a tiny little ghetto of the >programming world is a guaranteed way to live with half- or un-supported >code, no decent libraries, and yet another legacy of pain we'd have to >support. > > I'm not picking on programming in particular. It's a dumb-ass move EVERY >time a library is presented with a problem for which there are experts and >decades of research literature, and it choses to ignore all of that and >decide to throw a committee of librarians (or whomever else happens to be in >the building at the time) at it based on the vague idea that librarians are >just that much smarter (or cheaper) than everyone else (I'm looking at you, >usability...) > > -Bill- > > > > >-- >Bill Dueber >Library Systems Programmer >University of Michigan Library
