[Lift] Re: BSON support in lift-json
Hi,
Here's another idea which might work. It is a bit hackish and needs
further analysis, but small experiments I did were promising.
What if we change JsonAST just a little by adding a new abstract node
called JLiteral to represent simple nonstructural types (note, it is
sealed).
object JsonAST {
sealed abstract class JValue
sealed abstract class JLiteral extends JValue
case class JInt(value: BigInt) extends JLiteral
...
}
Then in a same source file extend that by adding BSON extensions.
object BsonAST {
abstract class JObjectId(val id: Long) extends JsonAST.JLiteral
...
object JObjectId {
def apply(id: Long) = new JObjectId(id) {}
def unapply(x: JObjectId): Option[Long] = Some(x.millis)
}
}
Note, BsonAST extension literals are not case classes but abstract
classes + manually coded extractors and factories. Now users of basic
JSON processing would not need to care about BSON extensions. For them
the pattern matcher would work as it works now. For instance
exhaustiveness checking still emits warnings if cases are incomplete.
BSON users can import BSON extensions and use extractors:
import JsonAST._
import BsonAST._
bson map {
case JInt(x) => ...
case JDate(t) => ...
...
}
Cheers Joni
On Mar 8, 8:38 pm, Ross Mellgren wrote:
> I personally think hybrid approaches make sense for certain designs even if
> they are a little odd.
>
> My thought originally was having a new member of the ADT which is not final
> which represents "extensions", e.g.
>
> JValue
> \_ JExtension
> \_ JDate
>
> where JExtensions could be ignored or passed through unchanged by most of the
> stuff, and only special readers/writers would know what to do with them.
>
> -Ross
>
> On Mar 8, 2010, at 12:45 PM, David Pollak wrote:
>
>
>
> > On Mon, Mar 8, 2010 at 12:50 AM, Joni Freeman
> > wrote:
> > This is a tricky one. The problem with extending the AST is that the
> > AST is implemented as an algebraic data type. And by definition it is
> > not possible to extend such a type.
>
> > Just throwing an idea out and it's likely a bad one.
>
> > I ran into a similar issue with Box and chose to do a hybrid (aka
> > Frankenstein) ADT/OO paradigm.
>
> > Perhaps it's possible to provide subclasses of certain ADT items (e.g.,
> > JDate extends JInt) such that if you pattern match on a JInt, you get the
> > millis as long, but if you pattern match against JDate, you get a date
> > extracted if it's the JDate subclass.
>
> > Once again, it's likely to be a bad idea as it caused a lot of angst in Box
> > and I'm not sure if the paradigm is one that's worth perpetuating.
>
> > One way to add BSON support is to create a new AST for it which
> > includes all extended literals. Then add a few core functions for that
> > ADT (map, etc.) and maybe a function which can encode BSON as JSON
> > (bvalue.toJson). Encoding BSON as JSON would give some features for
> > free, for instance toXml. Anyway, this approach would probably cause
> > some code duplication between lift-json and lift-bson.
>
> > Converting the JSON AST to an object oriented design would be another
> > approach. Then adding new AST nodes would not be a problem. But that
> > would be a huge change to the lib. Probably too big at this phase.
>
> > Since BSON is a superset of JSON we could refactor current lift-json
> > to be lift-bson and then implement lift-json on top of it. On a
> > cursory look this feels cleanest but there might be some performance
> > penalties for normal JSON processing due to conversions.
>
> > To be honest, I'm not yet sure what would be the best approach.
>
> > Cheers Joni
>
> > On Mar 5, 10:08 pm, Ross Mellgren wrote:
> > > The JSON stuff is mostly just an AST and encoding/decoding from the JSON
> > > wire format is almost just an addon. Then, it would be a matter of adding
> > > AST objects for those new things. Could be a use for phantom types ;-)
>
> > > I'd be interested to hear Joni's view on how it might fit, since he's the
> > > most familiar.
>
> > > -Ross
>
> > > On Mar 5, 2010, at 1:26 PM, Tim Nelson wrote:
>
> > > > I definitely agree with keeping the BSON code separate or possibly
> > > > having a strict JSON mode.
>
> > > > Tim
>
> > > > On Fri, Mar 5, 2010 at 12:13 PM, Timothy Perrett
> > > > wrote:
> > > >> Probably a sub-ordinate module would be preferable... one that builds
> > > >> on the lift-json stuff and doesn't pollute the "normal" JSON usage.
>
> > > >> Joni, what are your thoughts?
>
> > > >> Cheers, Tim
>
> > > >> On 5 Mar 2010, at 17:59, Tim Nelson wrote:
>
> > > >>> I finally had the opportunity to look into the couchdb code and I must
> > > >>> say it is rather impressive.
>
> > > >>> I would like to utilize the code in JSONRecord.scala in scamongo [1].
> > > >>> However, MongoDB uses a variation of JSON they call BSON, which they
> > > >>> actually just published a spec [2] for, due to interest outside of
> > > >>> MongoDB. Basically, it adds support for date, ObjectId [3],
Re: [Lift] Re: BSON support in lift-json
I personally think hybrid approaches make sense for certain designs even if they are a little odd. My thought originally was having a new member of the ADT which is not final which represents "extensions", e.g. JValue \_ JExtension \_ JDate where JExtensions could be ignored or passed through unchanged by most of the stuff, and only special readers/writers would know what to do with them. -Ross On Mar 8, 2010, at 12:45 PM, David Pollak wrote: > > > On Mon, Mar 8, 2010 at 12:50 AM, Joni Freeman wrote: > This is a tricky one. The problem with extending the AST is that the > AST is implemented as an algebraic data type. And by definition it is > not possible to extend such a type. > > Just throwing an idea out and it's likely a bad one. > > I ran into a similar issue with Box and chose to do a hybrid (aka > Frankenstein) ADT/OO paradigm. > > Perhaps it's possible to provide subclasses of certain ADT items (e.g., JDate > extends JInt) such that if you pattern match on a JInt, you get the millis as > long, but if you pattern match against JDate, you get a date extracted if > it's the JDate subclass. > > Once again, it's likely to be a bad idea as it caused a lot of angst in Box > and I'm not sure if the paradigm is one that's worth perpetuating. > > > One way to add BSON support is to create a new AST for it which > includes all extended literals. Then add a few core functions for that > ADT (map, etc.) and maybe a function which can encode BSON as JSON > (bvalue.toJson). Encoding BSON as JSON would give some features for > free, for instance toXml. Anyway, this approach would probably cause > some code duplication between lift-json and lift-bson. > > Converting the JSON AST to an object oriented design would be another > approach. Then adding new AST nodes would not be a problem. But that > would be a huge change to the lib. Probably too big at this phase. > > Since BSON is a superset of JSON we could refactor current lift-json > to be lift-bson and then implement lift-json on top of it. On a > cursory look this feels cleanest but there might be some performance > penalties for normal JSON processing due to conversions. > > To be honest, I'm not yet sure what would be the best approach. > > Cheers Joni > > On Mar 5, 10:08 pm, Ross Mellgren wrote: > > The JSON stuff is mostly just an AST and encoding/decoding from the JSON > > wire format is almost just an addon. Then, it would be a matter of adding > > AST objects for those new things. Could be a use for phantom types ;-) > > > > I'd be interested to hear Joni's view on how it might fit, since he's the > > most familiar. > > > > -Ross > > > > On Mar 5, 2010, at 1:26 PM, Tim Nelson wrote: > > > > > I definitely agree with keeping the BSON code separate or possibly > > > having a strict JSON mode. > > > > > Tim > > > > > On Fri, Mar 5, 2010 at 12:13 PM, Timothy Perrett > > > wrote: > > >> Probably a sub-ordinate module would be preferable... one that builds > > >> on the lift-json stuff and doesn't pollute the "normal" JSON usage. > > > > >> Joni, what are your thoughts? > > > > >> Cheers, Tim > > > > >> On 5 Mar 2010, at 17:59, Tim Nelson wrote: > > > > >>> I finally had the opportunity to look into the couchdb code and I must > > >>> say it is rather impressive. > > > > >>> I would like to utilize the code in JSONRecord.scala in scamongo [1]. > > >>> However, MongoDB uses a variation of JSON they call BSON, which they > > >>> actually just published a spec [2] for, due to interest outside of > > >>> MongoDB. Basically, it adds support for date, ObjectId [3], binary > > >>> data, regular expressions, and code (JavaScript) data types. > > > > >>> My question is, what would it take to add support to lift-json for > > >>> these other data types? Is this even feasible? > > > > >>> Thanks, > > >>> Tim > > > > >>> [1]http://github.com/eltimn/scamongo > > >>> [2]http://bsonspec.org/ > > >>> [3]http://www.mongodb.org/display/DOCS/Object+IDs > > > > >>> -- > > >>> You received this message because you are subscribed to the Google > > >>> Groups "Lift" group. > > >>> To post to this group, send email to lift...@googlegroups.com. > > >>> To unsubscribe from this group, send email to > > >>> liftweb+unsubscr...@googlegroups.com. > > >>> For more options, visit this group > > >>> athttp://groups.google.com/group/liftweb?hl=en. > > > > >> -- > > >> You received this message because you are subscribed to the Google > > >> Groups "Lift" group. > > >> To post to this group, send email to lift...@googlegroups.com. > > >> To unsubscribe from this group, send email to > > >> liftweb+unsubscr...@googlegroups.com. > > >> For more options, visit this group > > >> athttp://groups.google.com/group/liftweb?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Lift" group. > > > To post to this group, send email to lift...@googlegroups.com. > > > To unsubscribe from this group, send e
Re: [Lift] Re: BSON support in lift-json
On Mon, Mar 8, 2010 at 12:50 AM, Joni Freeman wrote: > This is a tricky one. The problem with extending the AST is that the > AST is implemented as an algebraic data type. And by definition it is > not possible to extend such a type. > Just throwing an idea out and it's likely a bad one. I ran into a similar issue with Box and chose to do a hybrid (aka Frankenstein) ADT/OO paradigm. Perhaps it's possible to provide subclasses of certain ADT items (e.g., JDate extends JInt) such that if you pattern match on a JInt, you get the millis as long, but if you pattern match against JDate, you get a date extracted if it's the JDate subclass. Once again, it's likely to be a bad idea as it caused a lot of angst in Box and I'm not sure if the paradigm is one that's worth perpetuating. > > One way to add BSON support is to create a new AST for it which > includes all extended literals. Then add a few core functions for that > ADT (map, etc.) and maybe a function which can encode BSON as JSON > (bvalue.toJson). Encoding BSON as JSON would give some features for > free, for instance toXml. Anyway, this approach would probably cause > some code duplication between lift-json and lift-bson. > > Converting the JSON AST to an object oriented design would be another > approach. Then adding new AST nodes would not be a problem. But that > would be a huge change to the lib. Probably too big at this phase. > > Since BSON is a superset of JSON we could refactor current lift-json > to be lift-bson and then implement lift-json on top of it. On a > cursory look this feels cleanest but there might be some performance > penalties for normal JSON processing due to conversions. > > To be honest, I'm not yet sure what would be the best approach. > > Cheers Joni > > On Mar 5, 10:08 pm, Ross Mellgren wrote: > > The JSON stuff is mostly just an AST and encoding/decoding from the JSON > wire format is almost just an addon. Then, it would be a matter of adding > AST objects for those new things. Could be a use for phantom types ;-) > > > > I'd be interested to hear Joni's view on how it might fit, since he's the > most familiar. > > > > -Ross > > > > On Mar 5, 2010, at 1:26 PM, Tim Nelson wrote: > > > > > I definitely agree with keeping the BSON code separate or possibly > > > having a strict JSON mode. > > > > > Tim > > > > > On Fri, Mar 5, 2010 at 12:13 PM, Timothy Perrett > > > wrote: > > >> Probably a sub-ordinate module would be preferable... one that builds > on the lift-json stuff and doesn't pollute the "normal" JSON usage. > > > > >> Joni, what are your thoughts? > > > > >> Cheers, Tim > > > > >> On 5 Mar 2010, at 17:59, Tim Nelson wrote: > > > > >>> I finally had the opportunity to look into the couchdb code and I > must > > >>> say it is rather impressive. > > > > >>> I would like to utilize the code in JSONRecord.scala in scamongo [1]. > > >>> However, MongoDB uses a variation of JSON they call BSON, which they > > >>> actually just published a spec [2] for, due to interest outside of > > >>> MongoDB. Basically, it adds support for date, ObjectId [3], binary > > >>> data, regular expressions, and code (JavaScript) data types. > > > > >>> My question is, what would it take to add support to lift-json for > > >>> these other data types? Is this even feasible? > > > > >>> Thanks, > > >>> Tim > > > > >>> [1]http://github.com/eltimn/scamongo > > >>> [2]http://bsonspec.org/ > > >>> [3]http://www.mongodb.org/display/DOCS/Object+IDs > > > > >>> -- > > >>> You received this message because you are subscribed to the Google > Groups "Lift" group. > > >>> To post to this group, send email to lift...@googlegroups.com. > > >>> To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > > >>> For more options, visit this group athttp:// > groups.google.com/group/liftweb?hl=en. > > > > >> -- > > >> You received this message because you are subscribed to the Google > Groups "Lift" group. > > >> To post to this group, send email to lift...@googlegroups.com. > > >> To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > > >> For more options, visit this group athttp:// > groups.google.com/group/liftweb?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "Lift" group. > > > To post to this group, send email to lift...@googlegroups.com. > > > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > > > For more options, visit this group athttp:// > groups.google.com/group/liftweb?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > -- Lift, the simply functional web framework http://liftwe
[Lift] Re: BSON support in lift-json
This is a tricky one. The problem with extending the AST is that the AST is implemented as an algebraic data type. And by definition it is not possible to extend such a type. One way to add BSON support is to create a new AST for it which includes all extended literals. Then add a few core functions for that ADT (map, etc.) and maybe a function which can encode BSON as JSON (bvalue.toJson). Encoding BSON as JSON would give some features for free, for instance toXml. Anyway, this approach would probably cause some code duplication between lift-json and lift-bson. Converting the JSON AST to an object oriented design would be another approach. Then adding new AST nodes would not be a problem. But that would be a huge change to the lib. Probably too big at this phase. Since BSON is a superset of JSON we could refactor current lift-json to be lift-bson and then implement lift-json on top of it. On a cursory look this feels cleanest but there might be some performance penalties for normal JSON processing due to conversions. To be honest, I'm not yet sure what would be the best approach. Cheers Joni On Mar 5, 10:08 pm, Ross Mellgren wrote: > The JSON stuff is mostly just an AST and encoding/decoding from the JSON wire > format is almost just an addon. Then, it would be a matter of adding AST > objects for those new things. Could be a use for phantom types ;-) > > I'd be interested to hear Joni's view on how it might fit, since he's the > most familiar. > > -Ross > > On Mar 5, 2010, at 1:26 PM, Tim Nelson wrote: > > > I definitely agree with keeping the BSON code separate or possibly > > having a strict JSON mode. > > > Tim > > > On Fri, Mar 5, 2010 at 12:13 PM, Timothy Perrett > > wrote: > >> Probably a sub-ordinate module would be preferable... one that builds on > >> the lift-json stuff and doesn't pollute the "normal" JSON usage. > > >> Joni, what are your thoughts? > > >> Cheers, Tim > > >> On 5 Mar 2010, at 17:59, Tim Nelson wrote: > > >>> I finally had the opportunity to look into the couchdb code and I must > >>> say it is rather impressive. > > >>> I would like to utilize the code in JSONRecord.scala in scamongo [1]. > >>> However, MongoDB uses a variation of JSON they call BSON, which they > >>> actually just published a spec [2] for, due to interest outside of > >>> MongoDB. Basically, it adds support for date, ObjectId [3], binary > >>> data, regular expressions, and code (JavaScript) data types. > > >>> My question is, what would it take to add support to lift-json for > >>> these other data types? Is this even feasible? > > >>> Thanks, > >>> Tim > > >>> [1]http://github.com/eltimn/scamongo > >>> [2]http://bsonspec.org/ > >>> [3]http://www.mongodb.org/display/DOCS/Object+IDs > > >>> -- > >>> You received this message because you are subscribed to the Google Groups > >>> "Lift" group. > >>> To post to this group, send email to lift...@googlegroups.com. > >>> To unsubscribe from this group, send email to > >>> liftweb+unsubscr...@googlegroups.com. > >>> For more options, visit this group > >>> athttp://groups.google.com/group/liftweb?hl=en. > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Lift" group. > >> To post to this group, send email to lift...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> liftweb+unsubscr...@googlegroups.com. > >> For more options, visit this group > >> athttp://groups.google.com/group/liftweb?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Lift" group. > > To post to this group, send email to lift...@googlegroups.com. > > To unsubscribe from this group, send email to > > liftweb+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/liftweb?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
