Jeff, Well, thanks for trying to be the cranky old man instead of me. But just won't do. Now I have to rant.
My skepticism regarding using object fields for the purposes should be pretty obvious to anyone that's paid attention. Regarding the Summit demo...I watched the video and thought that it was very well presented and that it took a ton of work to do. I also think that Olivier is a very nice person personally and professionally. It was a pity that there was effectively no Q & A. Two points from the presentation: 1) The "optimization" from the object field depended on knowing the *exact* question(s) that you wanted answered in advance. Unanticipated question? Then the object search might end up being sequential. 2) The overall thrust of the demo seemed to be to show that object fields represent some kind of a sweet spot between a one table, one field database and a fully normalized database. The claim being that full normalization is slower and harder to do. Hmmm. Didn't make sense. I downloaded the demo, looked at the relational joining code, rewrote it to something more sensible....and the so-called speed "optimization" from object fields went away. I only spent two minutes on the code, so perhaps I did something horribly wrong? Could be, doubt it. I've been asking about real-world use of object fields for quite some time and also pushing questions and ideas towards 4D. I still don't get it. If you're thinking about using object fields (think "subtable without a fixed schema") ask yourself, "why?" What is it buying you? A lot of uses I've seen are to solve problems that don't exist in a normalized table, but do exist in a denormalized table that uses full fields. So, a lot of it sounds to me like taking badly designed tables and redoing them into badly design tables. My hope for object fields was not to use them for user-facing, normalized tables but to use them for a couple of other classes of table: * Programmer data where you're storing what amount to manifests or parameter blocks. So, job definitions that you pass around for execution, logging data for forwarding to the outside world, preferences and configuration documents of various sorts. * Pre-built JSON blocks to pass to the outside world. In this case, 4D is part of a pipeline where data is summarized/calculated and stored ready for export to some other system (like a visualization platform) or ready to be pulled by some other system (like a JavaScript that runs some kind of rendering.) 4D told me that this is "wrong" because if I have a fixed schema, I should use a full table. Okay, so 4D is *not* trying to build a NoSQL/doc store system. Because if they were, then what I've described is about as normal a use as you can come up with for those systems. * Fine, object fields aren't meant for storing JSON blocks that have a regular format. Well, that's probably just an opinion of some (important) people at 4D. In practice, they're just not efficient for it as the object takes almost exactly the amount of space required for the JSON serialization in UTF16...which takes a ton of space, often for no purpose. * So what are object fields for? From what 4D said in response to me, it sounds like they're for when you have data that has an irregular format. Hmmm. I don't have a lot of that apart from rendered JSON, logging data (when there are a mixture of logs), or internal messages. * Whatever problems you had with subtables, expect comparable problems with object fields. * I built a 1.75 million row table with an object field and did some experiments. I couldn't find any speed advantage to using an object query versus placing values in a field and querying there. Then again, I could also index the object field and get the same query there, which was cool. That seems promising. Then again, I have to use a different query command to do this. I couldn't find any advantage to the object field per se, so why not use a regular field/ Can't come up with a reason. I mean, I would be willing to use a different set of commands and/or operators for some benefit (proximity/fuzzy text searches as an obvious example, or a big speed boost)....but for no advantage? Why make the code harder. If 4D offered some compression and/or collation options on object fields, I'd use them for pipelining JSON. As it stands, they don't and seem unresponsive to the suggestion. I can absolutely see using object fields for internal messaging and prefs, as mentioned. Object fields for user data? While there are doubtlessly cases where this is tempting, only a few of those are likely to make sense. There are a few situations where the relational model just pinches your toes. (When you have "kind of" relations with a small number of categories.) They happen in any big structure, but they should be uncommon. I get a bit of a tight neck when I run into one...They're an edge case. Real, but not standard. Using object fields for the wholesale replacement of normal fields? Frankly, that's just a mistake. Using object fields for 'optimization'....I haven't seen it. I also haven't looked too hard, so I'm entirely ready to learn that I'm wrong in some cases here. But they do not optimize for space and they don't seem to optimize for speed either. So what do they optimize for? Not saying that they need to, just saying that I haven't seen that they do. I guess the "index" on an object field might be more space efficient than b-trees on a bunch of individual fields? No idea. 4D refuses to say literally *anything* on these subjects, so who knows? Hey don't get me wrong, it's great that 4D is trying to innovate. But it's entirely unclear to me why object fields (as implemented) were worth so much of their development time when there are so many other, basic bugs and feature requests needing attention. I'm really puzzled what problem they were solving and for whom. Seriously people, 4D's hostility to the relational model has been around since longer than I have, and it makes no sense. The beauty of normalized data is that you can easily answer *unanticipated questions.* Also, you can store your information in a way that's easy to keep in order. The problem with subtables wasn't just the implementation of the idea, it was the idea itself. Object fields for relational data? Much the same. ********************************************************************** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:[email protected] **********************************************************************
