:-) Cheers, =David
> On Aug 16, 2016, at 9:41 PM, David Daniel <[email protected]> wrote: > > Now that I reread it I was completely off. \_('-')_/ I am sorry. > > On Tue, Aug 16, 2016 at 8:35 AM, David Daniel <[email protected]> > wrote: > >> I think I understand better now. Thank you for the explanation as I am >> still trying to get an understanding of DTO's and the converter and how >> they can be used in my application. >> >> David Daniel >> >> On Tue, Aug 16, 2016 at 8:29 AM, David Leangen <[email protected]> wrote: >> >>> >>> Hi David D., >>> >>> Thanks for your input. >>> >>> What you write makes sense, but I don’t think it really applies to what I >>> am trying to suggest. >>> >>> All I am asking for is an introspective view on the DTO data tree >>> structure, nothing more. Since this is being parsed anyway, this code >>> provides the perfect opportunity to open up this view of the data. Anything >>> beyond that would be out of scope, but would be able to interact nicely and >>> easily with the data tree exposed by this new interface. >>> >>> This is in no way a persistence structure, or any kind of database, >>> especially not relational. Having a parsable, manipulatable view of the >>> data structure would, however, allow much easier processing of the data to >>> map to a database, whether it be relational, key/value, graph, etc. >>> >>> Aa an example, a few years back, I had to write exactly this type of >>> functionality in order to do a deep storage of a data structure in Berkeley >>> DB. The top-level objects “shared” other objects, so it was decided that it >>> would be wasteful to have millions of duplicate copies of those mid-level >>> objects. When storing and retrieving data, I had to introspect the data >>> structure and dejoin the mid-level objects, store them in their own >>> key/value store, then join them again when the data was retrieved. Document >>> stores and graph DBs also need instructions as to how to store the data. >>> Not just RDBs. >>> >>> I agree with what you write, that the relationships are too complex to >>> deduce from the DTO structure. Objects that are embedded in the data do not >>> define the relationships; there is not enough information for that. That >>> type of information would be beyond the scope of what I am proposing. >>> However, the annotations (for instance) that you would use to mark the >>> relationships would work very well with this data structure. With that >>> access to the data tree, I could easily kick off a process against the tree >>> that looks for these annotations so that I can interface with my >>> persistence layer. >>> >>> >>> That is for persistence. If you want to work with, for instance, Lucene, >>> for indexing the data, I think that this type of introspective access to >>> the data tree would also be very useful. >>> >>> All I am suggesting is that we provide access to the tree, which would >>> _enable_ many types of processing against it, not that we actually include >>> anything new. >>> >>> I do understand that it is a step away from a simple “Converter”, but the >>> parsing is essentially the same. Since the hard work is already being done, >>> why not take advantage of it here? Even if this tree view ends up being a >>> completely different service, the same code base could easily serve the two. >>> >>> >>> Cheers, >>> =David >>> >>> >>>> On Aug 16, 2016, at 8:35 PM, David Daniel <[email protected]> >>> wrote: >>>> >>>> In my opinion with the scenarios you described you are no longer >>> describing >>>> the data but a persistence structure. Indexes are about duplicating the >>>> data so it can be retrieved faster given a storage structure and some >>> idea >>>> of that structure is needed to create them. If the persisting side >>> could >>>> determine the structure solely off the presence of the key then I think >>> it >>>> is not bad to have in but it is impossible for a key to cover every >>> area. >>>> I think I just don't like the idea of tying the structure into the >>>> definition without value types in java. I am also wary about pulling in >>>> object relationships because that is getting into type/category theory >>>> which java does not have a standard way of doing. With relational data >>> if >>>> you have a key then you can just have a limited number relationships by >>>> default like 1 to 1 or 1 to many or many to many but I don't like the >>> idea >>>> of having DTO's as relational defined objects while most things are >>> moving >>>> away from that. I also think you cannot accurately define most >>>> relationships in a small structure. How do you define multiple >>> elements as >>>> sum types or product types. How do you show one DTO has an inherited >>>> relationship vs a compositional one. I think there are many functional >>>> languages that run on the jvm like scala, clojure, kotlin and hazkel >>> that >>>> do a good job of allowing programmers to create their own relationships >>>> that can be defined on data and I dont think the spec should be creating >>>> another. I think having DTO's as they are and then the programmer can >>> use >>>> the converter to convert to other objects that have more meaningful >>>> capabilities is enough for the standard. I think the things discussed >>>> above should be in a separate library that adds functionality onto DTO >>> and >>>> Converter similar to cats in scala. >>>> >>>> My 2 cents, >>>> David Daniel >>>> >>>> On Tue, Aug 16, 2016 at 2:00 AM, David Leangen <[email protected]> >>> wrote: >>>> >>>>> >>>>> Hi! >>>>> >>>>> The Converter service does a lot of introspection and parsing of the >>> DTO >>>>> data structure. In many cases, a DTO is a very simple object structure. >>>>> However, it can also be a very complex structure, too. >>>>> >>>>> According to my understanding of the objectives of the Converter, one >>>>> important goal is to be able to persist data. The idea is that the DTO >>>>> describes the data, the whole data, and nothing but the data, so help >>> me >>>>> Uncle. Thus, it is the ideal way to ship the state of the system off to >>>>> PersistenceLand. >>>>> >>>>> I can buy into this vision. >>>>> >>>>> If we do buy into this vision, then we may be missing out on a few >>> great >>>>> opportunities here. When data gets persisted, we often need to >>> understand >>>>> the relationships between the embedded objects. Or, we may want to be >>> able >>>>> to create an index on the data. These are a few of the reasons why we >>> would >>>>> want to have some kind of x-ray vision on the data structure. Since we >>>>> already go through all the trouble of parsing the data structure in >>> order >>>>> to convert it, and since this is ~95% of the work, it would be really >>> nice >>>>> to provide access to this information in order to easily link in >>> services >>>>> that require this intimate knowledge. Otherwise, all the parsing would >>> have >>>>> to be done over and over again for each service. >>>>> >>>>> I believe that it would only take a few methods to be able to leverage >>> all >>>>> the parsing work done by the Converter. I can think of: >>>>> >>>>> DataTree Converter.toTree(DTO dto); // DataTre gives a tree view of >>> the >>>>> structure >>>>> Object tree.valueAt(DTO dto, String path); // Dot-separated path value >>>>> within the tree structure >>>>> void tree.set(DTO dto, String path, Object value); // Set the value at >>>>> the given location in the tree structure >>>>> void process(DTO dto, Consumer<?> function); // Visit each node for >>> some >>>>> kind of processing >>>>> >>>>> Those are just some examples. Perhaps a new API would be necessary, >>> but my >>>>> main point here is that since we are going through all this work of >>>>> implementing a parser, this is the IDEAL time to create this type of >>> view >>>>> on the data. >>>>> >>>>> >>>>> wdyt? >>>>> >>>>> I can explain further the idea if you like. For now, I just wanted to >>> get >>>>> a quick feedback to see if there is any openness to this kind of thing. >>>>> >>>>> >>>>> =David >>>>> >>>>> >>>>> >>> >>> >>
