Now that I reread it I was completely off. \_('-')_/ I am sorry.
On Tue, Aug 16, 2016 at 8:35 AM, David Daniel <[email protected]>
wrote:
> I think I understand better now. Thank you for the explanation as I am
> still trying to get an understanding of DTO's and the converter and how
> they can be used in my application.
>
> David Daniel
>
> On Tue, Aug 16, 2016 at 8:29 AM, David Leangen <[email protected]> wrote:
>
>>
>> Hi David D.,
>>
>> Thanks for your input.
>>
>> What you write makes sense, but I don’t think it really applies to what I
>> am trying to suggest.
>>
>> All I am asking for is an introspective view on the DTO data tree
>> structure, nothing more. Since this is being parsed anyway, this code
>> provides the perfect opportunity to open up this view of the data. Anything
>> beyond that would be out of scope, but would be able to interact nicely and
>> easily with the data tree exposed by this new interface.
>>
>> This is in no way a persistence structure, or any kind of database,
>> especially not relational. Having a parsable, manipulatable view of the
>> data structure would, however, allow much easier processing of the data to
>> map to a database, whether it be relational, key/value, graph, etc.
>>
>> Aa an example, a few years back, I had to write exactly this type of
>> functionality in order to do a deep storage of a data structure in Berkeley
>> DB. The top-level objects “shared” other objects, so it was decided that it
>> would be wasteful to have millions of duplicate copies of those mid-level
>> objects. When storing and retrieving data, I had to introspect the data
>> structure and dejoin the mid-level objects, store them in their own
>> key/value store, then join them again when the data was retrieved. Document
>> stores and graph DBs also need instructions as to how to store the data.
>> Not just RDBs.
>>
>> I agree with what you write, that the relationships are too complex to
>> deduce from the DTO structure. Objects that are embedded in the data do not
>> define the relationships; there is not enough information for that. That
>> type of information would be beyond the scope of what I am proposing.
>> However, the annotations (for instance) that you would use to mark the
>> relationships would work very well with this data structure. With that
>> access to the data tree, I could easily kick off a process against the tree
>> that looks for these annotations so that I can interface with my
>> persistence layer.
>>
>>
>> That is for persistence. If you want to work with, for instance, Lucene,
>> for indexing the data, I think that this type of introspective access to
>> the data tree would also be very useful.
>>
>> All I am suggesting is that we provide access to the tree, which would
>> _enable_ many types of processing against it, not that we actually include
>> anything new.
>>
>> I do understand that it is a step away from a simple “Converter”, but the
>> parsing is essentially the same. Since the hard work is already being done,
>> why not take advantage of it here? Even if this tree view ends up being a
>> completely different service, the same code base could easily serve the two.
>>
>>
>> Cheers,
>> =David
>>
>>
>> > On Aug 16, 2016, at 8:35 PM, David Daniel <[email protected]>
>> wrote:
>> >
>> > In my opinion with the scenarios you described you are no longer
>> describing
>> > the data but a persistence structure. Indexes are about duplicating the
>> > data so it can be retrieved faster given a storage structure and some
>> idea
>> > of that structure is needed to create them. If the persisting side
>> could
>> > determine the structure solely off the presence of the key then I think
>> it
>> > is not bad to have in but it is impossible for a key to cover every
>> area.
>> > I think I just don't like the idea of tying the structure into the
>> > definition without value types in java. I am also wary about pulling in
>> > object relationships because that is getting into type/category theory
>> > which java does not have a standard way of doing. With relational data
>> if
>> > you have a key then you can just have a limited number relationships by
>> > default like 1 to 1 or 1 to many or many to many but I don't like the
>> idea
>> > of having DTO's as relational defined objects while most things are
>> moving
>> > away from that. I also think you cannot accurately define most
>> > relationships in a small structure. How do you define multiple
>> elements as
>> > sum types or product types. How do you show one DTO has an inherited
>> > relationship vs a compositional one. I think there are many functional
>> > languages that run on the jvm like scala, clojure, kotlin and hazkel
>> that
>> > do a good job of allowing programmers to create their own relationships
>> > that can be defined on data and I dont think the spec should be creating
>> > another. I think having DTO's as they are and then the programmer can
>> use
>> > the converter to convert to other objects that have more meaningful
>> > capabilities is enough for the standard. I think the things discussed
>> > above should be in a separate library that adds functionality onto DTO
>> and
>> > Converter similar to cats in scala.
>> >
>> > My 2 cents,
>> > David Daniel
>> >
>> > On Tue, Aug 16, 2016 at 2:00 AM, David Leangen <[email protected]>
>> wrote:
>> >
>> >>
>> >> Hi!
>> >>
>> >> The Converter service does a lot of introspection and parsing of the
>> DTO
>> >> data structure. In many cases, a DTO is a very simple object structure.
>> >> However, it can also be a very complex structure, too.
>> >>
>> >> According to my understanding of the objectives of the Converter, one
>> >> important goal is to be able to persist data. The idea is that the DTO
>> >> describes the data, the whole data, and nothing but the data, so help
>> me
>> >> Uncle. Thus, it is the ideal way to ship the state of the system off to
>> >> PersistenceLand.
>> >>
>> >> I can buy into this vision.
>> >>
>> >> If we do buy into this vision, then we may be missing out on a few
>> great
>> >> opportunities here. When data gets persisted, we often need to
>> understand
>> >> the relationships between the embedded objects. Or, we may want to be
>> able
>> >> to create an index on the data. These are a few of the reasons why we
>> would
>> >> want to have some kind of x-ray vision on the data structure. Since we
>> >> already go through all the trouble of parsing the data structure in
>> order
>> >> to convert it, and since this is ~95% of the work, it would be really
>> nice
>> >> to provide access to this information in order to easily link in
>> services
>> >> that require this intimate knowledge. Otherwise, all the parsing would
>> have
>> >> to be done over and over again for each service.
>> >>
>> >> I believe that it would only take a few methods to be able to leverage
>> all
>> >> the parsing work done by the Converter. I can think of:
>> >>
>> >> DataTree Converter.toTree(DTO dto); // DataTre gives a tree view of
>> the
>> >> structure
>> >> Object tree.valueAt(DTO dto, String path); // Dot-separated path value
>> >> within the tree structure
>> >> void tree.set(DTO dto, String path, Object value); // Set the value at
>> >> the given location in the tree structure
>> >> void process(DTO dto, Consumer<?> function); // Visit each node for
>> some
>> >> kind of processing
>> >>
>> >> Those are just some examples. Perhaps a new API would be necessary,
>> but my
>> >> main point here is that since we are going through all this work of
>> >> implementing a parser, this is the IDEAL time to create this type of
>> view
>> >> on the data.
>> >>
>> >>
>> >> wdyt?
>> >>
>> >> I can explain further the idea if you like. For now, I just wanted to
>> get
>> >> a quick feedback to see if there is any openness to this kind of thing.
>> >>
>> >>
>> >> =David
>> >>
>> >>
>> >>
>>
>>
>
