Chris, TypeHandling sounds like it could fix the issue, but I can't really say until I find out how it works (sounds like there is little documentation on the subject).
One major concern of mine is: Even if TypeHandling enforce the element to be formatted a certain way, does it allow me to access to the child fields of a complex element (no longer a simple type) from within the ValidationLayer? The cost of even reading in these types into PCS is paid for by the fact that, with my current solution, I can access that data and use that inside PCS to develop tools and other things. If we switch to TypeHandling, and I lose that ability, then the solution is no better off that the unparsed tag solution Brian suggested (which is indeed far quicker to implement). I am open to any new ideas, but I might need some more information. -Michael On May 25, 2011, at 11:45 PM, Mattmann, Chris A (388J) wrote: > Hey Guys, > > This made me think of the following issue I filed: > > https://issues.apache.org/jira/browse/OODT-147 > > I think we can deal with a lot of this there. What do you think of my > proposal, Michael? Brian, I know what you think :-) Because you and I talked > about it on the issue. > > Cheers, > Chris > > On May 24, 2011, at 7:08 AM, holenoter wrote: > >> >> hey michael, >> >> if you just need to store this information in the elements.xml file then >> just add new tags . . . you don't need to change any code: >> >> <element id="some-id" name="some-name"> >> (current elements here) >> .... >> <!-- unparsed tags --> >> <maxSize>10</maxSize> >> </element> >> >> Also . . . the dublin core stuff is fine for peate . . . if you want to >> create a patch for the trunk at apache that is a different story, but peate >> will never need the dublin core feature, so the patch as it currently stands >> is perfectly okay >> >> -brian >> >> On May 24, 2011, at 07:18 AM, "Starch, Michael D (388L)" >> <[email protected]> wrote: >> >>> Brian, >>> >>> PEATE is busy as always, but doing well. >>> >>> We need to be able to store this information somewhere so we can just tell >>> the DBA to use these values. As it stands now we do a lot of >>> back-and-forth to get the right values to support the database, and we need >>> to stream-line the process by having this information pre-selected by the >>> project. Due to the data's similarity to the other data in the elements >>> file we selected this as the place to put it and Chris agreed. >>> >>> Does this answer your questions? >>> >>> -Michael >>> >>> >>> >>> >>> >>> >>> >>> On May 24, 2011, at 1:47 AM, holenoter wrote: >>> >>>> hey michael, >>>> >>>> hope everything is going well for peate . . . >>>> >>>> yup dublin core element was used because of the time crunch before i left >>>> . . . being it hasn't been used up to this point anywhere, good odds it >>>> probably won't be used anytime in the "peate future" . . . however, dublin >>>> core is the wrong place to put this feature because dublin core is used to >>>> describe the element itself, not its value's type . . . pretty much >>>> chris's recommendation is the way to go, however if you are going to be >>>> working towards a patch to the trunk, instead of extending Element as >>>> chris recommended just add a "type" member variable to the existing >>>> Element class and then modify the XMLValidationLayer and have it read a >>>> <type> tag into that "type" member variable in Element . . . i think there >>>> is a strong case for having the Element understand to some degree its >>>> value's type . . . not sure why you need more information in your >>>> elements.xml file than i already added? . . . the database should be doing >>>> this validation (otherwise your code will have to become double, float, >>>> long, int, etc... aware) . . . the current ColumnBasedDataSourceCatalog >>>> handles all these types already via Oracle (just set any double, float, >>>> long, int, etc... to a <dcElement>NUMBER</dcElement> in the elements.xml >>>> file. . .you have to set Oracle double precision anyway even if you where >>>> to treat it as a double in the filemgr) . . . if you want to just keep the >>>> information in one spot (i.e. elements.xml) than just add unused tags to >>>> the elements.xml file with the information you want (XMLValidationLayer >>>> will just ignore these tags) . . . for example you can add a <maxSize> tag >>>> to elements.xml without affecting the filemgr. . .as far as validating max >>>> size and the like, this really should be happening in your metadata >>>> extractors. >>>> >>>> -brian >>>> >>>> On May 24, 2011, at 12:59 AM, "Mattmann, Chris A (388J)" >>>> <[email protected]> wrote: >>>> >>>>> Hi Michael, >>>>> >>>>> Thanks for your email! >>>>> >>>>> You can find the OODT Apache archives here: >>>>> >>>>> http://mail-archives.apache.org/mod_mbox/oodt-dev/ >>>>> >>>>> As for standards, best practices are certainly part of them but we do >>>>> have a few standards that we try and stick to including Dublin Core, >>>>> ISO-11179, RDF, etc., when we are defining metadata elements. For your >>>>> purposes in elements.xml in CAS, the biggest approach is to use unique, >>>>> namespaced identifiers for element IDs, product type IDs, etc. >>>>> >>>>> HTH! >>>>> >>>>> Cheers, >>>>> Chris >>>>> >>>>> On May 23, 2011, at 1:14 PM, Starch, Michael D (388L) wrote: >>>>> >>>>>> Chris, >>>>>> >>>>>> Is there some sort of coding standards document that I should be abiding >>>>>> by, or does oodt just follow standard java "best practices"? This is the >>>>>> kind of information that seems like it would be easier for me to look up >>>>>> rather than ask on the developer list. >>>>>> >>>>>> Finally, how can I access the archived dev-list mailings? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Michael >>>>>> >>>>>> >>>>>> On May 19, 2011, at 11:06 PM, Mattmann, Chris A (388J) wrote: >>>>>> >>>>>>> Hi Michael, >>>>>>> >>>>>>> I'd be happy to reply below, but yes I'd suggest initially CC'ing >>>>>>> [email protected]. Giving this information is great, but I prefer the >>>>>>> community to benefit from these types of discussions (and it means that >>>>>>> I or you or Albert or Cecilia can point others to this question if it >>>>>>> is asked again rather than typing the same answer again or fwd'ing >>>>>>> something private). Also no worries about researching it -- you can ask >>>>>>> questions on that list -- researched or not. We welcome feedback from >>>>>>> the community. >>>>>>> >>>>>>> Answers inline below: >>>>>>> >>>>>>> On May 19, 2011, at 2:44 PM, Starch, Michael D (388L) wrote: >>>>>>> >>>>>>>> Chris, >>>>>>>> >>>>>>>> In order to create tables in the new database architecture that Brian >>>>>>>> Foster set up for use, we need the ability to specify some information >>>>>>>> beyond that which is stored in elements.xml. We wish to store this >>>>>>>> information as part of PCS, rather than continually haggling back and >>>>>>>> forth with our DBA in order to get this set in an ad-hoc manner. >>>>>>>> >>>>>>>> The data we need to store is: >>>>>>>> >>>>>>>> -Is it a vector type? >>>>>>>> -Data-type >>>>>>>> -Max data size >>>>>>>> and potentially a few more items. >>>>>>> >>>>>>> Gotcha. >>>>>>> >>>>>>>> >>>>>>>> As it stands now, Brian has used the DCElement tag in elements.xml >>>>>>>> file in order to store some of this information but not all of it. >>>>>>>> Thus it seems natural that the rest of this information gets stored >>>>>>>> along side it. >>>>>>>> >>>>>>>> There seem to be three ways to accomplish this: >>>>>>>> >>>>>>>> 1. Add more information to the DCElement tag, >>>>>>>> 2. Add additional tags to the element file containing this information >>>>>>>> 3. Create a sperate file to store this information (Not part of PCS) >>>>>>> >>>>>>> I've got a 4th option. >>>>>>> >>>>>>> 4. Write a new ValidationLayer implementation, that extends >>>>>>> XMLValidationLayer, but adds your desired information to elements.xml. >>>>>>> For that extra information, I'd favor your proposed option #2 -- I >>>>>>> think it's cleaner. >>>>>>> >>>>>>>> >>>>>>>> As head of oodt, we were looking for you input as to which option to >>>>>>>> choose, or if you wish this question to be brought to [email protected], >>>>>>>> where can I research this information before asking the question on >>>>>>>> that list? >>>>>>>> >>>>>>>> Below is a brief analysis of each option, from our perspective. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Michael Starch >>>>>>>> >>>>>>>> 1. Add more information to the DCElements tag. >>>>>>>> -I believe Brain said this tag was unused, so he "borrowed" it to suit >>>>>>>> this purpose. >>>>>>>> -Adding more information to the tag would "hijack" this tag to a >>>>>>>> further extent. >>>>>>>> -The changes could be made locally to our ColumnBasedDataSourceCatalog >>>>>>>> in the manner Brian already used. >>>>>>>> -If the DCElement is used ouside of the new use Brian invented, this >>>>>>>> change could affect the intended (original) use >>>>>>> >>>>>>> I don't like this one (nor do I think others will) since it hijacks >>>>>>> dcElement (which is supposed to be a mapping to a Dublin Core element >>>>>>> name, rather than used for other information). >>>>>>> >>>>>>>> >>>>>>>> 2. Add more tags to elements.xml >>>>>>>> -Most elegant solution, as each data field has a tag specified to hold >>>>>>>> exactly it >>>>>>>> -No need to use tags which were originally intended for some other >>>>>>>> purpose >>>>>>>> -It is an architectural change, changing the format of the >>>>>>>> elements.xml file >>>>>>>> -Must be done carefully to prevent us from deviating from apache-oodt >>>>>>>> when it is not necessary to do so. >>>>>>> >>>>>>> +1 for this option. >>>>>>> >>>>>>>> >>>>>>>> 3. A separate file >>>>>>>> -Creates data/configuration duplication, as some of these fields must >>>>>>>> stay (in some form) in elements.xml >>>>>>>> -Separates similar pieces of data >>>>>>>> -Doesn't alter PCS as it exists now >>>>>>> >>>>>>> This is fine too, but I prefer #2. >>>>>>> >>>>>>>> >>>>>>>> We favor number 1 or number 2 as it keeps similar data together, and >>>>>>>> feels like it is the most logical place to store this information. >>>>>>>> >>>>>>>> I personally favor number 2, as it does not use tags for unintended >>>>>>>> purposes, but I do not wish to make a change that affects our >>>>>>>> compatibility with apache code. Hence your input. >>>>>>>> >>>>>>> >>>>>>> +1 to your preference. I think the way to accomplish it is to write a >>>>>>> YourExtendedValidationLayer extends XMLValidationLayer and that >>>>>>> provides a customized: >>>>>>> 1. YourElementClass extends Element >>>>>>> - provides extra props (getters+setters) that you want to leverage >>>>>>> 2. implementations of the SerDe for your specialized elements.xml that >>>>>>> includes those extra tags >>>>>>> 3. provides the ability to access this ValidationLayer information and >>>>>>> Element information in ColumnBasedDataSourceCatalog. >>>>>>> >>>>>>> BTW, I think it would be great to bake up patches for Apache OODT for >>>>>>> what you are working on, including ColumnBasedDataSourceCatalog >>>>>>> (already there but in a branch, would be nice to forward port to trunk, >>>>>>> and your proposed ValidationLayer and Element extensions). Thoughts? Do >>>>>>> you have the time? I think the community at Apache would sincerely >>>>>>> appreciate the effort. >>>>>>> >>>>>>> Cheers, >>>>>>> Chris >>>>>>> >>>>>>> P.S. Would you be OK with me forwarding this thread to [email protected]? I >>>>>>> think there's some good info here that I'd hate to be lost in the >>>>>>> ether... >>>>>>> >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>> Chris Mattmann, Ph.D. >>>>>>> Senior Computer Scientist >>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>>>>>> Office: 171-266B, Mailstop: 171-246 >>>>>>> Email: [email protected] >>>>>>> WWW: http://sunset.usc.edu/~mattmann/ >>>>>>> Phone: +1 (818) 354-8810 >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>> Adjunct Assistant Professor, Computer Science Department >>>>>>> University of Southern California, Los Angeles, CA 90089 USA >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>> >>>>>> >>>>> >>>>> >>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>> Chris Mattmann, Ph.D. >>>>> Senior Computer Scientist >>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>>>> Office: 171-266B, Mailstop: 171-246 >>>>> Email: [email protected] >>>>> WWW: http://sunset.usc.edu/~mattmann/ >>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>> Adjunct Assistant Professor, Computer Science Department >>>>> University of Southern California, Los Angeles, CA 90089 USA >>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>> >>> > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >
