That's pretty cool. Sorry but I was not taking into account the powerful queries that you might bring into the equation with this stuff. It all makes sense now.
It would definitely be a big feature that might be divided into smaller tasks. If we go ahead with this stuff I really would like to help with the implementation. Kind regards, Alberto 2015-07-15 12:41 GMT+02:00 Kasper Sørensen <[email protected]>: > That's absolutely true. It's not that I want to stop discovering what we > can, but I was more thinking of also adding a mechanism to plug your own > metadata. I guess it's pretty rare that a database itself offers a "domain > oriented" metadata system where I could tell it that "this field is a zip > code, and together with field X, Y and Z it forms a single address". > > While from a querying perspective what I would love to archieve is that I > could express something like the following: > > "Query all the addresses in the database". > "Do a SUM, AVG, MAX and MIN on all the ordinal-scale numbers in the > database". > > It will also help a lot in generating templates for data integration. If I > am trying to move data from one table to another then I can probably do a > lot of automatic mapping based on the metadata. Same goes for reporting I > guess and stuff like that. > > Best regards, > Kasper > > 2015-07-15 12:06 GMT+02:00 Alberto Rodriguez <[email protected]>: > > > Ok, so you are not thinking of discovering more metadata "on-the-fly", > your > > approach is statically define metadata for the datasource and load and > > mix-in it with the existing metadata right? > > > > IMHO with this approach we will add a strong dependency between the data > > itself and the "external" metadata, correct me if I'm wrong or not fully > > understand your proposal but if the datastore changes (one column is > > deleted or a new column is added) the metadata will get obsolete. > > > > Kind regards, > > > > Alberto > > > > 2015-07-13 19:26 GMT+02:00 Kasper Sørensen < > [email protected] > > >: > > > > > I was thinking of having something like pluggable annotations or > features > > > that could be added to tables, columns or groups of columns. Maybe also > > to > > > other entities. But since a lot of this is not available as a thing > that > > > can be explored in the datastore itself I guess it would need to be > > stored > > > externally. > > > > > > Examples of features that I could imagine: > > > > > > Data type features: NominalScale, OrdinalScale, ... > > > Data conversion features: IntegerAsString, DateAsString, > TimestampAsLong > > > ... > > > Domain features: FirstName, LastName, AddressLine, AddressCity, > > > AddressCountry, DateYear, DateMonth ... > > > And groupings of columns also in domain like fatures: Name (composed of > > > e.g. first and last name), Address (composed of multiple address > fields), > > > Date (composed of year, month etc.) > > > > > > I would like to store them so that I can save and load them, saving the > > > developer for the work of restoring all the metadata again and again. > Is > > > that not sensible? > > > > > > Best regards, > > > Kasper > > > > > > > > > 2015-07-13 9:55 GMT+02:00 Alberto Rodriguez <[email protected]>: > > > > > > > Hi all, > > > > > > > > We are also facing similar issues so I completely agree with this > > > feature. > > > > In fact, we added recently in our service layer a new field for our > > > > metadata called "format", we needed this field to specify different > > > format > > > > types for the dates returned by our datastores. > > > > > > > > However, I'm not really sure how to implement this feature... I guess > > we > > > > should keep getting the metadata "core" from our datasources but what > > > about > > > > the new metadata??: > > > > > > > > - How to fill it out? Will the integrator of MM provide functions > to > > > > define when an element is going to be "x" and when is going to be > "y"? > > > > (thinking here of providing lambda functions) > > > > - Do we really need a metadata store? > > > > > > > > Kind regards, > > > > > > > > > > > > 2015-07-10 12:51 GMT+02:00 Kasper Sørensen < > > > [email protected] > > > > >: > > > > > > > > > Hi all, > > > > > > > > > > All the time I see more and more need for us to add metadata to our > > > > > MetaModel based connectors. That could be for instance metadata > about > > > > scale > > > > > (nominal, ordinal etc.) so that we can automate some stats > collection > > > > etc. > > > > > or it could be more "meaning" oriented features to describe e.g. > > "This > > > > is a > > > > > first name" or "This is a city" or "These two fields (first and > last > > > > name) > > > > > are together defining a name of a person". > > > > > > > > > > We have such mechanisms in our application levels many places, but > > not > > > at > > > > > the core framework of MetaModel and that's a pity because it makes > it > > > > > harder for us to share. > > > > > > > > > > So I'm thinking of adding such a layer to the metadata of > MetaModel. > > > But > > > > > one thing that's difficult is then about representing that metadata > > in > > > > some > > > > > "metadata store" which isn't necesarily the same as the data source > > > > itself. > > > > > It could be an XML file or it could be a complete metadata > database. > > > And > > > > I > > > > > think that this metadata would be mutable by the integrator of > > > MetaModel > > > > > because it is rarely fully revealed by the data source itself. > > > > > > > > > > What do you think? Nice feature or?` > > > > > > > > > > Kasper > > > > > > > > > > > > > > >
