Hi Ralph,
Thank you for pointing me to those schemas.
Of course, this is the exact issue I was hoping to avoid. For example, in
the three graphs you mentioned, there are 261 definitions, 107 (41%) of
which do not have any comments explaining their intended meaning or use
(see query below). Of the 261 definitions, roughly 120 are classes.
Now my task must be understanding the meaning and intended use of all of
the classes and determining what Azure (or GCP or AWS) resource most
closely aligns so that I can extend or use the correct class, otherwise, I
won't be able to use EDG functionality like lineage, etc... If I didn't
care about integrating with the EDG ontologies, it would be easier to just
make my own ontologies to support Azure, GCP & AWS. I can export a json
file from Azure that effectively contains instance data from which I can
derive an ontology - which also means I can auto-populate EDG with the
Azure structure.
This type of mapping needs to be done only once. Maybe this is an
opportunity for TQ to create another asset collection type and sell it as
an add-on?
As a side note, the ETL and Infrastructure schemas you shared are not
accessible from the Includes tab. I had to open them using the Files asset
collection to get them to show up in the EDG UI. (This also entailed
unlocking the workspace). Maybe there is another way that I missed.
Also, I noticed a number of "empty property shapes" on
edg:AzureDataBricksPipeline, coming from a superclass. (see attached
file). Walking up the tree shows one undefined property shape at
edg:System. The label on this class does not match the URI - rdfs:label =
"Software System". Five more can be found at edg:EnterpriseEnabler.
Additionally, edg:AssetClass contains four undefined property shapes. This
is only what I found in the cursory look. A query would be better (or
maybe a SHACL rule?) to find all of the "hanging" shapes. Of course, it
may be that the missing shapes are defined graphs that are not imported
into these graphs.
Tim
SELECT ?commentflag (COUNT(?commentflag) AS ?numcomments)
WHERE {
?s rdfs:isDefinedBy ?defininggraph .
?s rdf:type ?stype .
OPTIONAL { ?s rdfs:comment ?scomment . } .
FILTER (?defininggraph IN
(<http://edg.topbraid.solutions/1.0/schema/technical-assets/infrastructure>,
<http://edg.topbraid.solutions/1.0/schema/technical-assets/ETL>,
<http://edg.topbraid.solutions/1.0/schema/technical-assets>))
FILTER (?stype NOT IN (sh:PropertyShape, sh:PropertyGroup))
BIND (BOUND(?scomment) AS ?commentflag)
}
GROUP BY ?commentflag
On Monday, August 8, 2022 at 5:51:19 PM UTC-4 Ralph Hodgson wrote:
> Tim,
>
> Currently EDG has some coverage of ETL and Infrastructure/Cloud resources.
> The intent has been to consider ETL blocks as “black boxes” - that is, not
> to model the details pf what metadata repositories are already modeling but
> to depict how these building blocks play in the larger ecosystem.
>
> Here is an Azure example, a pipeline:
>
> edg:AzureDatabricksPipeline
> a edg:AssetClass ;
> edg:acronym "ADBPL" ;
> rdfs:comment "An 'Azure Databricks Pipeline' is a pipeline that is based
> on Databricks running on Microsoft Azure." ;
> rdfs:isDefinedBy <
> http://edg.topbraid.solutions/1.0/schema/technical-assets/ETL> ;
> rdfs:label "Azure Databricks Pipeline" ;
> rdfs:subClassOf edg:ETLpipeline ;
> .
>
>
> This is defined in the ETL schema (as stated by the refs:isDefinedBy
> statement)
>
> Diving deeper into other ETL and Infrastructure components can be done. I
> would recommend extending EDG’s classes for functional components. There
> are 3 schemas to look into:
>
>
> 1. SCHEMA_EDG-technical-assets-v1.0.ttl
> 2. SCHEMA_EDG-technical-assets-ETL-v1.0.ttl
> 3. SCHEMA_EDG-technical-assets-Infrastructure-v1.0.ttl
>
> Ralph
>
>
> On Aug 8, 2022, at 5:34 PM, Fan Li <[email protected]> wrote:
>
> Hi Tim, I have similar needs to model AWS resources. Additionally I would
> like to model Office365 services (e.g. SharePoint lists / document
> libraries, MS Teams).
> On Monday, August 8, 2022 at 4:43:49 PM UTC-4 Tim Smith wrote:
>
>> Hi,
>> I am going to model some of my Azure environments using EDG. The first
>> task is to determine what classes are required to represent the Azure
>> resources. Below is a small list of available Azure resources. There are
>> hundreds more.
>>
>> The hardest part of this effort will be to sort through the EDG
>> ontologies to find either the class that best semantically represents the
>> Azure resource or which class to extend to capture Azure specific details.
>>
>> Given how common cloud resources are, has their been any work done to map
>> these environments to EDG to facilitate using EDG to govern cloud
>> environments? I'm not looking for an import from the cloud capability
>> (although that would be fantastic!), just a resource to EDG ontology
>> mapping. I'm sure I'm not the first person wanting to do this.
>>
>> Thanks,
>>
>> Tim
>>
>>
>> *Azure Resource*
>> AKS (Kubernetes)
>> AKS (Kubernetes) Pod
>> Analysis Services
>> App Insights
>> App Service Plan
>> Automation Account
>> Azure Active Directory
>> Cognitive Services
>> Container Instance
>> Container Registry
>> Cosmos DB
>> Databricks
>> Disk
>> Event Grid Topic
>> Event Hub
>> Function App
>> Key Vault
>> Load Balancer
>> Log Analytics Workspace
>> Logic App
>> Machine Learning Service
>> Maps
>> MySQL Server
>> Network Security Group
>> Notification Hub
>> PostgresSQL Database
>> Power BI
>> Power BI Gateway
>> Private DNS
>> Private Endpoint
>> Private Link Service
>> QnA Maker
>> Redis Cache
>> Resouce Group
>> Secret
>> Service Principal
>> SignalR
>> SQL Data Warehouse
>> SQL Database
>> SQL Database Managed Instance
>> Storage Account
>> Stream Analytics
>> Subscription
>> Synapse Analytics
>> Tags
>> Tenant
>> Virtual Network
>> VMs
>> Web App
>> Web App PaaS Blueprint
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "TopBraid Suite Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/topbraid-users/ae908d28-4925-4127-9711-fdfef4b6b5dbn%40googlegroups.com
>
> <https://groups.google.com/d/msgid/topbraid-users/ae908d28-4925-4127-9711-fdfef4b6b5dbn%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
>
--
You received this message because you are subscribed to the Google Groups
"TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/topbraid-users/0ebd4d9c-7ddc-4833-a86d-2896b023a1fdn%40googlegroups.com.
