dstandish commented on PR #25250: URL: https://github.com/apache/airflow/pull/25250#issuecomment-1193989243
> > You are right @ephraimbuddy , was trying to add something that Brent needed but there must be a better way > > A few options: > > 1. The UI is just inefficient and has to do multiple API requests. > 2. We make a UI-only API endpoint with exactly what we need > 3. We refactor the id and display name for a dataset. Task id, DAG id and Run id are all readable Yeah i guess what I'm agreeing with is, maybe it doesn't make sense to include the full dataset object (as i've done here) in the event API. cus thinking, if you're just after the events, i guess it's a bit inefficient. but, we could just add just the URI, for example, instead of the full object. and, that's what i set out to do initially, but then i couldn't immediately figure out how to map the response field to `dataset.uri` (i.e. the attr on the attr -- there's a `datatset` attr on DatasetEvent through SQLA relationship). i could made it happen by adding a `uri` property on DatasetEvent but i didn't really want to.... so i just added the full object. if you know how to map `dataset.uri` to a field in response, lemme know. --- all of these options, i don't have a strong feeling either way really. but i don't think there's anything wrong with having the endpoint be more friendly than the database schema. the database schema is optimized for db access and storage. but the rest API is not bound by those considerations necessarily. @bbovenzi what's the context again? TI (i.e. downstream) or DAG (i.e. upstream)? Either way we know, or could know, the datasets that it maps to, and could provide a dictionary from ID to URI (e.g. in serialized repr). Could we use something like that to lookup the ID in the event response? (and therefore keep the API "normalized" _and_ avoid multiple queries? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
