Nice work. I added some comments on parameters in the path component and representation of cloud storage uris.
Bolke On Wed, 3 Jan 2024 at 22:51, Jarek Potiuk <ja...@potiuk.com> wrote: > Happy 2024, > > Added some nits - mostly informational, generally LGTM > > On Tue, Jan 2, 2024 at 11:23 AM Tzu-ping Chung <t...@astronomer.io.invalid> > wrote: > > > > Happy 2024 everyone! > > > > I’m going to kick off the new year by formally proposing a new AIP. This > attempts to standardise the URI format used by Dataset events. This is > driven a lot by the lack of adoption of Datasets. It turns out (maybe not > surprisingly when I think about it) simply triggering events from a literal > string name isn’t particularly useful (usable) in practical contexts, and > some “smarter” features are generally sought after in most cases. > > > > Two most popular examples are listening on a directory for file > additions, or making operators emit Dataset events automatically like > OpenLineage events. Both are technically doable, but rather impractical > without abusing the literal string Dataset identifier. By introducing a > standard semantic, those can be more easily implemented on the scheduler > (listening) side instead. > > > > Please find the document on Confluence: > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-60+Standard+URI+representation+for+Airflow+Datasets > > > > Both comments on the specification and/or implementation, or proposals > to add to the URI formats are welcomed. Note that we don’t need to add all > the formats in the AIP; this only attempts to establish a process to do so, > so we can add new ones to the documentation in the future. > > > > TP > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > > -- -- Bolke de Bruin bdbr...@gmail.com
