dstandish opened a new pull request #15100: URL: https://github.com/apache/airflow/pull/15100
## Context PR https://github.com/apache/airflow/pull/15013 raised the issue that airflow's conn uri format cannot handle arbitrary json values. This PR adds support for this without breaking backward compatibility. ## Overview URIs defined as they are today will continue to parse in the exact same way. But we add a new option for storing arbitrary json. To store nested json, encode using base 64, quote, and store under query param `__extra__`. Example: ``` >>> extra_dict = {'my_val': ['list', 'of', 'values'], 'extra': {'nested': {'json': 'val'}}} >>> c = Connection( >>> conn_type='scheme', >>> host='host/location', >>> schema='schema', >>> login='user', >>> password='password', >>> port=1234, >>> extra=json.dumps(extra_dict), >>> ) >>> uri = c.get_uri() >>> uri 'scheme://user:password@host%2Flocation:1234/schema?__extra__=eyJteV92YWwiOiBbImxpc3QiLCAib2YiLCAidmFsdWVzIl0sICJleHRyYSI6IHsibmVzdGVkIjogeyJqc29uIjogInZhbCJ9fX0%3D' ``` ## Details ### URI generation When generating the URI, the method `get_uri` will check whether the `extra` can be simply urlencoded without loss or error. If so, it will use urlencoding. When loss or error would occur, `get_uri` will use base 64 encoding. ### URI parsing First airflow will check for the existence of special query param `__extra__`. If it exists, airflow will parse it as base 64. If this param is not present, the traditional approach is used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
