On Tue, Jun 15, 2021 at 8:22 PM Canapathy, Subash <[email protected]> wrote:
> @Ash Berlin-Taylor <[email protected]> – I don’t think that is entirely > true. In 1.10 the connection templates code was part of the flask > application and not bundled with the provider. Managed services took the > webserver baseline as is and let the customers take decision on additives > like FB-business, oracle etc.. without bundling them into the managed > service software per AWS compliance guidelines. In 2.0 if we bake in all > the providers, it will mean that we are baking in their dependencies along > with. > > Just to clarify - in 1.10 there were no providers (there were backports for them) but the dependencies you are talking about were coming through extras (and I think you had very limited set of extras which indeed made you not depend on the dependencies). So you are right in 1.10 you did not have this coupling (all the connection types were baked-in airflow as you explained). > Regarding security constraints on why we disallow plugins and > requirements on the webserver, I will have to discuss this in person on PMC > but on a high level this comes down to remote code execution prevention on > managed instances, opening possibilities of exploiting vulnerabilities on > the flask-app-builder and the underlying python runtime. There is 2 levels > of isolation – one on the single tenancy of environments in MWAA under > separate VPCs, and secondly on Fargate that prevents exploits to break out > of the container boundaries into the hypervisor. Even with those, our > security team had other possibilities of exploits unearthed in penetration > testing that led to this decision. > Yep. I was involved in a vetting process of similar UI-facing apps and I understand where it comes from, however we would love to hear what your concerns are Subash. Maybe you can send a message to [email protected] and explain ? Coming back to my question. Would that help if you could generate such a "shim" package and you install it at the webserver? I can imagine a tool/command where you specify which built-in providers you installed, and run a command that could generate (out of Airflow sources corresponding to your version) a "whl" package containing: * entrypoint returning set of connections that you miss * extracted classes containing the meta-data and Field definitions for the UI for the relevant connections * no additional dependencies There is a small potential caveat with versioning for future versions of providers (but those connections change rarely and mostly in backwards compatible ways and likely you could regenerate the "shim" periodically). J.
