dstandish commented on PR #30762: URL: https://github.com/apache/airflow/pull/30762#issuecomment-1889865100
> @dstandish this would make sense if there was a way to define custom sql schema / commands as part of the installation process. As far as I am aware, there is no such thing, and any manual changes could potentially break on db migration. I think it would be more appropriate to plan some work for this, so anyone could use it to define custom schema without risking of breaking the database between upgrades? > > In any case, I don't foresee any trouble caused by this extra index, and it would help if it stays - my use case is quite basic so I think that someone else could benefit from this too. I don't think it's likely to break anything. You can simply name it `my_company_index_blah_blah` and we won't clash with it. Sure if we drop a column that it uses (unlikely) then you might have a problem. But I think that's I think a risk that you must accept given that you are doing something custom with the scheduler that requires a special index. Indexes are not free. When you have more indexes, that is more storage used, more work for postgres on each update, and more for the query planner to sort through. It doesn't make sense to force all airflow users in the world to have an index that doesn't help them at all, just so that you can use it. I think we have like 9 indexes or something on task instance? That's getting quite high. We can't only add indexes, we must also try to remove ones that are not useful. Here's the other thing.... The indexes we include in airflow, you know, they might not be perfect for everyone's cluster. It's possible that some clusters might have an unusual usage pattern and maybe as a result they might need to add special index, even without having custom logic or queries. And even in that case it might not make sense to add such an index to airflow, given that doing so would be a cost for everyone and a benefit perhaps only to one. Right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
