Hi Max, we have 3693 task instances in the database. And we have created an index over start_date for table task_instance as you suggested, but it is still slow. We don't think the problem is in the database, because when we run AdHoc Queries, they return fast.
Em qua, 9 de ago de 2017 às 12:47, Maxime Beauchemin < [email protected]> escreveu: > It seems like the default sort should be on start_date, desc, and yes there > should be an index on that. > > Also 100 per page is probably enough. > > Can you try [something like] that in your environment and report of loading > times? > > Also for context, how many task instance do you have total? > > Max > > On Tue, Aug 8, 2017 at 12:07 PM, Victor Monteiro <[email protected]> > wrote: > > > [image: Imagem PNG] > > Screen Shot 2017-08-08 at 2.42.55 PM.png > > <https://drive.google.com/a/ubee.in/file/d/0B7u1tjyaPWJQeVVtbFIzcS11eWc/ > > view?usp=drivesdk> > > > > I am sending the image one more time, hosted in google drive. > > https://drive.google.com/a/ubee.in/file/d/0B7u1tjyaPWJQeVVtbFIzcS11eWc/ > > view?usp=drivesdk > > > > Edgar, did you find any solution to speed up webserver? > > > > Em ter, 8 de ago de 2017 às 16:04, Edgar Rodriguez > > <[email protected]> escreveu: > > > > > I've been profiling the web UI for the last few days and I think I've > > been > > > able to identify some of the issues. I've seen similar response times > > from > > > the webserver. > > > A couple of things that I found specifically for the task instance view > > > are: > > > 1. Page sizes on views are usually too large, and all HTML rendering is > > > done server side, flask_admin introduces some latency rendering the > > > templates for 500 TIs at a time in the TaskInstanceModelView, see [ > > > AIRFLOW-1483 <https://issues.apache.org/jira/browse/AIRFLOW-1483>] > > > 2. Using unindexed column as default for ordering (required for > paging), > > > triggering a sort on TI requests, e.g. TaskInstanceModelView uses > > `job_id` > > > as default sort column, but there's no index for that, see > [AIRFLOW-1495 > > > <https://issues.apache.org/jira/browse/AIRFLOW-1495>] > > > > > > Cheers, > > > Edgar > > > > > > On Tue, Aug 8, 2017 at 11:56 AM, Victor Monteiro < > > [email protected]> > > > wrote: > > > > > > > Sorry, I am sending again. > > > > > > > > Also, it is always between 6s and 3s. > > > > > > > > > > > > Em ter, 8 de ago de 2017 às 15:21, Ash Berlin-Taylor < > > > > [email protected]> escreveu: > > > > > > > >> (Your screenshot didn't come through for me, possibly because the > list > > > >> stripped it? That said:) > > > >> > > > >> Is it always 6 seconds, or after making a few requests, enough so > that > > > >> each worker stands a chance to have loaded the app any deps does it > > > settle > > > >> down? > > > >> > > > >> i.e. the problem might just be that of warm-up. > > > >> > > > >> -ash > > > >> > On 8 Aug 2017, at 18:52, Victor Monteiro <[email protected] > > > > > >> wrote: > > > >> > > > > >> > Hi everyone. > > > >> > > > > >> > The problem is very straightforward. When doing a request to > airflow > > > >> webserver, it is taking too much time to send the first byte. > > > >> > > > > >> > > > > >> > > > > >> > As you can see in the picture, it took 6 seconds to send the first > > > >> byte. I already investigated the connection with the database and it > > > took > > > >> 36ms to list all task instances. So, I am starting to think there > is a > > > >> problem with airflow webserver or my deployment. > > > >> > > > > >> > To give you more details about deployment and configurations: > > > >> > web_server_worker_timeout = 120 > > > >> > workers = 4 > > > >> > sql_alchemy_pool_size = 5 > > > >> > sql_alchemy_pool_recycle = 3600 > > > >> > AWS RDS postgres > > > >> > AWS m4.large > > > >> > Does anyone know what can be causing this problem? > > > >> > > > > >> > Thank you :D > > > >> > > > > >> > > > >> > > > > > >
