t4n1o edited a comment on issue #19192:
URL: https://github.com/apache/airflow/issues/19192#issuecomment-1003779286
> 1. Do you actually have scheduler running at all? Does it have
coninuous access to the DB?
Yes, the scheduler is visible in the running processes, and also systemd.
The scheduler is the only program accessing the db. The db is on the same
RAID array/filesystem as the data.
> 2. Are you absolutely sure you are not using SequentialExecutior ?
What does your `airflow info` say - can you paste-bin output of it ? (airlfow
has built-in flag to send to pastebin). Please make sure also that you do it in
exactly the way your scheduler works. Miost likely you run your scheduler with
a different configuration than your webserver and that causes the problem.
I can't find the flag for pastebin but I pasted above.
> 3. Are you sure you are using Postgres and not Sqlite?
Yes
> 4. Where is your Python code (non-DAG)? Did you .airflowignore
non-DAG files from airflow's DAG folder?
/opt/data-workflows <= code
/opt/data-workflows/rust <= code
/opt/data-workflows/dags
There are only dags in the dag folder, I kept everything modular and
separated.
> 5. can you upgrade to Airlfow 2.2.3 (latest released) - it has
built-in warnings in case you use Sequential Executor/SQLite in the UI.
Yes, still the same problem on 2.2.3. No warnings about sqllite or
sequential executor
> 6. Can you change your DAGs to:
When I run your dag with the `sleep 1000`, I also can launch many dags in
parallel.But it still displays the weird msg on the web UI.
" The scheduler does not appear to be running. Last heartbeat was received
4 minutes ago. "
Perhaps there is something weird in the way BashOperator spawns subprocesses
that can block the entire scheduler when used this way. Or a bug with
subprocess. Or maybe it doesn't like the CWD to be changed.
```
bash_command="umask 002 && cd /opt/data_workflows/rust/ && ./parser",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
