Re: [PR] chore: deprecate tox in favor of act [superset]
jpadams commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2523398507 @mistercrunch and @rusackas I've had a todo to start work on a Dagger PR for Superset for the last six months, but haven't started yet. Bravo on making things better! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
mistercrunch merged PR #29382: URL: https://github.com/apache/superset/pull/29382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
rusackas commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2501926657 I, for one, will not miss tox. This'll be a much better DevEx. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
github-actions[bot] commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2501838357 @mistercrunch No ephemeral environment action detected. Please use '/testenv up' or '/testenv down'. [View workflow run](https://github.com/apache/superset/actions/runs/12038367960). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
mistercrunch commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2501838080 Not everything works here, but no one uses `tox` anymore AFAIK, and I think the setup is broken. I advocate for merging this and polishing/improving `act` and `docker-compose` support over time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
mistercrunch commented on code in PR #29382:
URL: https://github.com/apache/superset/pull/29382#discussion_r1858822795
##
docs/docs/contributing/development.mdx:
##
@@ -502,39 +502,117 @@ If using the eslint extension with vscode, put the
following in your workspace `
]
```
-## Testing
-### Python Testing
+## GitHub Actions and `act`
+
+:::tip
+`act` compatibility of Superset's GHAs is not fully tested. Running `act`
locally may or may not
+work for different actions, and may require fine tuning and local
secret-handling.
+For those more intricate GHAs that are tricky to run locally, we recommend
iterating
+directly on GHA's infrastructure, by pushing directly on a branch and
monitoring GHA logs.
+For more targetted iteration, see the `gh workflow run --ref {BRANCH}`
subcommand of the GitHub CLI.
+:::
+
+For automation and CI/CD, Superset makes extensive use of GitHub Actions
(GHA). You
+can find all of the workflows and other assets under the `.github/` folder.
This includes:
+- running the backend unit test suites (`tests/`)
+- running the frontend test suites (`superset-frontend/src/**.*.test.*`)
+- running our Cypress end-to-end tests (`superset-frontend/cypress-base/`)
+- linting the codebase, including all Python, Typescript and Javascript, yaml
and beyond
+- checking for all sorts of other rules conventions
+
+When you open a pull request (PR), the appropriate GitHub Actions (GHA)
workflows will
+automatically run depending on the changes in your branch. It's perfectly
reasonable
+(and required!) to rely on this automation. However, the downside is that it's
mostly an
+all-or-nothing approach and doesn't provide much control to target specific
tests or
+iterate quickly.
+
+At times, it may be more convenient to run GHA workflows locally. For that
purpose
+we use [act](https://github.com/nektos/act), a tool that allows you to run
GitHub Actions (GHA)
+workflows locally. It simulates the GitHub Actions environment, enabling
developers to
+test and debug workflows on their local machines before pushing changes to the
repository. More
+on how to use it in the next section.
+
+:::note
+In both GHA and `act`, we can run a more complex matrix for our tests,
executing against different
+database engines (PostgreSQL, MySQL, SQLite) and different versions of Python.
+This enables us to ensure compatibility and stability across various
environments.
+:::
-All python tests are carried out in
[tox](https://tox.readthedocs.io/en/latest/index.html)
-a standardized testing framework.
-All python tests can be run with any of the tox
[environments](https://tox.readthedocs.io/en/latest/example/basic.html#a-simple-tox-ini-default-environments),
via,
+### Lower-Level Control
+
+Note that for all GHAs, it's also possible to go lower level and the content
of any workflow
Review Comment:
Re-reading it I'm wondering what I was thinking. I think the general idea
was "if you can't run the GHA locally, you can find a way to run its content in
docker-compose", which kind of self-evident and not really helpful. I'm going
to remove this paragraph.
Now I think we should move forward with deleting `tox` and live with the
fact that `act` doesn't work for all GHAs. Rationale is tox isn't working
anyways, and GHA / docker-compose largely covers for what with need, and are
better abstractions overall.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
rusackas commented on code in PR #29382:
URL: https://github.com/apache/superset/pull/29382#discussion_r1857130300
##
docs/docs/contributing/development.mdx:
##
@@ -502,39 +502,117 @@ If using the eslint extension with vscode, put the
following in your workspace `
]
```
-## Testing
-### Python Testing
+## GitHub Actions and `act`
+
+:::tip
+`act` compatibility of Superset's GHAs is not fully tested. Running `act`
locally may or may not
+work for different actions, and may require fine tuning and local
secret-handling.
+For those more intricate GHAs that are tricky to run locally, we recommend
iterating
+directly on GHA's infrastructure, by pushing directly on a branch and
monitoring GHA logs.
+For more targetted iteration, see the `gh workflow run --ref {BRANCH}`
subcommand of the GitHub CLI.
+:::
+
+For automation and CI/CD, Superset makes extensive use of GitHub Actions
(GHA). You
+can find all of the workflows and other assets under the `.github/` folder.
This includes:
+- running the backend unit test suites (`tests/`)
+- running the frontend test suites (`superset-frontend/src/**.*.test.*`)
+- running our Cypress end-to-end tests (`superset-frontend/cypress-base/`)
+- linting the codebase, including all Python, Typescript and Javascript, yaml
and beyond
+- checking for all sorts of other rules conventions
+
+When you open a pull request (PR), the appropriate GitHub Actions (GHA)
workflows will
+automatically run depending on the changes in your branch. It's perfectly
reasonable
+(and required!) to rely on this automation. However, the downside is that it's
mostly an
+all-or-nothing approach and doesn't provide much control to target specific
tests or
+iterate quickly.
+
+At times, it may be more convenient to run GHA workflows locally. For that
purpose
+we use [act](https://github.com/nektos/act), a tool that allows you to run
GitHub Actions (GHA)
+workflows locally. It simulates the GitHub Actions environment, enabling
developers to
+test and debug workflows on their local machines before pushing changes to the
repository. More
+on how to use it in the next section.
+
+:::note
+In both GHA and `act`, we can run a more complex matrix for our tests,
executing against different
+database engines (PostgreSQL, MySQL, SQLite) and different versions of Python.
+This enables us to ensure compatibility and stability across various
environments.
+:::
-All python tests are carried out in
[tox](https://tox.readthedocs.io/en/latest/index.html)
-a standardized testing framework.
-All python tests can be run with any of the tox
[environments](https://tox.readthedocs.io/en/latest/example/basic.html#a-simple-tox-ini-default-environments),
via,
+### Lower-Level Control
+
+Note that for all GHAs, it's also possible to go lower level and the content
of any workflow
Review Comment:
I'm not sure I understand this sentence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
rusackas commented on code in PR #29382: URL: https://github.com/apache/superset/pull/29382#discussion_r1857103892 ## docs/docs/contributing/development.mdx: ## @@ -502,39 +502,117 @@ If using the eslint extension with vscode, put the following in your workspace ` ] ``` -## Testing -### Python Testing +## GitHub Actions and `act` + +:::tip +`act` compatibility of Superset's GHAs is not fully tested. Running `act` locally may or may not +work for different actions, and may require fine tunning and local secret-handling. Review Comment: ```suggestion work for different actions, and may require fine tuning and local secret-handling. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
mistercrunch commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2207201663 Yup, also would welcome some help here as I got tangled up in other things and it looks like this is getting pushed to the bottom of my todo list now. Things seem super promising, but there's quite a few little things to connect. I discovered there's support for `.env` files an `.secrets` files which both can reference env vars and set defaults. Seemed powerful. The last blocker I had was around the `pip install` not working (because python ssl lib not properly set up, and while trying to reach out to http**S**://pypy.org ) on the base docker images that `act` is using. Seems pretty doable to swap to any other image, but I didn't get too far. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
rusackas commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2206887857 Should we move this back to `draft` mode until some of the dust settles? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
mistercrunch commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2204732772 Thinking about the success criteria for this PR, seems 1. we should be able to run all "locally-relevant" GHA workflow. This means stuff like linting, running tests backend/frontend, pre-commit all, ... We don't need for workflows like the ephemeral envs ones and such to work locally. 2. should be relatively easy to set up, easier than `tox` at least -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
Re: [PR] chore: deprecate tox in favor of act [superset]
mistercrunch commented on PR #29382: URL: https://github.com/apache/superset/pull/29382#issuecomment-2204731191 Realizing there's significantly much more work to do here. Last time I pushed this forward I found some clear incompatibility between the base Ubuntu images used by github and the ones used by `act`, something around python's SSL not being configured right in the catthehacker/act-ubuntu image -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
