I am in the process of updating the documentation at https://cwiki.apache.org/confluence/display/BEAM/Python+Tips related to setup.py/pyproject.toml changes, but yes you can't call setup.py directly because it might fail due to the lack of presence of beam python's build time dependencies.
With regards to other files(eg:protos), we will follow the similar behavior as before(generating proros using `gen_protos.py`). On Thu, Oct 12, 2023 at 4:01 PM Robert Bradshaw <rober...@google.com> wrote: > Does this change any development practices? E.g. if I clone the repo, I'm > assuming I couldn't run "setup.py test" anymore. What about the generated > files (like protos, or the yaml definitions copied from other parts of the > repo)? > > On Thu, Oct 12, 2023 at 12:27 PM Anand Inguva via dev <dev@beam.apache.org> > wrote: > >> The PR https://github.com/apache/beam/pull/28385 is merged today. If >> there are any observed failures, please comment on the PR and I will follow >> up with a forward fix. Thanks. >> >> On Fri, Sep 1, 2023 at 2:30 PM Anand Inguva <ananding...@google.com> >> wrote: >> >>> Since there is positive feedback from the dev community, I am going >>> ahead and implementing this proposal for Python SDK. >>> >>> @aus...@apache.org <aus...@apache.org> Initially let's move forward >>> with the setuptools as backend for building package and as part of the >>> future work, we can find a better backend than setuptools. >>> >>> Thanks for the feedback. >>> Anand >>> >>> On Mon, Aug 28, 2023 at 12:00 PM Austin Bennett <aus...@apache.org> >>> wrote: >>> >>>> I've thought about this a ton, but haven't been in a position to >>>> undertake the work. Thanks for bringing this up, @Anand Inguva >>>> <ananding...@google.com> ! >>>> >>>> I'd point us to https://python-poetry.org/ ... [ which is where I'd >>>> look take us, but I'm also not able to do all the work, so my >>>> suggestion/preference doensn't matter that much ] >>>> >>>> https://python-poetry.org/docs/pyproject#the-pyprojecttoml-file <- for >>>> info on pyproject.toml file. >>>> >>>> Notice the use of a 'lock' file is very valuable, ex: >>>> https://python-poetry.org/docs/basic-usage/#committing-your-poetrylock-file-to-version-control >>>> >>>> I haven't come across `build`, that might be great too. I'd highlight >>>> that Poetry is pretty common across industry these days, rock-solid, >>>> ecosystem of interoperability, users, etc... If not familiar, PLEASE have >>>> a look at that. >>>> >>>> >>>> >>>> >>>> On Mon, Aug 28, 2023 at 8:04 AM Kerry Donny-Clark via dev < >>>> dev@beam.apache.org> wrote: >>>> >>>>> +1 >>>>> Hi Anand, >>>>> I appreciate this effort. Managing python dependencies has been a >>>>> major pain point for me, and I think this approach would help. >>>>> Kerry >>>>> >>>>> On Mon, Aug 28, 2023 at 10:14 AM Anand Inguva via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> Hello Beam Dev Team, >>>>>> >>>>>> I've compiled a design document >>>>>> <https://docs.google.com/document/d/17-y48WW25-VGBWZNyTdoN0WUN03k9ZhJjLp9wtyG1Wc/edit#heading=h.wskna8eurvjv>[1] >>>>>> proposing the integration of pyproject.toml into Apache Beam's Python >>>>>> build >>>>>> process. Your insights and feedback would be invaluable. >>>>>> >>>>>> What is pyproject.toml? >>>>>> pyproject.toml is a configuration file that specifies a project's >>>>>> build dependencies and other project-related metadata in a standardized >>>>>> format. Before pyproject.toml, Python projects often had multiple >>>>>> configuration files (like setup.py, setup.cfg, and requirements.txt). >>>>>> pyproject.toml aims to centralize these configurations into one place, >>>>>> making project setups more organized and straightforward. One of the >>>>>> significant features enabled by pyproject.toml is the ability to perform >>>>>> isolated builds. This ensures that build dependencies are separated from >>>>>> the project's runtime dependencies, leading to more consistent and >>>>>> reproducible builds. >>>>>> >>>>>> [1] >>>>>> https://docs.google.com/document/d/17-y48WW25-VGBWZNyTdoN0WUN03k9ZhJjLp9wtyG1Wc/edit#heading=h.wskna8eurvjv >>>>>> >>>>>> Thanks, >>>>>> Anand >>>>>> >>>>>
