I commented out the cython dep in pyproject and then removed the out of date .so files and was able to run `pytest apache_beam/pvalue_test.py` 🎉. Thanks for the help!
On Thu, Sep 19, 2024 at 7:21 PM Valentyn Tymofieiev via dev < dev@beam.apache.org> wrote: > or alternatively, commenting-out the cython dep in pyproject.toml might > remove cythonization too. > > On Thu, Sep 19, 2024 at 4:16 PM Joey Tran <joey.t...@schrodinger.com> > wrote: > >> cython isn't installed in the virtualenv. Running `pip uninstall cython` >> resulted in: >> `WARNING: Skipping cython as it is not installed.` >> >> On Thu, Sep 19, 2024 at 7:05 PM Robert Bradshaw via dev < >> dev@beam.apache.org> wrote: >> >>> >>> >>> On Thu, Sep 19, 2024 at 3:43 PM Joey Tran <joey.t...@schrodinger.com> >>> wrote: >>> >>>> Ah okay then. I commented out the goavro line and the image does finish >>>> building. It seems that the python still has be built which takes a bit >>>> (using ` pip install -e .[gcp,test]` inside the sdks/python directory). It >>>> looks like the source isn't synced with the host system it doesn't look >>>> like there's a git repo inside the container either so maybe the container >>>> env isn't worth the effort (especially if no one else is using or helping >>>> maintaining it) >>>> >>>> Going back to trying to get a local python environment working... I >>>> tried doing this a few months ago and eventually just gave up. I ran >>>> `./local-env-setup.sh` >>>> >>> >>> I honestly have no idea what this is. >>> >>> >>>> and then following the instructions in the python developer guide in >>>> the wiki[1], I ran `pip install -e .[gcp,test]`. I eventually hit an issue >>>> in what looks cythonizing windowed_value.py >>>> >>>> ``` >>>> [14/14] Cythonizing apache_beam/utils/windowed_value.py >>>> ``` >>>> >>>> There's a lot of messaging but it looks like the crux of the issue is: >>>> ``` >>>> clang: error: invalid arch name '-arch >>>> -I/private/var/folders/n1/6qk3ljm97h32j1g7qg.... >>>> ``` >>>> >>>> Googling around it seems like it might be some kind of mac M2 issue, >>>> but there aren't any obvious solutions. Has anyone that works on the python >>>> sdk with a new-ish mac run into this problem? >>>> >>> >>> I don't have a new mac, but you shouldn't need to cythonize for >>> development. I assume you have a virtual environment from following the >>> steps in the wiki? Can you "pip uninstall cython" then re-run "pip install >>> -e .[gcp,test]"? >>> >>> Best, >>>> Joey >>>> [1] https://cwiki.apache.org/confluence/display/BEAM/Python+Tips >>>> >>>> >>>> On Thu, Sep 19, 2024 at 5:23 PM Robert Burke <rob...@frantil.com> >>>> wrote: >>>> >>>>> To my knowledge, the "dev" docker image is unmaintained, and has been >>>>> for quite some time, hence not having been moved to Go Modules and >>>>> similar. >>>>> >>>>> I personally don't have time to delve into what's going on with it, >>>>> but IIRC, deleting the weird goavro lines would be a good first step. >>>>> >>>>> I'd be happy to review any PRs to help fix it though, if you get it >>>>> working for yourself. (Tag lostluck in GitHub). >>>>> >>>>> I think if you search the dev list, you might find previous >>>>> discussions on the topic of the dev container. >>>>> >>>>> Robert Burke >>>>> >>>>> >>>>> On Thu, Sep 19, 2024, 2:17 PM Joey Tran <joey.t...@schrodinger.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm trying again to get a working development image of beam. Working >>>>>> from the beam repo CONTRIBUTING.md, I ran `./start-build-env.sh` >>>>>> >>>>>> At image build time after getting a few steps in, the build fails >>>>>> with: >>>>>> >>>>>> => ERROR [7/8] RUN go get github.com/linkedin/goavro/v2 >>>>>> >>>>>> > [7/8] RUN go get github.com/linkedin/goavro/v2: >>>>>> >>>>>> >>>>>> 0.134 go: go.mod file not found in current directory or any parent >>>>>> directory. >>>>>> 0.134 'go get' is no longer supported outside a module. >>>>>> >>>>>> 0.134 To build and install a command, use 'go install' with a >>>>>> version, >>>>>> 0.134 like 'go install example.com/cmd@latest' >>>>>> >>>>>> 0.134 For more information, see >>>>>> https://golang.org/doc/go-get-install-deprecation >>>>>> 0.134 or run 'go help get' or 'go help install'. >>>>>> ------ >>>>>> 2 warnings found (use docker --debug to expand): >>>>>> - LegacyKeyValueFormat: "ENV key=value" should be used instead of >>>>>> legacy "ENV key value" format (line 8) >>>>>> - LegacyKeyValueFormat: "ENV key=value" should be used instead of >>>>>> legacy "ENV key value" format (line 7) >>>>>> Dockerfile:10 >>>>>> -------------------- >>>>>> 8 | ENV GOPATH >>>>>> /home/jtran/beam/sdks/go/examples/.gogradle/project_gopath >>>>>> 9 | # This next command still runs as root causing the >>>>>> ~/.cache/go-build to be owned by root >>>>>> 10 | >>> RUN go get github.com/linkedin/goavro/v2 >>>>>> 11 | RUN chown -R jtran:100 /home/jtran/.cache >>>>>> 12 | >>>>>> -------------------- >>>>>> ERROR: failed to solve: process "/bin/bash -o pipefail -c go get >>>>>> github.com/linkedin/goavro/v2" did not complete successfully: exit >>>>>> code: 1 >>>>>> >>>>>> >>>>>> Any tips? >>>>>> Cheers, >>>>>> -- >>>>>> >>>>>> Joey Tran >>>>>> >>>>>> [image: Schrödinger, Inc.] <https://schrodinger.com/> >>>>>> >>>>>
