[
https://issues.apache.org/jira/browse/BEAM-11608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311043#comment-17311043
]
Alex Amato edited comment on BEAM-11608 at 3/30/21, 12:18 AM:
--------------------------------------------------------------
I'll help to clarify the issue here.
Basically when we setup the development environment to work on Java, Go or
Python we can run into several errors and issues. The goal here would be to
create a process to setup a development environment which is reliable and
consistent. In particular installing Java and IntelliJ are important parts, as
there are often issues that people run into. Its not uncommon to get stuck on
an error/build issue for multiple days when setting up an environment,
re-basing from master. etc.
A few approaches we have thought about.\
# *Scripting* - Using a shell script to setup the necessary steps to setup
python, go, java, IntelliJ etc. The downside of this is it may be difficult to
setup installation steps which work on different OSes. Plus this doesn't really
create a common environment. Simply putting this into a script likely won't
work too well on its own as users are likely to run into various errors and
issues depending on their OS, existing things installed in their environment,
which version/build of their OS, things installed, etc.
# *Docker* (Tried twice, but both attempts seem to be inconsistent) - Create a
DockerFile to run the necessary setup steps for git, Java, python, go,
IntelliJ, etc. Share the docker container with the dev community. One can run
the docker container to have a development environment. The downsides of this
are (1) to use intelliJ you will need to setup x forwarding, ([~rohdesam] has a
[branch|[https://github.com/apache/beam/pull/13430/files]] with a docker file
on his repo, but I ran into some errors when running it. Also there is a
separate docker [setup for this checked
in|https://github.com/apache/beam/commit/60fe232a75117fb8981b608e11be3d14a34bd83b],
though this one seemed to be doing too much work in a startup script rather
than a docker file. I also ran into errors running this as well). Downside (2)
here is that docker doesn't seem to be as consistent when other people run it
on their machines. You may try to determine the issues here, but it could be
due to some users being on mac and some on linux. Overall this option seems
less reliable/consistent as the VirtualMachine approach, IMO.
# *VirtualMachine* (i.e. Virtualbox) - Setting up a Linux(i.e. Ubuntu) VM with
a git repo, and all the installation steps completed for java, python, go, and
intelliJ. The image for the virtual machine can be shared and used as a
development environment. This solutions provides a consistent build environment
that one can just load up and start developing after rebasing from master. To
maintain this, if there are any issues after rebasing, then one person can fix
them on the image and create and share an updated version of the image. (I
looked into this with VirtualMachineManager, but found it was complicated to
share the image, I suspect you may be able to share a single file or folder to
share the VM image in Virtualbox, which will hopefully be simpler). The main
downside is needing extra ram or CPU to get this to work.
IMO the VirtualMachine solution is the most promising one. If it were up to me,
I would pursue this solution.
Docker and the scripting solution is likely to be inconsistent for different
people.
The main reason to pursue scripting is to prevent the documentation's script
commands from becoming out of date. As we will need to update our scripts when
there are issues.
was (Author: [email protected]):
I'll help to clarify the issue here.
Basically when we setup the development environment to work on Java, Go or
Python we can run into several errors and issues. The goal here would be to
create a process to setup a development environment which is reliable and
consistent. In particular installing Java and IntelliJ are important parts, as
there are often issues that people run into. Its not uncommon to get stuck on
an error/build issue for multiple days when setting up an environment,
re-basing from master. etc.
A few approaches we have thought about.\
# *Scripting* - Using a shell script to setup the necessary steps to setup
python, go, java, IntelliJ etc. The downside of this is it may be difficult to
setup installation steps which work on different OSes. Plus this doesn't really
create a common environment. Simply putting this into a script likely won't
work too well on its own as users are likely to run into various errors and
issues depending on their OS, existing things installed in their environment,
which version/build of their OS, things installed, etc.
# *Docker* (Tried twice, but both attempts seem to be inconsistent) - Create a
DockerFile to run the necessary setup steps for git, Java, python, go,
IntelliJ, etc. Share the docker container with the dev community. One can run
the docker container to have a development environment. The downsides of this
are (1) to use intelliJ you will need to setup x forwarding, ([~rohdesam] has a
[branch|[https://github.com/apache/beam/pull/13430/files]] with a docker file
on his repo, but I ran into some errors when running it. Also there is a
separate docker [setup for this checked
in|https://github.com/apache/beam/commit/60fe232a75117fb8981b608e11be3d14a34bd83b],
though this one seemed to be doing too much work in a startup script rather
than a docker file. I also ran into errors running this as well). Downside (2)
here is that docker doesn't seem to be as consistent when other people run it
on their machines. You may try to determine the issues here, but it could be
due to some users being on mac and some on linux. Overall this option seems
less reliable/consistent as the VirtualMachine approach, IMO.
# *VirtualMachine* (i.e. Virtualbox) - Setting up a Linux(i.e. Ubuntu) VM with
a git repo, and all the installation steps completed for java, python, go, and
intelliJ. The image for the virtual machine can be shared and used as a
development environment. This solutions provides a consistent build environment
that one can just load up and start developing after rebasing from master. To
maintain this, if there are any issues after rebasing, then one person can fix
them on the image and create and share an updated version of the image. (I
looked into this with VirtualMachineManager, but found it was complicated to
share the image, I suspect you may be able to share a single file or folder to
share the VM image in Virtualbox, which will hopefully be simpler). The main
downside is needing extra ram or CPU to get this to work.
IMO the VirtualMachine solution is the most promising one. If it were up to me,
I would pursue this solution.
Docker and the scripting solution is likely to be inconsistent for different
people.
> Development environment set up automation
> -----------------------------------------
>
> Key: BEAM-11608
> URL: https://issues.apache.org/jira/browse/BEAM-11608
> Project: Beam
> Issue Type: Wish
> Components: build-system, website
> Reporter: Tyson Hamilton
> Priority: P3
>
> The manual steps in the contribution guide should be combined into a single
> command for users. For example, this could be done by putting all manual
> steps into a shell script.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
