Yicong-Huang commented on code in PR #5077:
URL: https://github.com/apache/texera/pull/5077#discussion_r3249281597
##########
amber/build.sbt:
##########
@@ -200,6 +200,23 @@ libraryDependencies += "com.thesamet.scalapb" %%
"scalapb-json4s" % "0.12.0"
// enable protobuf compilation in Test
Test / PB.protoSources += PB.externalSourcePath.value
+// Skipped with a warning if protoc or the betterproto plugin is missing.
Review Comment:
Currently, it is calling an external .sh bash file. Is it possible to embed
Python protocol gen inside here, so that we don't need to maintain a separate
gen bash script?
##########
bin/computing-unit-master.dockerfile:
##########
@@ -36,20 +36,34 @@ COPY project/ project/
COPY build.sbt build.sbt
COPY .jvmopts .jvmopts
-# Update system and install dependencies. python3-minimal is needed by
-# bin/licensing/concat_license_binary.py below.
+# python3-minimal is needed by bin/licensing/concat_license_binary.py
+# below; python3-pip + curl are for the protoc + betterproto[compiler]
+# install below.
RUN apt-get update && apt-get install -y \
netcat \
unzip \
+ curl \
libpq-dev \
python3-minimal \
+ python3-pip \
&& apt-get clean
+# protoc 3.19.4 (matches PB.protocVersion in amber/build.sbt) and the
+# betterproto plugin are required by the genPythonProto sbt task so the
+# generated amber/src/main/python/proto/ tree is populated before the
+# dist is packaged.
+RUN curl -fsSL -o /tmp/protoc.zip
https://github.com/protocolbuffers/protobuf/releases/download/v3.19.4/protoc-3.19.4-linux-x86_64.zip
\
+ && unzip -o /tmp/protoc.zip -d /usr/local \
+ && chmod +x /usr/local/bin/protoc \
+ && rm /tmp/protoc.zip \
+ && pip3 install --no-cache-dir 'betterproto[compiler]==2.0.0b7'
+
# Add .git for runtime calls to jgit from OPversion
COPY .git .git
COPY LICENSE NOTICE DISCLAIMER ./
COPY licenses/ licenses/
COPY bin/licensing/ bin/licensing/
+COPY bin/python-proto-gen.sh bin/python-proto-gen.sh
Review Comment:
I want to remove dependency on this script. If we can embed into build.sbt,
we don't need to modify dockerfiles
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
