sijie commented on a change in pull request #4174: [go function] support
localrun and cluster mode for go function
URL: https://github.com/apache/pulsar/pull/4174#discussion_r279926646
##########
File path:
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/RuntimeUtils.java
##########
@@ -89,31 +91,180 @@
if (StringUtils.isNotEmpty(extraDependenciesDir)) {
args.add("PYTHONPATH=${PYTHONPATH}:" + extraDependenciesDir);
}
+ } else if (instanceConfig.getFunctionDetails().getRuntime() ==
Function.FunctionDetails.Runtime.GO) {
+ //no-op
}
return args;
}
+ /**
+ *
+ * Different from python and java function, Go function uploads a complete
executable file(including:
+ * instance file + user code file). Its parameter list is provided to the
broker in the form of a yaml file,
+ * the advantage of this approach is that backward compatibility is
guaranteed.
Review comment:
> I not a GO expert but is there not a flag for the arg parsers in GO to
ignore unknown fields?
Can't answer that since I am not a Go export.
But I doubt there is a such thing in arg parsers in any language. Because as
an args parser, you need to distinguish what are options and what are args.
without a predefined list of options, it is hard to know what are options and
what are args, hence it is hard to know what can be ignored.
> you have to clean it up somehow.
In process mode, you can do File.deleteOnExit. Because the file is only used
during execution time. You don't really need to save it.
> Also this creates difficulties for container based runtimes like K8s. How
shall we pass the file to a container?
1. the yaml file can be save to a configmap.
2. container can mount the configmap as the file
3. parameters that depends on instance (e.g. instance id) can be passed as
environment variables. parameters can be substituted by running the
apply-config script similar to what we do for downloading the codeFile.
> While its not impossible it does add complications.
Every language has its own *complication*. In java, you have to handle
classpath and shading; in Python, you have to handle dependencies, pywheel, and
zip file; In Go, you have to handle the situations introduced due to Go is a
static-linking language. IMO this is not "complication". They are just
different situations introduced by different languages.
A better approach here (in future) is to have a language abstraction. Broker
/ Function worker only passes the function config and necessary metadata to the
language abstraction. Each language chooses its best way to construct the
command to invoke the function. So the language specific details can be hidden
from broker / function worker. Broker / Function worker doesn't care if a
language is passing the function config in a file to the instance or passing
the function config through command args.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
