What type of Spark configuration are you trying to augment?
As Romain mentioned, you can supply a custom Spark conf to your Beam
pipeline, but depending on your use case this may not be necessary, and
best avoided.
Also, keep in mind that if you use spark-submit you can add your
configurations
my last email want't clear, please ignore.
Thanks it's looks better (no error or exceptions)
my problem now is how to set Spark conf to my pipeline, this is what i have
?
onf = SparkConf();
conf.setAppName("InsightEdge Python Example")
conf.set("my.field1", "XXX")
conf.set("my.field2", "YYY")
