cameronlee314 commented on a change in pull request #1256: SAMZA-2420: Update
CommandLine to use config loader for local config file
URL: https://github.com/apache/samza/pull/1256#discussion_r372125185
##########
File path: samza-core/src/main/scala/org/apache/samza/util/CommandLine.scala
##########
@@ -19,55 +19,60 @@
package org.apache.samza.util
-import java.net.URI
-import joptsimple.{OptionParser, OptionSet}
+import joptsimple.{ArgumentAcceptingOptionSpec, OptionParser, OptionSet}
import joptsimple.util.KeyValuePair
-import org.apache.samza.config.{ConfigFactory, MapConfig}
-import org.apache.samza.config.factories.PropertiesConfigFactory
-import scala.collection.mutable.Buffer
+import org.apache.samza.config.{Config, ConfigLoaderFactory, JobConfig,
MapConfig}
+import org.apache.samza.config.loaders.PropertiesConfigLoaderFactory
+
import scala.collection.JavaConverters._
+import scala.collection.mutable
/**
* Defines a basic set of command-line options for Samza tasks. Tools can use
this
* class directly, or subclass it to add their own options.
*/
class CommandLine {
val parser = new OptionParser()
- val configFactoryOpt =
- parser.accepts("config-factory", "The config factory to use to read your
config file.")
+ val configLoaderFactoryOpt: ArgumentAcceptingOptionSpec[String] =
+ parser.accepts("config-loader-factory", "The config factory to use to read
your config file.")
.withRequiredArg
.ofType(classOf[java.lang.String])
.describedAs("com.foo.bar.ClassName")
- .defaultsTo(classOf[PropertiesConfigFactory].getName)
- val configPathOpt =
- parser.accepts("config-path", "URI location to a config file (e.g.
file:///some/local/path.properties). " +
+ .defaultsTo(classOf[PropertiesConfigLoaderFactory].getName)
+ val configLoaderPropertiesOpt: ArgumentAcceptingOptionSpec[KeyValuePair] =
+ parser.accepts("config-loader-properties", "URI location to a config file
(e.g. file:///some/local/path.properties). " +
"If multiple files are given they are all
used with later files overriding any values that appear in earlier files.")
.withRequiredArg
- .ofType(classOf[URI])
+ .ofType(classOf[KeyValuePair])
.describedAs("path")
- val configOverrideOpt =
+ val configOverrideOpt: ArgumentAcceptingOptionSpec[KeyValuePair] =
parser.accepts("config", "A configuration value in the form key=value.
Command line properties override any configuration values given.")
.withRequiredArg
.ofType(classOf[KeyValuePair])
.describedAs("key=value")
- var configFactory: ConfigFactory = null
+ var configLoaderFactory: ConfigLoaderFactory = _
- def loadConfig(options: OptionSet) = {
+ def loadConfig(options: OptionSet): Config = {
// Verify legitimate parameters.
- if (!options.has(configPathOpt)) {
+ if (!options.has(configLoaderPropertiesOpt)) {
Review comment:
In the previous case (i.e. `ConfigFactory`), the `configURI` was a required
argument for the interface, and I don't think you can have an empty `URI`. For
`ConfigLoaderFactory` interface, it is required to pass a `Config`, but it is
possible to have an empty `Config`, so I think you could just remove the check
to handle the general case.
The additional complexity that the `ConfigLoaderFactory` brings is that each
concrete type requires different configs to be in `Config`, so if there is an
error due to a missing config value, then that's maybe where you need to do the
`parser.printHelpOn(System.err); System.exit(-1);`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
