[ https://issues.apache.org/jira/browse/SAMZA-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368047#comment-16368047 ]
ASF GitHub Bot commented on SAMZA-1460: --------------------------------------- GitHub user dnishimura opened a pull request: https://github.com/apache/samza/pull/423 SAMZA-1460: StreamAppender does not explicitly create logging topic Creates the StreamAppender stream explicitly instead of relying on auto stream creation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dnishimura/samza samza-1460-streamappender-create-logging-topic Alternatively you can review and apply these changes as the patch at: https://github.com/apache/samza/pull/423.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #423 ---- commit 9c6437fc1b450c8b60ff6543a180d3b4ae929d4e Author: Daniel Nishimura <dnishimura@...> Date: 2018-02-17T00:27:12Z In StreamAppender, explicitly create the stream in the setup. ---- > StreamAppender does not explicitly create logging topic > ------------------------------------------------------- > > Key: SAMZA-1460 > URL: https://issues.apache.org/jira/browse/SAMZA-1460 > Project: Samza > Issue Type: Bug > Reporter: Yi Pan (Data Infrastructure) > Priority: Major > > In StreamAppender, the logging topic is not explicitly created. We will be > relying on Kafka cluster topic auto-creation feature to create the logging > topic: > {code} > protected void setupSystem() { > config = getConfig(); > SystemFactory systemFactory = null; > Log4jSystemConfig log4jSystemConfig = new Log4jSystemConfig(config); > if (streamName == null) { > streamName = getStreamName(log4jSystemConfig.getJobName(), > log4jSystemConfig.getJobId()); > } > String systemName = log4jSystemConfig.getSystemName(); > String systemFactoryName = log4jSystemConfig.getSystemFactory(systemName); > if (systemFactoryName != null) { > systemFactory = Util.<SystemFactory>getObj(systemFactoryName); > } else { > throw new SamzaException("Could not figure out \"" + systemName + "\" > system factory for log4j StreamAppender to use"); > } > setSerde(log4jSystemConfig, systemName, streamName); > systemProducer = systemFactory.getProducer(systemName, config, new > MetricsRegistryMap()); > systemStream = new SystemStream(systemName, streamName); > systemProducer.register(SOURCE); > systemProducer.start(); > log.info(SOURCE + " has been registered in " + systemName + ". So all the > logs will be sent to " + streamName > + " in " + systemName + ". Logs are partitioned by " + key); > } > {code} > It would be better to explicitly create the logging topic via > SystemAdmin.createStream() to work w/ logging systems that do not support > topic auto-creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)