[
https://issues.apache.org/jira/browse/FLUME-1180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269032#comment-13269032
]
Arvind Prabhakar commented on FLUME-1180:
-----------------------------------------
+1.
Thanks for the patch Mubarak.
> ChannelSelectorFactory creates incorrect selector for multiplexing selector
> type
> --------------------------------------------------------------------------------
>
> Key: FLUME-1180
> URL: https://issues.apache.org/jira/browse/FLUME-1180
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Reporter: Mubarak Seyed
> Labels: newbie
> Attachments: FLUME-trunk-1180.patch
>
>
> loadSources() of PropertiesFileConfigurationProvider creates a config with
> "." for key values and ChannelSelectorFactory tries to get value for key
> "type", which defaults to ReplicatingChannelSelector as "type" will not be
> found in config.
> {code}
> agent.sources = avro-source1
> agent.sinks = hdfs-sink1 hdfs-sink2
> agent.channels = mem-channel-1 mem-channel-2
> #source
> agent.sources.avro-source1.channels = mem-channel-1 mem-channel-2
> agent.sources.avro-source1.type = avro
> agent.sources.avro-source1.bind = 0.0.0.0
> agent.sources.avro-source1.port = 9090
> agent.sources.avro-source1.selector.type = multiplexing
> agent.sources.avro-source1.selector.header = tableName
> agent.sources.avro-source1.selector.mapping.table1 = mem-channel-1
> agent.sources.avro-source1.selector.mapping.table2 = mem-channel-2
> agent.sources.avro-source1.selector.default = mem-channel-1
> #sinks
> agent.sinks.hdfs-sink1.channel = mem-channel-1
> agent.sinks.hdfs-sink2.channel = mem-channel-2
> agent.sinks.hdfs-sink1.type = hdfs
> agent.sinks.hdfs-sink1.hdfs.batchSize = 1000
> agent.sinks.hdfs-sink1.hdfs.txnEventMax = 1000
> agent.sinks.hdfs-sink1.hdfs.fileType = DataStream
> agent.sinks.hdfs-sink1.hdfs.rollSize = 1073741824
> agent.sinks.hdfs-sink1.hdfs.rollCount = 0
> agent.sinks.hdfs-sink1.hdfs.rollInterval = 0
> agent.sinks.hdfs-sink1.serializer =
> com.mycompany.Table1AvroEventSerializer$Builder
> agent.sinks.hdfs-sink1.serializer.compressionCodec = snappy
> agent.sinks.hdfs-sink1.serializer.syncIntervalBytes = 2048000
> agent.sinks.hdfs-sink1.hdfs.path = hdfs://NN/logs/table1/%m%d%Y/%H
> agent.sinks.hdfs-sink2.type = hdfs
> agent.sinks.hdfs-sink2.hdfs.batchSize = 1000
> agent.sinks.hdfs-sink2.hdfs.txnEventMax = 1000
> agent.sinks.hdfs-sink2.hdfs.fileType = DataStream
> agent.sinks.hdfs-sink2.hdfs.rollSize = 1073741824
> agent.sinks.hdfs-sink2.hdfs.rollCount = 0
> agent.sinks.hdfs-sink2.hdfs.rollInterval = 300
> agent.sinks.hdfs-sink2.serializer =
> com.mycompany.Table2AvroEventSerializer$Builder
> agent.sinks.hdfs-sink2.serializer.compressionCodec = snappy
> agent.sinks.hdfs-sink2.serializer.syncIntervalBytes = 2048000
> agent.sinks.hdfs-sink2.hdfs.path = hdfs://NN/logs/table2/%m%d%Y/%H
> agent.channels.mem-channel-1.type = memory
> agent.channels.mem-channel-1.capacity = 100000
> agent.channels.mem-channel-1.transactionCapacity = 1000
> agent.channels.mem-channel-2.type = memory
> agent.channels.mem-channel-2.capacity = 100000
> agent.channels.mem-channel-2.transactionCapacity = 1000
> {code}
> but the config for ??selector?? gets parsed as
> {code}
> {.type=multiplexing, .default=mem-channel-1, .mapping.table2=mem-channel-2,
> .mapping.table1=mem-channel-1, .header=tableName}
> {code}
> The fix is in ??loadSources()?? of ??PropertiesFileConfigurationProvider??
> {code}
> Map<String, String> selectorConfig = context.getSubProperties("selector" +
> "."); //fix
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
