[
https://issues.apache.org/jira/browse/FLUME-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241842#comment-13241842
]
[email protected] commented on FLUME-1052:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4502/
-----------------------------------------------------------
(Updated 2012-03-29 21:58:26.193754)
Review request for Flume.
Changes
-------
Added the dependencies into this patch. Made sure it builds.
Summary
-------
Main config component.
This addresses bug FLUME-1052.
https://issues.apache.org/jira/browse/FLUME-1052
Diffs (updated)
-----
flume-ng-configuration/pom.xml PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/Context.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/ComponentConfigurationFactory.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/ConfigurationException.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfigurationError.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfigurationErrorType.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelSelectorConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelSelectorType.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/channel/ChannelType.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkGroupConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkType.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java
PRE-CREATION
flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
PRE-CREATION
flume-ng-core/pom.xml 37fb112
flume-ng-core/src/main/java/org/apache/flume/ChannelSelector.java fba2dcb
flume-ng-core/src/main/java/org/apache/flume/Context.java 5294e31
flume-ng-core/src/main/java/org/apache/flume/SinkProcessor.java d863ed0
flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b
flume-ng-core/src/main/java/org/apache/flume/channel/AbstractChannel.java
352bf08
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorFactory.java
800f471
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelSelectorType.java
511fc65
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelType.java d8419e8
flume-ng-core/src/main/java/org/apache/flume/channel/DefaultChannelFactory.java
963a6a3
flume-ng-core/src/main/java/org/apache/flume/conf/ConfigurableComponent.java
PRE-CREATION
flume-ng-core/src/main/java/org/apache/flume/conf/Configurables.java 84492e5
flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java
b89dfa0
flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkProcessor.java
257bab3
flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
6160a17
flume-ng-core/src/main/java/org/apache/flume/sink/SinkType.java 6b08c09
flume-ng-core/src/main/java/org/apache/flume/source/DefaultSourceFactory.java
a610e6f
flume-ng-core/src/main/java/org/apache/flume/source/SourceType.java 9082470
flume-ng-core/src/test/java/org/apache/flume/channel/AbstractBasicChannelSemanticsTest.java
6e71e46
flume-ng-core/src/test/java/org/apache/flume/channel/MockChannel.java 24b01e2
flume-ng-node/src/main/java/org/apache/flume/conf/properties/FlumeConfiguration.java
d66f6d1
flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java
50b9f0c
pom.xml c91222f
Diff: https://reviews.apache.org/r/4502/diff
Testing
-------
Functional testing done.
Thanks,
Hari
> Core configuration component
> ----------------------------
>
> Key: FLUME-1052
> URL: https://issues.apache.org/jira/browse/FLUME-1052
> Project: Flume
> Issue Type: Sub-task
> Affects Versions: v1.0.0
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: v1.2.0
>
>
> Currently the configuration provider implementation encompasses all the
> syntactic and structural validation rules for loading the configuration.
> Externalizing this functionality to a library will allow external tools to
> easily operate on flume configuration files and be able to help parse and
> validate these files.
> Currently the configuration of each component sits deep inside the component
> themselves. There is no way to verify if a configuration is valid before run
> time. In most systems like Flume, it is likely that these confs will be
> deployed from one single host, on to the machines where flume agents are
> running. Only when each agent starts, invalid confs are identified because
> the Agents would terminate by throwing exceptions. This is a first attempt to
> make a component-aware configuration system which is independent of the
> Flume, and does not require the Flume jar to be installed. Each component
> needs a configuration manager, which configures the components.
> Provide abstract Configuration stubs for each component type, sources,
> channels, sinks, selectors etc, which are in the new package, independent on
> ng-core. Now for each of the channels extend the abstract class and check the
> config properties for each of the required parameters for that component, for
> example: MultiplexingChannelSelectorConfigurator would look for default
> channel etc. If a particular component does not have a configuration class
> then let the current mechanism continue.
> This will require implementation for each component, but it should not be too
> complex. One additional advantage we get from this is that we can separate
> out the config validation from the components into these stubs, but we will
> still need to read the values out of the Context as we do currently(else we
> end up making the configuration dependent on flume-ng-core itself which we
> wanted to avoid).
> A problem with this is making a change to the configuration would require
> changes in the configuration classes and in the components also(where the
> configuration is read and the component is actually configured). I could not
> figure out a way of avoiding this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
