[
https://issues.apache.org/jira/browse/HBASE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Smith updated HBASE-765:
-----------------------------
Description:
Spring can configure classes/object graphs via xml. I am pretty much able to
configure the entire MR object graph to launch MR jobs via spring except class
IndexConfiguration.java. So instead of only using addFromXML() to configure
IndexConfiguration, it would be nice to add support so Spring could set all
class variables needed for initialization in IndexConfiguration without
invoking addFromXML().
Since the class IndexConfiguration already has setters and getters for almost
all its members, it's almost compliant for a spring configuration bean except
one issue: no ability to configure columnMap outside of calling addFromXML().
The easiest way i can figure is to allow a setter for the column map and put
any logic for checking the map integrity there. By adding a few methods to
IndexConfiguration.java , it should solve the issue.
I will be submitting a patch for it here shortly.....
EDIT: after further review, there are 2 more classes that are using addFromXML()
IndexOutputFormat and IndexTableReduce use the xml string to configure the
indexconfig object.
I will have to modify these 3 classes so I will be developing the patch over
this week, maybe next week too.
EDIT 2: It seems that the distributed nature of Hadoop's MapReduce api is
getting in the way of spring loading these classes.
Since a new instance of new IndexConfiguration() needs to be called remotely,
(on each node in the cluster running the MR job), it makes it hard to have
spring create the class when ultimately, Hadoop has to create the class for
each task, etc.
I am going to submit changes to hadoop issue 2866.
http://www.mail-archive.com/[EMAIL PROTECTED]/msg18574.html
Or maybe create a new issue for it.
There is also the possibility of getting spring to support map reduce natively.
Ill ask them about it, but for now, I would say that this should go on the
back burner, or just keep punting it down until remote objects can get created
via factory interfaces instead of reflection internally as it is now.
was:
Spring can configure classes/object graphs via xml. I am pretty much able to
configure the entire MR object graph to launch MR jobs via spring except class
IndexConfiguration.java. So instead of only using addFromXML() to configure
IndexConfiguration, it would be nice to add support so Spring could set all
class variables needed for initialization in IndexConfiguration without
invoking addFromXML().
Since the class IndexConfiguration already has setters and getters for almost
all its members, it's almost compliant for a spring configuration bean except
one issue: no ability to configure columnMap outside of calling addFromXML().
The easiest way i can figure is to allow a setter for the column map and put
any logic for checking the map integrity there. By adding a few methods to
IndexConfiguration.java , it should solve the issue.
I will be submitting a patch for it here shortly.....
EDIT: after further review, there are 2 more classes that are using addFromXML()
IndexOutputFormat and IndexTableReduce use the xml string to configure the
indexconfig object.
I will have to modify these 3 classes so I will be developing the patch over
this week, maybe next week too.
> Adding basic Spring DI support to IndexConfiguration class.
> -----------------------------------------------------------
>
> Key: HBASE-765
> URL: https://issues.apache.org/jira/browse/HBASE-765
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.16.0, 0.1.0, 0.1.1, 0.1.2, 0.1.3
> Environment: n/a
> Reporter: Ryan Smith
> Priority: Minor
> Fix For: 0.19.0, 0.2.2
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Spring can configure classes/object graphs via xml. I am pretty much able to
> configure the entire MR object graph to launch MR jobs via spring except
> class IndexConfiguration.java. So instead of only using addFromXML() to
> configure IndexConfiguration, it would be nice to add support so Spring could
> set all class variables needed for initialization in IndexConfiguration
> without invoking addFromXML().
> Since the class IndexConfiguration already has setters and getters for almost
> all its members, it's almost compliant for a spring configuration bean except
> one issue: no ability to configure columnMap outside of calling addFromXML().
> The easiest way i can figure is to allow a setter for the column map and put
> any logic for checking the map integrity there. By adding a few methods to
> IndexConfiguration.java , it should solve the issue.
> I will be submitting a patch for it here shortly.....
> EDIT: after further review, there are 2 more classes that are using
> addFromXML()
> IndexOutputFormat and IndexTableReduce use the xml string to configure the
> indexconfig object.
> I will have to modify these 3 classes so I will be developing the patch over
> this week, maybe next week too.
> EDIT 2: It seems that the distributed nature of Hadoop's MapReduce api is
> getting in the way of spring loading these classes.
> Since a new instance of new IndexConfiguration() needs to be called remotely,
> (on each node in the cluster running the MR job), it makes it hard to have
> spring create the class when ultimately, Hadoop has to create the class for
> each task, etc.
> I am going to submit changes to hadoop issue 2866.
> http://www.mail-archive.com/[EMAIL PROTECTED]/msg18574.html
> Or maybe create a new issue for it.
> There is also the possibility of getting spring to support map reduce
> natively. Ill ask them about it, but for now, I would say that this should
> go on the back burner, or just keep punting it down until remote objects can
> get created via factory interfaces instead of reflection internally as it is
> now.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
