[
https://issues.apache.org/jira/browse/MAPREDUCE-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated MAPREDUCE-6761:
----------------------------------
Attachment: MAPREDUCE-6761.2.patch
Thanks [~rchiang], [~templedf] for the reviews.
Added a TestCluster testcase, to check the ServiceLoader handling
About the for/iterator issue:
First I was leaning more to the iterator version, since when I checked the
source code of the ServiceLoader, the hasNext() only loaded the configuration
files, and the next() instantiated the actual objects (The exception is thrown
only in the next() method). So if there was a line with an error in the
configuration file, it was still possible to get the next element, and continue
with the parsing. If the following lines were correct, those could be used too.
After [~templedf] comment, I checked the documentation too, and found this:
{noformat}
Its hasNext and next methods can therefore throw a ServiceConfigurationError
if a provider-configuration file violates the specified format
[..]
If such an error is thrown then subsequent invocations of the iterator will make
a best effort to locate and instantiate the next available provider, but in
general
such recovery cannot be guaranteed.
{noformat}
Which basically means, it is possible that the getNext could throw an
exception, and the iterator might be stuck - this could cause infinite loop
when trying to iterate through the services.
So based on these information, I decided to stay on the safe side, and do not
use features found in the implementation, but only stated as “best effort” in
the documentation, since these might be changed without any notification. This
means, that the parsing of the configuration files will be done until the first
erroneous element found - as it was before MAPREDUCE-6473. And I can use the
foreach loop with a try/catch to handle the exception.
The problem with the MAPREDUCE-6473 was that when the initProviderList throws
an exception it was not catched at all, so the Providers loaded before the
error ware not used, and the Cluster creation is failed.
Thanks again,
Peter
PS: I love and hate your nitpicking infrastructure "You should put an '.' at
the end of your sentence in the comment" :)
> Regression when handling providers - invalid configuration
> ServiceConfiguration causes Cluster initialization failure
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6761
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6761
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 3.0.0-alpha2
> Reporter: Peter Vary
> Assignee: Peter Vary
> Labels: supportability
> Attachments: MAPREDUCE-6761.2.patch, MAPREDUCE-6761.patch
>
>
> When a rouge org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
> defines a provider that is not on classpath, then the initialization is
> failed with the following exception:
> java.util.ServiceConfigurationError:
> org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider: Provider
> org.apache.hadoop.mapred.YarnClientProtocolProvider not found
> at java.util.ServiceLoader.fail(ServiceLoader.java:239)
> at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.hadoop.mapreduce.Cluster.initProviderList(Cluster.java:84)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:114)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:108)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:101)
> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:477)
> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:455)
> This regression is caused by MAPREDUCE-6473
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
