[
https://issues.apache.org/jira/browse/BEAM-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Sisk updated BEAM-2457:
-------------------------------
Description:
I've noticed a number of user reports where jobs are failing with the error
message "Unable to find registrar for hdfs":
*
https://stackoverflow.com/questions/44497662/apache-beamunable-to-find-registrar-for-hdfs/44508533?noredirect=1#comment76026835_44508533
*
https://lists.apache.org/thread.html/144c384e54a141646fcbe854226bb3668da091c5dc7fa2d471626e9b@%3Cuser.beam.apache.org%3E
*
https://lists.apache.org/thread.html/e4d5ac744367f9d036a1f776bba31b9c4fe377d8f11a4b530be9f829@%3Cuser.beam.apache.org%3E
This isn't too many reports, but it is the only time I can recall so many users
reporting the same error message in a such a short amount of time.
We believe the problem is one of two things:
1) bad uber jar creation
2) incorrect HDFS configuration
However, it's highly possible this could have some other root cause.
It seems like it'd be useful to:
1) Follow up with the above reports to see if they've resolved the issue, and
if so what fixed it. There may be another root cause out there.
2) Improve the error message to include more information about how to resolve it
3) See if we can improve detection of the error cases to give more specific
information (specifically, if HDFS is miconfigured, can we detect that somehow
and tell the user exactly that?)
4) update documentation
was:
I've noticed a number of user reports where jobs are failing with the error
message "Unable to find registrar for hdfs":
*
https://stackoverflow.com/questions/44497662/apache-beamunable-to-find-registrar-for-hdfs/44508533?noredirect=1#comment76026835_44508533
*
https://lists.apache.org/thread.html/144c384e54a141646fcbe854226bb3668da091c5dc7fa2d471626e9b@%3Cuser.beam.apache.org%3E
*
https://lists.apache.org/thread.html/e4d5ac744367f9d036a1f776bba31b9c4fe377d8f11a4b530be9f829@%3Cuser.beam.apache.org%3E
This isn't too many reports, but it is the only time I can recall so many users
reporting the same error message in a such a short amount of time.
We believe the problem is one of two things:
1) bad uber jar creation
2) incorrect HDFS configuration
However, it's highly possible this could have some other root cause.
It seems like it'd be useful to:
1) Follow up with the above reports to see if they've resolved the issue, and
if so what fixed it. There may be another root cause out there.
2) Improve the error message to include more information about how to resolve it
3) See if we can improve detection of the error cases to give more specific
information (specifically, if HDFS is miconfigured, can we detect that somehow
and tell the user exactly that?)
> Error: "Unable to find registrar for hdfs" - need to prevent/improve error
> message
> ----------------------------------------------------------------------------------
>
> Key: BEAM-2457
> URL: https://issues.apache.org/jira/browse/BEAM-2457
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Affects Versions: 2.0.0
> Reporter: Stephen Sisk
> Assignee: Davor Bonaci
>
> I've noticed a number of user reports where jobs are failing with the error
> message "Unable to find registrar for hdfs":
> *
> https://stackoverflow.com/questions/44497662/apache-beamunable-to-find-registrar-for-hdfs/44508533?noredirect=1#comment76026835_44508533
> *
> https://lists.apache.org/thread.html/144c384e54a141646fcbe854226bb3668da091c5dc7fa2d471626e9b@%3Cuser.beam.apache.org%3E
> *
> https://lists.apache.org/thread.html/e4d5ac744367f9d036a1f776bba31b9c4fe377d8f11a4b530be9f829@%3Cuser.beam.apache.org%3E
>
> This isn't too many reports, but it is the only time I can recall so many
> users reporting the same error message in a such a short amount of time.
> We believe the problem is one of two things:
> 1) bad uber jar creation
> 2) incorrect HDFS configuration
> However, it's highly possible this could have some other root cause.
> It seems like it'd be useful to:
> 1) Follow up with the above reports to see if they've resolved the issue, and
> if so what fixed it. There may be another root cause out there.
> 2) Improve the error message to include more information about how to resolve
> it
> 3) See if we can improve detection of the error cases to give more specific
> information (specifically, if HDFS is miconfigured, can we detect that
> somehow and tell the user exactly that?)
> 4) update documentation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
