[
https://issues.apache.org/jira/browse/SAMZA-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13922023#comment-13922023
]
Chris Riccomini commented on SAMZA-171:
---------------------------------------
I'd be thrilled if you'd take on that work. We've been talking about the need
for these docs for some time, but they keep falling below the line. If you have
time to write up something and throw a patch up for it, that would be
fantastic. Let me know if you need any help.
> Http Yarn package path failing
> ------------------------------
>
> Key: SAMZA-171
> URL: https://issues.apache.org/jira/browse/SAMZA-171
> Project: Samza
> Issue Type: Bug
> Reporter: Ethan Setnik
>
> When specifying an http package path for yarn jobs the jobs fail with the
> error:
> 14/03/05 16:28:40 WARN security.UserGroupInformation:
> PriviledgedActionException as:samza (auth:SIMPLE) cause:java.io.IOException:
> No FileSystem for scheme: http
> 14/03/05 16:28:40 INFO localizer.ResourceLocalizationService: DEBUG: FAILED {
> http://s3.amazonaws.com/samza_packages/wikipedia-job-package.tar.gz, 0,
> ARCHIVE, null }, No FileSystem for scheme: http
> 14/03/05 16:28:40 INFO localizer.LocalizedResource: Resource
> http://s3.amazonaws.com/samza_packages/wikipedia-job-package.tar.gz
> transitioned from DOWNLOADING to FAILED
> 14/03/05 16:28:40 INFO container.Container: Container
> container_1394035672475_0003_02_000001 transitioned from LOCALIZING to
> LOCALIZATION_FAILED
> 14/03/05 16:28:40 INFO localizer.LocalResourcesTrackerImpl: Container
> container_1394035672475_0003_02_000001 sent RELEASE event on a resource
> request {
> http://s3.amazonaws.com/samza_packages/wikipedia-job-package.tar.gz, 0,
> ARCHIVE, null } not present in cache.
> yarn.package.path=http://s3.amazonaws.com/samza_packages/wikipedia-job-package.tar.gz
> It looks like some work has already been done to support this feature by
> configuring the "fs.http.imp". I also noticed that this configuration was
> updated in SAMZA-63.
> hConfig.set("fs.http.impl", classOf[HttpFileSystem].getName)
> However my understanding is that the job package itself contains the
> necessary HttpFileSystem class to load http packages, and YARN does not
> support this configuration out of the box so i'm at a loss as to how to load
> a remote package over http.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
