[ 
https://issues.apache.org/jira/browse/BAHIR-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108601#comment-16108601
 ] 

ASF subversion and git services commented on BAHIR-122:
-------------------------------------------------------

Commit e491610d8a7a9cff1ed5087c1c0cbe6b2c29eb39 in bahir's branch 
refs/heads/master from [~ire7715]
[ https://git-wip-us.apache.org/repos/asf?p=bahir.git;h=e491610 ]

[BAHIR-122][PubSub] Make "ServiceAccountCredentials" really broadcastable

Instead of requiring key files on each instance of the cluster, we read
the key file content on the driver node and store the binary in the
ServiceAccountCredentials. When the provider is called, it retrieves the
credential with the in-memory key file.

Closes #48


> [PubSub] Make "ServiceAccountCredentials" really broadcastable
> --------------------------------------------------------------
>
>                 Key: BAHIR-122
>                 URL: https://issues.apache.org/jira/browse/BAHIR-122
>             Project: Bahir
>          Issue Type: Improvement
>          Components: Spark Streaming Connectors
>            Reporter: Ire Sun
>
> The origin implementation broadcast the key file path to Spark cluster, then 
> the executor read key file with the broadcasted path. Which is absurd, if you 
> are using a shared Spark cluster in a group/company, you certainly not want 
> to (and have no right to) put your key file on each instance of the cluster.
> If you store the key file on driver node and submit your job to a remote 
> cluster. You would get the following warning:
> {{WARN ReceiverTracker: Error reported by receiver for stream 0: Failed to 
> pull messages - java.io.FileNotFoundException}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to