[
https://issues.apache.org/jira/browse/SAMZA-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892743#comment-15892743
]
ASF GitHub Bot commented on SAMZA-1085:
---------------------------------------
GitHub user fredji97 opened a pull request:
https://github.com/apache/samza/pull/71
SAMZA-1085: adding the samza-yarn dependency on httpClient and bouncyCastle
httpClient-4.4.1 and httpCore-4.4.1 are used for https post request in CSR.
bcpkix-jdk15on-1.54 and bcprov-jdk15on-1.54 from bouncyCastle are used for
json parsing on CSR request/response.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fredji97/samza dependency1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/71.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #71
----
commit d46b5371e8a3c2001d5d346d93617a921ab8fd1f
Author: Fred Ji <[email protected]>
Date: 2017-03-02T18:24:06Z
SAMZA-1085: adding the samza-yarn dependency on httpClient and
bouncyCastle, which are used for https post request and json parsing on CSR
request/response
----
> add a new filesystem for localizing the certificate from CSR in yarn
> --------------------------------------------------------------------
>
> Key: SAMZA-1085
> URL: https://issues.apache.org/jira/browse/SAMZA-1085
> Project: Samza
> Issue Type: New Feature
> Reporter: Fred Ji
> Assignee: Fred Ji
>
> Currently, for the samza jobs running in yarn, there is no support for ACL
> when accessing the dependent service such as Kafka, or any restful service.
> In order to protect the data and isolate the data among the samza jobs, we
> may need to have ACL enabled for these services. For https services
> supporting TLS/SSL, one of the requirements is to have the key store which
> includes the client side certificates for these Samza jobs so when they call
> the service, services know who they are and whether they have access or not.
> When Samza runs on cluster based system such as YARN, the certificate
> distribution could be very challenging because the NM servers do not have app
> specific certificate originally. One of common ways is to have CA which sign
> the certificates for the Samza jobs and then Samza jobs keep these
> certificates in the localized directory and use them for https communication.
> The process of requesting the app specific certificate is called CSR. It is
> mostly a https request to CA, but besides that, it also needs to generate the
> public key and private key pair, and the certificate string needs to put
> together with private key in the local directory for the later app level
> https communication.
> Considering the complicated process of CSR, we are considering create a new
> scheme called certfs, and have a config called "fs.certfs.impl" which map to
> a class for localization. The default class will be CertFSFileSystem, but the
> user can implement other specific class for fs.certfs.impl.
> CertFSFileSystem extends org.apache.hadoop.fs.FileSystem, and majorly do the
> following:
> 1. get the original certfs:// uri;
> 2. reconstruct the https:// uri for csr from certfs://
> 3. generate public/private key pair
> 4. prepare the csr request (ssl context including keystore, truststore, and
> https csr payload)
> 5. send request and fetch the signed certificate
> 6. combine the signed certificate and private key, and localize the key store
> for this specific app
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
