Looks good and +1 for markdown documentations to provide per release specific information.
On Sat, Oct 21, 2017 at 8:47 AM, larry mccay <lmc...@apache.org> wrote: > New Revision... > > This revision acknowledges the reality that we often have multiple phases > of feature lifecycle and that we need to account for each phase. > It has also been made more generic. > I have created a Tech Preview Security Audit list and a GA Readiness > Security Audit list. > I've also included suggested items into the GA Readiness list. > > It has also been suggested that we publish the information as part of docs > so that the state of such features can be easily determined from these > pages. We can discuss this aspect as well. > > Thoughts? > > *Tech Preview Security Audit* > For features that are being merged without full security model coverage, > there need to be a base line of assurances that they do not introduce new > attack vectors in deployments that are from actual releases or even just > built from trunk. > > *1. UIs* > > 1.1. Are there new UIs added with this merge? > 1.2. Are they enabled/accessible by default? > 1.3. Are they hosted in existing processes or as part of a new > process/server? > 1.4. If new process/server, is it launched by default? > > *2. APIs* > > 2.1. Are there new REST APIs added with this merge? > 2.2. Are they enabled by default? > 2.3. Are there RPC based APIs added with this merge? > 2.4. Are they enabled by default? > > *3. Secure Clusters* > > 3.1. Is this feature disabled completely in secure deployments? > 3.2. If not, is there some justification as to why it should be available? > > *4. CVEs* > > 4.1. Have all dependencies introduced by this merge been checked for known > issues? > > > ------------------------------------------------------------ > ------------------------------------------------------------ > -------------------------- > > > *GA Readiness Security Audit* > At this point, we are merging full or partial security model > implementations. > Let's inventory what is covered by the model at this point and whether > there are future merges required to be full. > > *1. UIs* > > 1.1. What sort of validation is being done on any accepted user input? > (pointers to code would be appreciated) > 1.2. What explicit protections have been built in for (pointers to code > would be appreciated): > 1.2.1. cross site scripting > 1.2.2. cross site request forgery > 1.2.3. click jacking (X-Frame-Options) > 1.3. What sort of authentication is required for access to the UIs? > 1.3.1. Kerberos > 1.3.1.1. has TGT renewal been accounted for > 1.3.1.2. SPNEGO support? > 1.3.1.3. Delegation token? > 1.3.2. Proxy User ACL? > 1.4. What authorization is available for determining who can access what > capabilities of the UIs for either viewing, modifying data and/or related > processes? > 1.5. Is there any input that will ultimately be persisted in configuration > for executing shell commands or processes? > 1.6. Do the UIs support the trusted proxy pattern with doas impersonation? > 1.7. Is there TLS/SSL support? > > *2. REST APIs* > > 2.1. Do the REST APIs support the trusted proxy pattern with doas > impersonation capabilities? > 2.2. What explicit protections have been built in for: > 2.2.1. cross site scripting (XSS) > 2.2.2. cross site request forgery (CSRF) > 2.2.3. XML External Entity (XXE) > 2.3. What is being used for authentication - Hadoop Auth Module? > 2.4. Are there separate processes for the HTTP resources (UIs and REST > endpoints) or are they part of existing processes? > 2.5. Is there TLS/SSL support? > 2.6. Are there new CLI commands and/or clients for accessing the REST APIs? > 2.7. What authorization enforcement points are there within the REST APIs? > > *3. Encryption* > > 3.1. Is there any support for encryption of persisted data? > 3.2. If so, is KMS and the hadoop key command used for key management? > 3.3. KMS interaction with Proxy Users? > > *4. Configuration* > > 4.1. Are there any passwords or secrets being added to configuration? > 4.2. If so, are they accessed via Configuration.getPassword() to allow for > provisioning to credential providers? > 4.3. Are there any settings that are used to launch docker containers or > shell out command execution, etc? > > *5. HA* > > 5.1. Are there provisions for HA? > 5.2. Are there any single point of failures? > > *6. CVEs* > > Dependencies need to have been checked for known issues before we merge. > We don't however want to list any CVEs that have been fixed but not > released yet. > > 6.1. All dependencies checked for CVEs? > > > > > On Sat, Oct 21, 2017 at 10:26 AM, larry mccay <lmc...@apache.org> wrote: > >> Hi Marton - >> >> I don't think there is any denying that it would be great to have such >> documentation for all of those reasons. >> If it is a natural extension of getting the checklist information as an >> assertion of security state when merging then we can certainly include it. >> >> I think that backfilling all such information across the project is a >> different topic altogether and wouldn't want to expand the scope of this >> discussion in that direction. >> >> Thanks for the great thoughts on this! >> >> thanks, >> >> --larry >> >> >> >> >> >> On Sat, Oct 21, 2017 at 3:00 AM, Elek, Marton <h...@anzix.net> wrote: >> >>> >>> >>> On 10/21/2017 02:41 AM, larry mccay wrote: >>> >>>> >>>> "We might want to start a security section for Hadoop wiki for each of >>>>> the >>>>> services and components. >>>>> This helps to track what has been completed." >>>>> >>>> >>>> Do you mean to keep the audit checklist for each service and component >>>> there? >>>> Interesting idea, I wonder what sort of maintenance that implies and >>>> whether we want to take on that burden even though it would be great >>>> information to have for future reviewers. >>>> >>> >>> I think we should care about the maintenance of the documentation >>> anyway. We also need to maintain all the other documentations. I think it >>> could be even part of the generated docs and not the wiki. >>> >>> I also suggest to fill this list about the current trunk/3.0 as a first >>> step. >>> >>> 1. It would be a very usefull documentation for the end-users (some >>> answers could link the existing documentation, it exists, but I am not sure >>> if all the answers are in the current documentation.) >>> >>> 2. It would be a good example who the questions could be answered. >>> >>> 3. It would help to check, if something is missing from the list. >>> >>> 4. There are future branches where some of the components are not >>> touched. For example, no web ui or no REST service. A prefilled list could >>> help to check if the branch doesn't break any old security functionality on >>> trunk. >>> >>> 5. It helps to document the security features in one place. If we have a >>> list for the existing functionality in the same format, it would be easy to >>> merge the new documentation of the new features as they will be reported in >>> the same form. (So it won't be so hard to maintain the list...). >>> >>> Marton >>> >> >> >
