Added: ace/site/trunk/content/docs/analysis/security-analysis.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/analysis/security-analysis.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/analysis/security-analysis.mdtext (added) +++ ace/site/trunk/content/docs/analysis/security-analysis.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,88 @@ +Title: Security Analysis + +Security is an important concern for ACE. The analysis needs to differentiate between the individual needs of each sub-system and the overall flow inside the system. Furthermore, several scenarios need to be taken into account and addressed. In general, safety issues are not part of this analysis but will be addressed separately. + +Threat scenarios and possible countermeasures are given subdivided by and investigated in regard to authentication, authorization, integrity, non repudiation, and confidentiality. We need answers to the following questions, what kind of different "attacks" from both external and internal interfaces can we identify (threats); how can we authenticate the different actors (human and machine) so we really know who we're talking to (authentication); who is allowed to do what in the system (authorization); who did what at which point of time (non repudiation); and how do we encrypt and ensure the integrity of the communication/software/configuration data (confidentiality). + +Security on the target and relay server needs special attention because they are most likely provided by a third party, might be accessible from the outside, and not easily reachable for maintenance. It is for example possible that a target is at a remote location, accessible via the internet, and requires days to be accessed physically. + +Threat Scenarios +================ + +This analysis focuses on the OSGi framework and management agent part of the system and its interaction with a (relay) server as well as between the client (for this analysis we assume the client is a separate node, for our web based UI it just happens to be part of the server) and a server. The most likely scenarios are forced breakdown of the system (denial of service attack), malicious data that might change system behavior, attempts to take over control, and espionage. + +1. (D)DOS - In general, it is not possible to prevent denial of service attacks. Attackers normally can find a way to overload the system. Regarding the management agent it would be for example possible to provide the agent with a huge amount of data to install so that the target either is running out of disk space or out of other processing resources. The same is possible for any other entity in the system if an attacker finds a way to make it accept data. +2. Malicious Data - An attacker might use malicious data as part of a DOS attack but it could be also used to gain control over the system or change some aspects of its behavior to make it easier to take over control or cause other harm. +3. Hostile Takeover - Attackers might be interested in taking control over (parts of) the system in order to either do espionage, change the behavior of the system to do work for them, or plainly destroy/disable entities (e.g., to harm competitors). +4. Eavesdropping - An attacker might be able to listen in on the communication between a target and its (relay-) server or the client and the server. This might allow to learn about the configuration of a target and getting hold of the installed software. +5. Physical Access - Another type of attack would be to gain physical access e.g., disassemble a target or a relay server in an attempt to steal its data and/or impersonate it. Probably the only way to avoid that is hardware encryption, which for ACE is out of scope (but can be used to further harden the system). + +Countermeasures +=============== + +On the target there are two entities that are important namely, the (relay) server which is providing the target with instructions and data/code, and the management agent (i.e., the target itself). Regarding the communication between a client and the server the secure checkout and commit of object repository versions are important as well as the auditlog. The interaction between the server and a relay server is a two way data exchange where the relay server is comparable to a target in regard to the instructions and data/code it needs to get from the server and to a server that sends the auditlog to a client. One plus point from the security side is that the target is only polling the server â hence, it is not accepting any connection requests from the outside. This reduces the risk of a DOS attack but by no means makes it invulnerable against it (especially since there is a high likelihood that the underlying platform is vulnerable to DOS attacks as well). One way of working a round the polling restrictions are ARP and DNS injection attacks that might make the target contact the wrong server. This allows for malicious data, DOS attacks, and hostile takeovers. + +A good start to limit attack possibilities is to decouple the sub-net of the target from the internet / external world by using relay servers but this doesn't prevent the mentioned attacks and threats in all cases. Furthermore, relay servers need to support both polling and being polled due to their different roles (they are polled by the targets, need to poll deployment packages or object repositories from the server, and push the auditlogs of targets to the server). Finally, the server is only polled. + +### Authentication + +As mentioned above, the most likely way of attacking a target or relay server is to spoof its connection to the server (whether it is a relay server or the real one). It is dangerous to rely on DNS and/or IP addresses because both might be wrong. Given the issues at stake, authentication will need to be based on certificates. An entity of the system should have a certificate (that has the id as part of it's common name) as its identity. + +Furthermore, it needs to have a keystore of trusted root certificates (CA) and a certificate revocation list (CRL). The (relay) server needs to have a certificate as its identity that is part of a chain of trust to one of the trusted root certificates of the target or client and vice versa. Basically, this can be achieved via two ways, one is to use https with server and client certificates; the other to use certificates to sign all messages/data using our own protocol. + +### Authorization + +We have to differentiate between several areas where authorization is needed. The provisioning part needs to make sure it is installing deployment packages from an authorized server. + +The target itself is running an OSGi framework and can subsequently, make use of the built-in security. This is needed if deployed software components can not be trusted and would be advisable to foster "least privilege" security in general. However, the management agent will need to be able to cooperate with the framework infrastructure to set-up needed rights. Special care needs to be taken to avoid installing malicious software in a framework with security disabled or with too powerful a set of rights. Due to the life-cycle capabilities of OSGi, a malicious or faulty bundle could for example uninstall the management agent itself if the bundle is started in the absence of security or with admin permission (This aspect is not part of this analysis and will be discussed as a separate user story). + +Assuming the additional requirements in regard to integrity and authentication are satisfied it should be sufficient to ensure the server is authorized to make changes to the target â hence, in a certificate based approach separate chains of trust can be used to determine whether a server is trusted and is authoritative for a given target. In other words, the certificate of the server can be treated as a capability (revocation is then possible via a certificate revocation list). The same applies for clients and relay servers, respectively. + +### Integrity + +Due to the fact that authorization to provision a given version (i.e., a set of bundles) is mainly based on whether or not the current authenticated server is authoritative for a target it is of great importance that the actual deployment package has not been tampered with. + +The deployment admin specification already defines a way to ensure integrity building upon the fact that deployment packages are Java JAR files (which can be signed). Therefore, it makes sense to only allow deployment packages that are signed by a certificate that the target has in a chain of trust. + +Furthermore, taking into account relay servers the trusted certificates can be limited further to for example only allow the actual server certificate. + +Deployment packages can be signed by any number of certificates so it is possible to sign a deployment package multiple times in order to make it available to different targets that follow non uniform certificate trust strategies. The same is possible for the object repositories and the auditlog. + +### Non Repudiation + +Several entities can be responsible for changes in the system. The individual entities need to make sure they record in a non repudiation fashion who was doing what for any action taken. Conversely, the server and possibly the relay servers need a way to ensure that for example auditlog entries are really from the target they are claimed to be. + +One way to tackle this is to use certificates to sign all data and to make sure that for all data accepted from a different entity, the signature (including the fingerprint of the signing certificate) is recorded. Taking the auditlog as an example, a target would use its certificate to sign all entries in the auditlog. Subsequently, a server or a client can be certain that a given auditlog is originating from the target it is claimed to come from (assuming the private key of the target certificate has not been exploited). + +Furthermore, it will be easy to invalidate data from compromised entities by adding their certificates to the certificate revocation list. + +Another, more involved example, can be a target that receives a deployment package and installs it. In this case, the manifest containing all the signatures of the content of the signed deployment package as well as all the fingerprints of the certificates that signed it need to be added to the targets auditlog and this entry would be signed by the target certificate. After the log is synchronized back to the server (possibly via several relay servers or even manually) the server can determine who signed the deployment package and where it has been installed. The same applies for clients. + +### Confidentiality + +In most cases the software that needs to be provisioned as well as the configuration of the targets needs to be kept confidential since it may contain business secrets. This can only be ensured by means of encryption because we have to take scenarios into account where communication happens via a none secure channel like the internet. + +One secure set-up would be to use asynchronous encryption which would furthermore not rely on a point-to-point protocol but rather enable all the way confidentiality. Alas, the deployment packages might be big and asynchronous encryption would be to slow in this case. + +The alternative is to use SSL (most likely by means of HTTPS). The downside of SSL as for example in HTTPS is that it is often hard to set-up and relatively inconvenient and static to use if the possibility of a man in the middle attack needs to be ruled out. + +Possibly the biggest problem, in our scenario, is that we can not assume that the common name of an entity reflects its IP/DNS name. Relay servers might be operating in networks not under the customers or our control and the same applies to targets and clients (which could have dynamic IP's and hostnames for example). This problem can be overcome by ignoring the common name in regard to authentication which might make it necessary to create some integration code for certain platforms and containers (e.g., the JVM, by default, assumes that it can resolve the common name as a host name). The downside is that such an approach would open the possibility for man in the middle attacks. Only in combination with client certificates this can be prevented (alas, this might need some more adaption on the server side). + +Finally, the certificates on both, the server and the target side, respectively, would need to be in a chain of trust. Assuming this precondition holds, the only way to eavesdrop would then be to exploit one of the certificate's private key (e.g., via disassembling the target by an attacker that has physical access or by means of gaining access to the target via a different vulnerability). Such a key could be blacklisted by adding it to the certificate revocation list upon discovery of its exploitation. + +### Encryption + +The physical access threat makes it possible that attackers might get hold of data (like installed bundles). Https and certificates can prevent eavesdropping while data is distributed but if an attacker can get hold of the target or a relay server it is still possible to access the data. As mentioned above, for the target the only way to prevent this would be hardware supported encryption but for relay servers it is sufficient to encrypt the data itself. We might need to support this eventually but it is not looked into further in this analysis. + +Certificate based Flow Analysis +=============================== + +All entities (the server, the client, the relay server, and the target), have a CRL and a keystore; the former contains revoked certificates and the later the known and trusted certificate authorities. In general, for all involved certificates, for a certificate to be valid it has to be the case that it is in a chain-of-trust relation to at least one of the trusted certificate authorities and is not revoked. Furthermore, there exists a special trusted certificate known as the server authority and vice versa for the target and client. The interaction between the entities is via HTTPS and needs a valid server and client certificate. The common name of the certificate represents the target, client, or server id, respectively. As a further restriction the server certificate has to be in a chain of trust to the server certificate authority, the client certificate has to be in a chain of trust to the client certificate authority, and the target certificate has to be in a chain of trust to the target certificate authority. The data exchanged between the entities needs to be signed by the respective counterpart certificate authority. For example, a deployment package send from the server to the target needs to be signed by a valid certificate that is in a chain of trust to the server certificate authority and auditlog entries send from the target to the server must be signed by its target certificate. In other words, the signer needs to be the one that created the specific data. CLR and keystore can be treated as yet another object repository (because they need to be signed) â hence, they can be synced from a server to clients, relay servers, and subsequently, targets. + +<object data="security-analysis-flow.svg" type="image/svg+xml" class="span12" height="868"></object> + +Conclusion +========== + +The set-up takes aforementioned countermeasure to the identified threat into account. The https connection ensures the confidentiality via encryption. Due to the server and client certificate connection authentication and authorization are addressed. The requirement of separately signed content provides integrity and non repudiation in the absence of compromised certificate private keys. Certificates with known exploited keys can be revoked by adding them to the CRLs. Authority derives from the chain of trust relation to the server and target certificate authority. +
Added: ace/site/trunk/content/docs/analysis/src/security-analysis-flow.graffle URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/analysis/src/security-analysis-flow.graffle?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/analysis/src/security-analysis-flow.graffle ------------------------------------------------------------------------------ svn:mime-type = application/xml Added: ace/site/trunk/content/docs/analysis/template-mechanism.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/analysis/template-mechanism.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/analysis/template-mechanism.mdtext (added) +++ ace/site/trunk/content/docs/analysis/template-mechanism.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,36 @@ +Title: Template Mechanism + +Some artifacts (see Object Graph in Client) can need some customization before being provisioned, e.g. configuration files might need some information that is managed by one of the distributions. + +The customization will be done when a new version is created, i.e., on call of `approve()` on a StatefulTargetObject. A customized version of the artifact (which is located somewhere in an OBR, reachable using a URL) is uploaded to the same OBR, and the URL to the customized one is stored in the DeploymentVersionObject. + +Proposed design +=============== + +In addition to the interfaces ArtifactHelper and ArtifactRecognizer, we introduce a ArtifactPreprocessor, which has a single method `preprocess(ArtifactObject object, Properties props)`, in which Properties contains customization information (see below), and the method returns the URL of the altered artifact (or, if nothing has changed, the original artifact, or, if this changed artifact is identical to one that has already been created before, that old URL). This ArtifactPreprocessor can be published as a service (see the section on remoting below), but for local purposes, the ArtifactHelper interface gets an extra method `getPreprocessor()`, which returns an instance of the preprocessor to be used for the type of artifact this helper helps. + +As an added service, we could create a basic preprocessor, VelocityBasedPreprocessor which uses the Velocity template engine to process an artifact and store it in a configured OBR; this preprocessor can be instantiated and returned by each ArtifactHelper that needs a basic processor (if no processing can be done for some type of artifact, `getPreprocessor` should return null). + +### Customization information + +For each template that has 'holes' to fill in, it can 'reach' all RepositoryObjects that are reachable from the TargetObject this template will be provisioned to, leading to a tree of data. Inspired by Velocity's way of finding contextual data, we propose to store the for each RepositoryObject in its own Properties object, adding its attributes and tags to it as two Properties objects using the keys "attributes" and "tags", and a List<Properties> summing up all children (so, for a target, all its distributions) using the key "children"; in the end, this becomes a tree of Properties objects. + +This way, the Velocity template can use syntax like + + #foreach( $license in $gateway.children) + #if ($license.attributes.vendor=="luminis") + Default license by luminis + #else + Custom license by $license.attributes.vendor + #end + #end + +### Support for remoting + +Some customers might want to keep all information hidden from us, only allowing us the metadata on the server. In this case, we can deploy a ArtifactPreprocessor on the customer's site, which is then responsible for doing everything a local ArtifactPreprocessor can do, and returning a URL to the altered artifact. Then, in stead of returning an instance of the ArtifactPreprocessor, the ArtifactHelper will return some RemoteArtifactPreprocessor which implements the ArtifactPreprocessor interface, but talks to a servlet on the customer's server. + +### On the 'needsApprove' state in the StatefulTargetObject + +With the mechanism above, `determineStoreState` in StatefulTargetObject would need to create a full deployment version every time we need to know whether approval is necessary. This is undesirable, because, in a remoting scenario, it means we have to pass lots of data to a servlet, oftentimes only to find out that we created a version identical to the one we already had. +So, in stead of this rigid semantics, the 'needsApprove' state will become more of a 'tainted' state, which becomes true when something happens that could have an impact on this StatefulTargetObject. We can quite easily determine what targets are affected by a given change in the model by following the associations from that object to the targets. + Added: ace/site/trunk/content/docs/architecture.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/architecture.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/architecture.mdtext (added) +++ ace/site/trunk/content/docs/architecture.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,125 @@ +Title: Architecture + +Introduction +============ + +Apache ACE is a system that can be used to provision software to OSGi based and other targets. It manages the life cycle of bundles and provides a controlled, centralized way to install, update and uninstall software and related artifacts such as configuration data, native code and device firmware. + +### Purpose and scope + +The purpose of this document is to provide a comprehensive overview of the software architecture of the system. It uses a number of different views to depict the important aspects of the system and captures and conveys the significant architectural decisions which have been made on the system. + +It serves three purposes: + +1. *Abstraction of the system.* The architecture provides a relatively small, intellectually graspable model of how the system is structured and how components interact. +2. *Mutual communication.* The model described here forms the common basis for all of the stakeholders to communicate with each other and form consensus about the system. +3. *Major design decisions.* The description of the architecture represents the earliest set of design decisions that have a significant impact on the system as a whole. They are relevant to ensure the qualities and features of the architecture. + +### Readers' guide + +The document starts with an introduction, outlining the scope and purpose of this document and defining some acronyms and abbreviations. It then goes on by first sketching the architectural context, exploring the domain and the constraints. That is followed by the architectural design, which uses the 4+1 view as a guide to describe the system. + +All stakeholders should read the architectural context, as it explains what the system does and how it interacts with its surroundings. + +Software engineers in particular should read the architectural design, which outlines the foundation and describes the high level design that forms the basis for further analysis and design. + + + +### Definitions, acronyms and abbreviations + +Definition | Explanation +------------ | -------------------------------------------------- +OSGi | The OSGi alliance is an independent non-profit organization that maintains the OSGi standard. OSGi technology provides a service-oriented, component-based environment for developers and offers standardized ways to manage the software life cycle. | +Provisioning | Software provisioning is the process of installing and updating software. +Target | A target, or OSGi gateway, is a computer or device that has an OSGi framework installed. +REST | Representational State Transfer (REST) is a style of software architecture for distributed hypermedia systems. The term was introduced in the doctoral dissertation in 2000 by Roy Fielding, one of the principal authors of the Hypertext Transfer Protocol (HTTP) specification, and has come into widespread use in the networking community. +JMX | Java Management Extensions provides the tools for building distributed, web-based, modular and dynamic solutions for managing and monitoring devices, applications and service-driven networks. Since Java 5, it is part of the Java SE platform. +SOAP | SOAP is a protocol for exchanging XML-based messages over computer networks, normally using HTTP/HTTPS. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework that more abstract layers can build on. +DIY | An acronym for "do it yourself". + +Architectural Context +--------------------- + +### Domains + +The system consists of the following domains, as shown in the picture below. + +On the left hand side, the user interface and dependency repository are shown. The dependency repository, sometimes referred to in an analogy as the "shop", is mainly concerned with the dependency management domain, effectively linking artifacts to targets through various mechanisms of grouping and filtering based on requirements and capabilities of the individual artifacts. + +On the right hand side, the deployment aspect deals with the actual provisioning of versioned sets of artifacts to targets. + +Finally, on the target itself, the life cycle is monitored and managed by the management agent. + +The OBR does not need to be a part of our system, but it is used both in the dependency and deployment domain. + +The lower part of the image deals with feedback. Feedback is responsible for providing historic data of all changes to the actual life cycle of the target, which is collected in the audit log and synchronized back to the server. + +#### Dependency management + +Whenever you are dealing with collections of artifacts, you want to start grouping them to form logical subsystems. Doing this makes the artifacts more manageable by users that are not intimately familiar with the architecture of the software components that are being deployed. + +As an analogy, we often use the example of IKEA. They create modular furniture, and sell their modules in configurations that make sense to their users. You can order a cupboard that consists of planks, screws and doors, and they give those configurations names. Apache ACE pretty much allows you to do the same thing, which is why we like the "shop" analogy. In ACE we group our artifacts into named distributions, and users can install one or more of these onto a target. + +When installing artifacts together, we also need to make sure this collection actually works together. Each bundle can have dependencies on other bundles, services, packages or even specific hardware or operating systems. These dependencies all need to be managed. + +Within an OSGi framework, there are two layers that feature dependencies: + +1. the module layer, that has package dependencies; +2. the service layer, that has service dependencies. + +Traditionally, in OSGi, a bundle contains enough meta-data to analyze package dependencies and ensure that these can be resolved. Service dependencies, however, are a lot harder to analyze because of the extremely dynamic nature of a service and the fact that there is no meta-data available. Also, modern dependency management frameworks can express dependencies on more than just services, allowing users to have configuration dependencies or even custom dependencies like a dependency on the time of day. + +For other dependencies, such as required screen sizes, or the presence of specific hardware, no meta-data is available in the bundle, so that is one thing we need to add externally. + +Summing it up, it is important to make sure that you first of all have a clear overview of the artifacts and their different deployable distributions and secondly that you end up deploying sets of artifacts that work together well in the environments in which they're deployed. + +#### Deployment + +In short, deployment is responsible for getting software artifacts onto target systems. The general strategy is to have a management agent on each OSGi based target that receives and deploys these artifacts. + +An important aspect of deployment is the actual distribution of artifacts. The provisioning server takes an OBR as its source and artifacts somehow need to find a way to the targets. In real-life scenarios, there often won't be a completely open, two way connection between server and target, so catering for all kinds of scenarios here is important. + +#### Life cycle management + +Life cycle management deals with managing the life cycle of artifacts within the OSGi based target. + +The management agent is responsible for managing these life cycles, when to update and even to figure out what update strategy to implement. The management agent is also responsible for monitoring any changes and reporting those back via the audit log (see "Feedback" below). + +Different types of artifacts might have different life cycles. From a provisioning point of view, each artifact is either present on a target or it is not. Bundles have a more extensive life cycle, where the following states can be identified: + +* installed; +* resolved; +* starting; +* active; +* stopping; +* uninstalled. + +The following state diagram shows these states and their transitions: + +TODO + +The transitions are explained below: + +* *install* - Each bundle starts its life cycle when it is first installed in the OSGi framework. When a bundle is installed it is stored persistently in the framework. +* *start* - As soon as the bundle is started, it will transition through a couple of states. The first step is the resolving of package dependencies. Here the bundle is "wired up" and if that succeeds, it ends up in the resolved state. From there it will go to starting, where the bundle activator gets instantiated and a bundle can become an active entity (it can start threads, initialize, etc.). Finally it ends up in the active state. +* *update* - As soon as a bundle is installed, it can be updated. When a bundle is updated, if it was active, it will be stopped. Subsequently it will have to be resolved again, and started. +* *stop* - When a bundle is active, it can be stopped. It will first go to the stopping state, where it will have to cleanup (basically undo everything it did during starting). It will end up as resolved. +* *uninstall* - When a bundle is no longer needed, it can be uninstalled. That's a final state, from there it can never be started again. + +#### Feedback + +Feedback is responsible for collecting log data and synchronizing it back to the server. By default, all life cycle data is collected in the audit log. This mechanism is extensible and can support domain specific extensions. + +An audit log is a full historic account of all events that are relevant for a certain target. In this case, it provides historic data of all changes to the actual life cycle of the target, both triggered by the management agent and by other mechanisms on the target (for example, it also shows when the framework itself was started and stopped). + +Architectural Design +-------------------- + +### Architectural foundation + +#### Service oriented, component based architecture + +#### Principles + +### Use Case View + Propchange: ace/site/trunk/content/docs/architecture.mdtext ------------------------------------------------------------------------------ svn:eol-style = native Added: ace/site/trunk/content/docs/auth_api.svg URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/auth_api.svg?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/auth_api.svg ------------------------------------------------------------------------------ svn:mime-type = image/svg+xml Added: ace/site/trunk/content/docs/auth_connectionfactory.svg URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/auth_connectionfactory.svg?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/auth_connectionfactory.svg ------------------------------------------------------------------------------ svn:mime-type = image/svg+xml Added: ace/site/trunk/content/docs/auth_main_components.svg URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/auth_main_components.svg?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/auth_main_components.svg ------------------------------------------------------------------------------ svn:mime-type = image/svg+xml Added: ace/site/trunk/content/docs/coding-standards.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/coding-standards.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/coding-standards.mdtext (added) +++ ace/site/trunk/content/docs/coding-standards.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,492 @@ +Title: Coding Standards + +This is a Java coding style guide for the Apache ACE project. + +[TOC] + +Summary +------- + +This style guide is intended to help the computer professional produce better Java programs. It presents a set of specific guidelines for using the features of Java in a disciplined manner. The goal is to develop high quality, reliable, reusable, portable software. For a number of reasons, no programming language can ensure the achievements of these desirable objectives on its own. Programming must be embedded in a disciplined development process that addresses a number of topics in a well managed way. The use of Java is one of those. It must conform to good programming practice based on well established software engineering principles. This style guide is intended to bridge the gap between these principles and the actual practice of programming in Java. + +Clear, readable, understandable source text eases program evolution, adaptation and maintenance. First, such source text is more likely to be correct and reliable. Second, effective code adaptation is a prerequisite to code reuse, a technique that has the potential for drastic reductions in system development costs. Easy adaptation requires thorough understanding of the software, and that is facilitated considerably by clarity. Finally, since maintenance (really evolution) is a costly process that continues throughout the life of a system, clarity plays a major role in keeping maintenance costs down. Over the entire life cycle, code has to be read and understood far more often than it is written; the investment of effort in writing readable, understandable code is thus well worthwhile. Many of the guidelines in this style guide are designed to promote clarity of the source text. + +This style guide is intended for those involved in the development of real software systems written in Java. Different roles in a software project can exploit the style guide in different ways. The programmer can use it as a reference on good Java style. It can be used in code reviews as a common reference. Finally, lessons learned in real projects can be captured by extending the style guide. + +Class layout and comments +------------------------- + +This chapter describes the layout for classes, interfaces, enums and annotations. These all share a set of common properties, so they will be described together and for readability all called 'classes' here. + +### Files and filenames + +Files longer than 2000 lines are cumbersome and should be avoided. + +#### File names + +The file must be named after the class it represents. As for most cases each file contains only one class, this is an easy naming convention. For nested or inner classes the name of the main class must be the name of the file. As names in Java are case-sensitive, the filename is case-sensitive also. + +#### File organization + +Each Java source file contains a single class or interface. Of course, this excludes inner classes as these must be defined without an (outer) class, and thus in the same file. + +Java source files have the following ordering: + +- beginning comments; +- package and import statements; +- class and interface declarations. + +#### Beginning comments + +Beginning comments are used for licensing and copyright information only. Here at Apache, we embed the ASL 2.0 headers at the top of every file. Note that they are not according to the JavaDoc style (See: How to write doc comments for JavaDoc - Sun Microsystems, Inc.). + +##### Package and import statements + +The first non-comment line of most Java source files is a package statement. After an empty line import statements can follow. For example: + + :::java + package org.apache.ace.core.ui; + + import java.awt.Frame; + import java.io.InputStream; + +A few notes must be made here: + +1. *Package rules.* When not using an explicit package statement in your code the code still is in a package, the default package. This easily results in name clashes and as package naming should be a part of the design, always use an explicit package name. For naming rules of packages see [naming conventions](#namingconventions); +1. *Import statements* need to be explicit in order to overcome name clashes. They must be grouped by name; +1. *Import order.* First in this section should be the standard Java imports like: java.lang.Throwable. Second should be the Java extensions (i.e. javax), third, the third party stuff. Finally the project-specific imports should be added. + +##### Class, interface, enum and annotation declarations + +The following comment block is an example for the comment that belongs to the declaration of a class, interface, enum or annotation. The JavaDoc syntax results in the following block: + + :::java + /** + * Configuration manager. Manages the configuration of an application. Has features + * to import and export whole configurations and notifies components that need to + * receive settings. + */ + +The following table describes the parts of a class, interface, enum or annotation declaration, in the order that they should appear. + +Part of declaration | Notes +------------------- | ---------------- +documentation | According to comment block as shown above. +class, interface, enum or annotation statement | +(static) variables | These should be grouped by functionality rather than by scope. +instance variables | These should be grouped by functionality rather than by scope. +constructors | Start with the default constructor if any. +methods | These methods should also be grouped by functionality rather than by scope or accessibility. E.g. a private class method can be in between two public instance methods. The goal is to make reading and understanding the code easier. When implementing an interface, group the methods that are part of the interface. +inner classes | Are placed at the bottom of the file. + +##### Annotations + +Annotations for classes and methods should be done on the line directly above the class or method. They should be indented to the same level. An example: + + :::java + @Manageable(description = "Starts the system.") + public void start() { + // ... + } + +Annotations for parameters can be inlined like this: + + :::java + public void setValue(@Validation("x > 0 && x < 10", "Should be between 0 and 10.") int x) { + // ... + } + +### Indentation + +Four spaces should be used as unit of indentation. Use spaces or let your editor convert tabs to spaces as some editors might show the tabs different than they were intended! Tabs *must* be set *exactly* every 4 spaces. + +#### Line length + +There is no explicit limit for the length of a line. Make sure that the flow of the code is clear and that, when printing the file, it is well formed when using a reasonable font. + +#### Wrapping lines + +When an expression will not fit on a single line, break it according to these general principles: + +- break after a comma; +- break before an operator; +- prefer higher level breaks to lower level breaks; +- indent the new line with a tab; +- if the above rules lead to confusing code or to code that's squished up against the right margin, please use common sense. + +### Comment + +#### Comment styles + +The Java language supports three different kinds of comments: + +1. single line comments; +1. block comments; +1. JavaDoc comments. + +##### Single line comments + +The compiler ignores everything from `//` to the end of the line. Use this style when adding a description or some kind of explanation on the same line of code or the line above. + + :::java + int a; // acceleration of the car + + // all names that should be searched + String[] names; + +##### Block comments + +The compiler ignores everything from `/*` to `*/`. Use this style for internal comments and copyright headers. + + :::java + /* + * This code is Copyright (c) 2012 Apache Software Foundation. All rights reserved. + * You are not allowed to remember or reproduce anything you read below. + */ + +##### JavaDoc comments + +This indicates a documentation comment (doc comment, for short). The compiler ignores this kind of comment, just like it ignores comments that use `/*` and `*/`. The JavaDoc tool uses doc comments when preparing automatically generated documentation (See: JavaDoc keywords and HTML tags). Note that JavaDoc only uses this documentation when it occurs at an expected position in the file like the class definition or a member declaration. + +These comments are used to provide English descriptions of the classes, interfaces, enums, annotations, methods and the description of data structures and algorithms. These comments should be used at the beginning of each class and before each method. The official JavaDoc guidelines (see references at the end of this document) should be followed, as they provide a good and clear writing style. + +A method block comment looks as follows: + + :::java + /** + * Position the splitter location at a specified position. + * This method can for instance be used when the last position + * is stored as a preference setting for the user. + * + * @param position New position of divider, defined in pixels + * from the left of the containing window. + * @exception org.apache.ace.units.si.exceptions.PositionException Whenever + * an invalid position is passed. + * @see com.sun.java.swing.JSplitPane + */ + public void setSplitterLocation(int position) throws PositionException { + +###### HTML tags + +For class headers, method headers and member variables JavaDoc is used in order to generate API documentation. Some HTML-tags that can be used in order to make the comment blocks more readable: + +Tag | Short description +-------------------- | ----------------- +`<p>` | New paragraph. +`<br>` | Break, a carriage return. For separation of two paragraphs, usage of `<p>` is preferred. +`<ul><li></li></ul>` | Unordered list of items. Each item should start with a `<li>` tag. Most browsers format this as a bullet list. +`<code></code>` | Code samples. Use this when refering to class names, method names, parameter names, etc. + +There is no need to embed the parameter name in the `@param` tag in `<code>` tags; this is done by JavaDoc automatically. The same holds for the exception name in the `@exception` or `@throws` tag. In the clarifying text however, use the `<code>` tags when refering to parameter names etc. The example below shows the `<code>` tag being used for the array parameter in the text, but not in its definition. + +Example: + + :::java + /** + * Prints a range from an object array. The range + * is specified by the first element to print, and + * ranges to the last element of the array. + * + * @param list contains the objects to print + * @param first index of first element in + * the <code>list</code> to print + */ + public void printRange(List<Printable> list, int first) { + +Java syntax and its layout +-------------------------- + +### Declarations + +When declaring a variable or method make the accessibility as restrictive as possible. When using multiple keywords use the following ordering of keywords: + +1. *accessibility* - Start with the accessibility as it makes clear if the method or variable is reachable at all; +1. *static* (if applicable); +1. *final* (if applicable); +1. *return type* (methods only) or type (for variables) - For readability, the type is as close to the name as possible. + +This order is also compatible with the order that is used in Java for the main() method. This results in following sequence: + + :::java + // A familiar one: + public static void main(String[] args) {} + private static String m_lastCreated = null; + private static final int RED = 4711; + +#### Number per line + +One declaration per line is recommended since it encourages commenting and it does not lead to confusing code. It also is more clear about the explicit initialization of variables as discussed in Initialization. + +Example: + + :::java + int level = 0; // level where user enters the system + int horizontalSize = 0; // horizontal size of current level layer + +is preferred over: + + :::java + int level, horizontalSize; // level and size of current level layer + +#### Placement + +In a method, declare local variables just before they are needed. This overcomes the problem of a big list of parameters at the beginning of a method and the use of a variable becomes more clearly in the context of the code, e.g. its initialization. + +#### Initialization + +The initialization of class variables is strictly not necessary because of the default initialization that takes place for these kinds of members. For some types, e.g. Booleans, this requires detailed knowledge of all the default values so it is more clear and explicit to initialize each member. + +Variables that are used and declared within methods must always be initialized explicitly (the compiler will generate an error when you forget this). + +#### Class and Interface Declarations + +When coding Java classes and interfaces, the following formatting rules should be followed: + +- no space between a method and its parameter list; +- `{` appears at the end of the same line as the declaration; +- `}` starts a line by itself indented to match its corresponding opening statement, except when it is a null statement, in which the case the `}` should appear immediately after the `{`. + +Example: + + :::java + public class DefaultStrategy extends Strategy { + private int m_attempts = 0; + + public DefaultStrategy(int attempts) { + super(); + m_attempts = attempts; + } + + void execute() {} + } + +### Statements + +#### Simple statements + +Each line should contain at most one statement. + +#### Compound statements + +Compound statements are statements that contain lists of statements enclosed in braces ("{...}"): + +- The enclosed statements should be indented one more level than the compound statement; +- The opening brace should be at the end of the line that begins the compound statement; the closing brace should begin a line and be indented to the beginning of the compound statement; +- Braces are used around all statements, even single statements, when they are part of a control structure, such as a if-else or for statement. This makes it easier to add statements without accidentally introducing bugs due to forgetting to add braces. + +#### if, if-else, if else-if else statements + +There are a lot of nested possibilities for if-else constructions. All these variations can be programmed in very cryptic ways that easily and often will lead to buggy code. By being more explicit in the used coding style a lot of confusion can be taken away. + +> When using only one statement in a compound block brackets are optional. It is good practice, and therefore required, to always use brackets because mistakes can be made easily when adding a second statement and brackets are forgotten. + +The following example illustrates the correct use of brackets in a few different if-then-else constructions: + + :::java + if (condition) { + statement1; + statement2; + } + else { + statement3; + } + + if (condition) { + statement1; + statement2; + } + else if (condition1) { + statement3; + statement4; + } + else { + statement5; + statement6; + } + +Note that in the example the else if construction is started at a new line so the statement can not be overlooked. + +#### switch + +When using a switch statement use following guidelines: + +- Consider including a default case, unless it would do nothing. The break in the default case is redundant, but it prevents a fall-through error if later another case is added; +- The so-called fall-through construction should be avoided. Only when there are good reasons to use it, make sure that it is very clear that a fall-through is used (comment it). + +The next example shows the sample code that uses the guidelines for a switch statement: + + :::java + switch (condition) { + case A: + statements; + // falls through here, because... + case B: + statements; + break; + default: + statements; + break; + } + +#### try - catch + +A try - catch statement should have the following format: + + :::java + try { + statements; + } + catch (ExceptionClass e) { + statements; + } + +When using finally to add code that always will be executed this will look like: + + :::java + try { + statements; + } + catch (ExceptionClass e) { + statements; + } + finally { + statements; + } + +Note that the catch and the finally start at a new line in order to be compliant to the guidelines for if-then-else statements. + +#### for loops + +New style for loops are generally preferred over old style ones, unless you explicitly need the index, or you have to make the code run on pre-Java 5 virtual machines. + +Old style, a good example that needs the index anyway: + + :::java + // lookup a value in a list, return the index + List<Element> list; + for (int i = 0; i < list.size(); i++) { + if (value.equals(list.get(i)) { + return index; + } + } + +New style, a good example that iterates over a list without any need for an index or type casts: + + :::java + // iterate over a list, printing all values + List<Element> list; + for (Element e : list) { + System.out.println(" - " + e); + } + +### White Space + +#### Blank lines + +Blank lines improve readability by setting of sections of code that are logically related. One blank line should always be used in the following circumstances: + +- between class and interface definitions; +- between methods; +- before a block or single line comment; +- between logical sections inside a method to improve readability. + +#### Blank spaces + +Blank spaces should be used in the following circumstances: + +- A keyword followed by a parenthesis should be separated by a space: + + :::java + while (index > 5) { + // ... + } + + Note that blanks should not be used between a method call and its opening parenthesis. This helps to distinguish keywords from function calls; + +- Blanks should appear after commas in argument lists; + +- All binary and ternary operators except "." should be separated from their operands by spaces. Blanks should never separate unary operators such as unary minus, increment("++") and decrement("--") from their operands: + + :::java + a += c + d; + a = (a + b) / (c * d); + a = (b > c) ? b : c; + xCoord--; + +- The expressions in a for statement should be separated by blanks: + + :::java + for (expr1; cond1; expr2) { + } + +- Casts should be followed by a blank: + + :::java + myInstance.doIt((TreeFrame) frame); + +### Naming conventions {#namingconventions} + +Naming conventions make programs more understandable by making them easier to read. They can also give information about the function of the identifier. + +**(inner) classes, interfaces, enums and annotations** +: Names should be nouns, in mixed case with the first letter of each word capitalised. Try to keep your names simple and descriptive. Use whole words and avoid acronyms and abbreviations.<br/> + Examples:<br/> + `class Raster`<br/> + `class TreeFrame` +**interfaces** +: Like class names, but if there is a name clash, the interface wins.<br/> + Example:<br/> + `interface RemoteRepository extends Repository` +**services** +: Same as interfaces, so don't append "Service" as you usually do not know if an interface is a service or not.<br/> + Example:<br/> + `interface ConnectionFactory` +**implementation classes** +: If a class implements an interface, it should use the name of the interface as part of its name, adding something specific for this implementation to it, or *Impl* if that does not make sense.<br/> + Examples:<br/> + `class FileBasedRepository implements Repository`<br/> + `class VersionServlet implements HttpServlet` +**exceptions** +: Like class names; always ending in "Exception".<br/> + Example:<br/> + `InputException` +**methods** +: Methods should be verbs in mixed case with the first letter lowercase. Within each method name capital letters separate words. Property methods or get-set methods are used as follows: + When a method is used to get a value start the method name with 'get'. When a method is used to set a value start the method name with 'set'. When a method returns a boolean start the method name with 'is'.<br/> + Examples:<br/> + `run();`<br/> + `runFast();`<br/> + `setBackground();`<br/> + `isOptional();` +**variables (except for (constant) static final variables and member variables)** +: All variables are in mixed case with a lowercase first letter. Words are separated by capital letters.<br/> + Examples:<br/> + `int index;`<br/> + `float myWidth;` +**member variables** +: The same capitalisation as for normal variables prefixed with 'm_'.<br/> + Examples:<br/> + `int m_index;`<br/> + `float m_myWidth;` +**constant (static final) variables, enum names** +: Names should be all uppercase with words separated by underscores ('_').<br/> + Example:<br/> + `public static final int MAX_LIMIT = 99;` +**packages** +: Lowercase only; avoid lengthy package names; always start with *org.apache.ace*.<br/> + Example:<br/> + `package org.apache.ace.demo.bundle;` + +Downloads +--------- + +For various coding style checkers and IDE's we have configuration files that support this style guide. You can download them from the list below: + +- [Checkstyle configuration](http://svn.apache.org/repos/asf/ace/trunk/etc/style-guide/checkstyle/) + +References +---------- + +- Java Code Conventions - Sun Microsystems, Inc., <http://java.sun.com/docs/codeconv/>; +- How to Write Doc Comments for JavaDoc - Sun Microsystems, Inc., <http://java.sun.com/j2se/javadoc/writingdoccomments/>; +- JavaDoc homepage - Sun Microsystems, Inc., <http://java.sun.com/j2se/javadoc/>. Propchange: ace/site/trunk/content/docs/coding-standards.mdtext ------------------------------------------------------------------------------ svn:eol-style = native Added: ace/site/trunk/content/docs/configuring-relay-servers.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/configuring-relay-servers.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/configuring-relay-servers.mdtext (added) +++ ace/site/trunk/content/docs/configuring-relay-servers.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,241 @@ +Title: Configuring and using ACE relay servers + +When using Apache ACE in a large deployment scenario, in which many targets need to be +provisioned, it might be necessary to scale the number of servers with additional relay +servers. How many targets can be handled by a single server depends a bit on the number of +changes that are made to the metadata, and how often each target contacts the ACE server +to get the latest changes. Just to give an impression of what Apache ACE can handle, we +tested a single ACE server, during the development of ACE, with several hundred of +targets, which caused not real problems apart from a significant load on your disk and +CPUs. If you come into a situation in which a single ACE server is not able to cope with +the load of your targets, or you want to replicate your metadata across several servers to +achieve better availability, you can use this document to set up and configure relay +servers. + +[TOC] + +## Overview + +In ACE, each target only talks to a single ACE server at a time. From this perspective, an +ACE server only serves two major tasks: + +1. receiving and storing log information from targets, and; +2. providing deployment packages containing software updates. + +In situations where lots of targets need to be provisioned with software, having only a +single ACE server can impose major scalability and availability issues. To solve this, ACE +can be set up to act as relay server for an upstream (or master) ACE server. This allows a +tree-topology of ACE servers to be created that can distribute the load of many targets +evenly, as denoted in the following image: {#fig1} + + +**Figure 1**: An example server topology, showing multiple servers relaying information from an upstream server. Many other topologies are possible as well. + +Note that target logs need to be propagated to the main ACE server in order to make this +information available for further inspection. For example, a target operator might want to +know the current status of a target, even if this target is not directly served by the +main ACE server. + +An ACE relay server is a stripped down ACE server that only includes the bare minimum +functionality to serve its targets. In addition to accepting log-updates from targets and +provisioning software updates to its targets, it relays log information it received from +its targets to the upstream ACE server and replicates (at least) the <tt>deployment</tt> +repository from the upstream ACE server for its own use. A schematic overview of the +functional blocks in a relay server is shown in figure 2: {#fig2} + + +**Figure 2**: A functional overview of how a relay server functions. + +In the following sections, each of these functional blocks will be explained in more depth +along with how to configure them. The configuration examples are shown as simple key-value +pairs separated with an equals (<tt>=</tt>) sign. Lines starting with a hash (<tt>#</tt>) +are considered comments. + +## Log synchronisation between target and relay + +Like in a single-server scenario, the relay must allow a target to upload its log before +it can be synchronised with the main ACE server. + +To configure the storage of logs from targets on the relay server, we need to instantiate +a log store by supplying the following configuration to the +<tt>org.apache.ace.log.server.store.factory</tt> managed service factory: + + :::properties + # a symbolic name used to reference to this store from other services + name=auditlog + +This will instantiate a new log store service named <tt>auditlog</tt>[^1] that can be used +to store the log information of targets. + +To allow targets to upload their logs to the relay server, a log-servlet needs to be +instantiated. This is done by supplying the following configuration to the +<tt>org.apache.ace.log.server.servlet.factory</tt> managed service factory: + + :::properties + # the symbolic name of the actual log store to store the uploaded logs in + name=auditlog + # the alias/endpoint at which the servlet is registered + org.apache.ace.server.servlet.endpoint=/auditlog + # whether or not to use authentication + authentication.enabled=false + +This will instantiate a servlet that listens on the <tt>/auditlog</tt>[^2] endpoint (for +example, <tt>http://my.relay.server:8080/auditlog</tt>). The given <tt>name</tt> +configuration key is used to lookup the actual log store we've created previously. + +## Log synchronisation between relay and main server + +With the log store and servlet configured and in place, a target is now able to +synchronise its logs with the relay server. However, once uploaded to the relay server, +the logs will not propagate automatically to the main ACE server. A separate +log-synchronisation task is responsible for this. To enable this task, we need to supply +the following configuration to the <tt>org.apache.ace.log.server.task.factory</tt> managed +service factory: + + :::properties + # the symbolic name of the actual log store to synchronise + name=auditlog + # how to synchronise logs, can be 'pull', 'push' or 'pushpull' + mode=push + +This will create a "task" service that will push all log information from the +<tt>auditlog</tt>[^1] store to the main server. The <tt>mode</tt> configuration key +denotes how to synchronise with the main server: + +* <tt>push</tt> synchronises (or "pushes") the logs from the relay server to the main server and is typically used in a relaying situation as described in this document; +* <tt>pull</tt> synchronises (or "pulls") the logs from the main server to the relay server and is typically used to create secondary backup servers and is not further discussed in this document; +* <tt>pushpull</tt> will perform a two-way synchronisation between relay and main server and can be used to support fail-over situations. + +In addition, the log synchronisation task (and other tasks as well, see below) needs to +know what upstream server it should synchronise with. For this information, it uses the +ACE discovery service, which is configured by supplying, for example, the following +configuration to the <tt>org.apache.discovery.property</tt>[^3] managed service: + + :::properties + # what is the URL to the *main* ACE server + serverURL = http://my.main.server:8080 + +## Deployment provider + +The relay needs to have at least a deployment repository[^4]. This repository is a +verbatim copy of the deployment repository on the master server and as such needs to be +replicated periodically. The configuration of the deployment provider consists of three +parts: a repository store, the repository servlet, and a repository provider. + +To create a "slave" deployment repository[^5] store, the following configuration needs to +be supplied to the <tt>org.apache.ace.server.repository.factory</tt> managed service +factory: + + :::properties + # the symbolic name of the repository, should be "deployment" + name=deployment + # the customer name, should be equal to the customer name used on the master server + customer=apache + # indicates that this is a slave/read-only repository + master=false + +To make the repository store accessible through a servlet, we need to supply the following +configuration to the <tt>org.apache.ace.repository.servlet.RepositoryServlet</tt> managed +service: + + :::properties + # the endpoint on which the deployment repository store is accessible + org.apache.ace.server.servlet.endpoint=/repository + # whether or not to enable authentication for this endpoint + authentication.enabled = false + +This will instantiate a servlet that listens on the <tt>/repository</tt> endpoint (for +example, <tt>http://my.relay.server:8080/repository</tt>). + +With the repository store and servlet configured, we can configure the deployment +repository provider, which is used to collect information about deployment artefacts that +should be deployed on a target. The deployment repository provider is configured by +supplying the following configuration to the +<tt>org.apache.ace.deployment.provider.repositorybased</tt> managed service: + + :::properties + # the URL on which the deployment repository store can be accessed + url = http://my.relay.server:8080/repository + # the symbolic name of the repository, should be "deployment" + name = deployment + # the customer name, should be equal to the customer name used on the master server + customer = apache + +## Deployment servlet + +The target is only interested in downloading deployment packages containing the updates +for its software and/or its management agent. To support this, two servlets need to be +instantiated, one for accessing the deployment packages of the target software, and one +for accessing the deployment packages for the management agent. + +To configure the servlet responsible for providing deployment packages of the target +software, we need to supply the following configuration to the +<tt>org.apache.ace.deployment.servlet</tt> managed service: + + :::properties + # the endpoint on which the deployment servlet for software-updates is accessible + org.apache.ace.server.servlet.endpoint=/deployment + # whether or not to enable authentication for this endpoint + authentication.enabled = false + +To configure the servlet responsible for providing agent updates, we need to supply the +following configuration to the <tt>org.apache.ace.deployment.servlet.agent</tt> managed +service: + + :::properties + # the endpoint on which the deployment servlet for agent-updates is accessible + org.apache.ace.server.servlet.endpoint=/agent + # whether or not to enable authentication for this endpoint + authentication.enabled = false + # the OBR used for retrieving the agent software + obr.url = http://my.obr.server:8080/obr/ + +## Task scheduler + +With the synchronisation and repository replication tasks in place, we need to tell the +ACE scheduler to periodically execute these tasks. This is done by supplying the following +configuration to the <tt>org.apache.ace.scheduler.cfg</tt> managed service: + + :::properties + # execute all LogSyncTasks once every 2 seconds... + org.apache.ace.log.server.task.LogSyncTask=2000 + # Synchronise with the master repository every 5 seconds... + org.apache.ace.repository.task.RepositoryReplicationTask=5000 + +The relay server is now completely configured and ready to serve its targets. + +A complete runnable example can be found in the <tt>run-relay</tt> project of the ACE +source repository. This example project starts a relay server on <tt>localhost:8282</tt> +and expects its upstream server to run at <tt>localhost:8080</tt> A list of all required +bundles to run a relay server can be found in the <tt>relay.bndrun</tt> file. + +## Target configuration + +To only thing that a target needs to know about the relay server is its (base) URL. This +can be supplied when starting the management agent on the target by adding the following +argument to its command line: + + :::sh + -Dagent.discovery.serverurls=http://my.relay.server:8080/ + +In fact, the <tt>agent.discovery.serverurls</tt> can take multiple server URLs (separated +by commas) to support a simple form of fail-over, like +<tt>http://first.relay.server:8080/,http://second.relay.server:8080</tt>. + +## Notes + +[^1]: this symbolic name *must* be equal to the name used on the upstream server. By +default, the name <tt>auditlog</tt> is used; + +[^2]: by convention, use the same name for the endpoint as is used for log store; + +[^3]: the property-implementation of the ACE discovery service uses a simple preconfigured +property to "discover" the upstream server. The discovery service allows other, more +sophisticated, implementations to be used instead; + +[^4]: the deployment repository contains all information about the different deployment +packages. Each deployment package is a set of artefacts that can be installed on a target; + +[^5]: a slave repository is a repository that is not allowed to be changed and receives +its content from another (master) repository. + Propchange: ace/site/trunk/content/docs/configuring-relay-servers.mdtext ------------------------------------------------------------------------------ svn:eol-style = native Added: ace/site/trunk/content/docs/deployment_strategy_classdiagram.svg URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/deployment_strategy_classdiagram.svg?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/deployment_strategy_classdiagram.svg ------------------------------------------------------------------------------ svn:mime-type = image/svg+xml Added: ace/site/trunk/content/docs/deployment_strategy_update_seq.svg URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/deployment_strategy_update_seq.svg?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/deployment_strategy_update_seq.svg ------------------------------------------------------------------------------ svn:mime-type = image/svg+xml Added: ace/site/trunk/content/docs/design/auditlog-protocol.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/design/auditlog-protocol.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/design/auditlog-protocol.mdtext (added) +++ ace/site/trunk/content/docs/design/auditlog-protocol.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,49 @@ +Title: Audit Log Protocol + +Audit logs record life cycle changes on targets. As such, an audit log can be used to see changes over time and track what is actually on a target. This design describes the protocol to exchange audit log information. + +Each target has an audit log. If the log somehow gets lost on the target, it will generate a new log with a new unique ID. A log contains entries, where each entry gets a sequence number. + +Protocol +======== + +The audit log protocol consists of three commands. The first command can be used for two parties to exchange information about available audit log entry sequence numbers. The other two commands allow you to send and receive data. + +Querying log information +======================== + +This command can be used to exchange information about available audit log entry sequence numbers. Information is exchanged between either a target and a (relay) server, or a relay server and a server. You ask the other party what sequence numbers it has, either for a specific target or for all targets. The result is a collection of sequence numbers. You can then act on that, sending the other party the entries it's missing and asking for entries you don't have. + +* `GET auditlog/query` - returns a full list of sequence numbers +* `GET auditlog/query?gwid=myid&logid=2007-07-01` - returns sequence numbers for a specific target +* `GET auditlog/query?filter=(vendor='luminis')` - returns sequence numbers for any target that matches the filter + +### About queries + +Queries (for log information or entries) come in three forms: + +1. Without any filter, you simply get everything. +2. With a filter on certain keys, all values of specified keys will have to match. +3. With an LDAP filter, where you can filter on arbitrary keys and use compound expressions and pattern matching. + +Sending log information +======================= + +By sending log information, you're pushing it to the other party. You will probably first have figured out, by querying, what data actually needs to be sent. + +* POST auditlog/send - returns status (ok, not ok) + +The data gets sent in the following format: + + gwid, logid, seqnr, eventnumber, type (, key, value)* + +Data is terminated by '\n' (a new-line). + +Receiving log information +========================= + +Here you're asking the other party for data. As part of the request, you can ask for specific information about one or more targets. If you're not specific, you will get everything. + +* `GET auditlog/receive` - returns list/collection of elements +* `GET auditlog/receive?gwid=myid` - (see querying) +* `GET auditlog/receive?filter=(region='asia')` - ... \ No newline at end of file Added: ace/site/trunk/content/docs/design/index.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/design/index.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/design/index.mdtext (added) +++ ace/site/trunk/content/docs/design/index.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,13 @@ +Title: Design + +## Articles/how to's + +* [Customizing Deployment Strategies](ace-deployment-strategies.html) +* [Configuring ACE authentication](ace-authentication.html) +* [Using client certificates authentication](using-client-certificates.html) + +## Miscellaneous + +* [Remote Interfaces](remote-interfaces.html) +* [Test Script](test-script.html) +* [Audit Log Protocol](auditlog-protocol.html) Added: ace/site/trunk/content/docs/design/remote-interfaces-components.svg URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/design/remote-interfaces-components.svg?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/design/remote-interfaces-components.svg ------------------------------------------------------------------------------ svn:mime-type = image/svg+xml Added: ace/site/trunk/content/docs/design/remote-interfaces.mdtext URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/design/remote-interfaces.mdtext?rev=1641501&view=auto ============================================================================== --- ace/site/trunk/content/docs/design/remote-interfaces.mdtext (added) +++ ace/site/trunk/content/docs/design/remote-interfaces.mdtext Mon Nov 24 22:42:00 2014 @@ -0,0 +1,81 @@ +Title: Remote Interfaces Design + +Summary +======= + +This document describes the generic design of the remote interfaces that can be used to access a system within the ACE topology. The design can be divided into three separate logical components: the connectors, the protocol adapters and session management. + +The connectors provide the interfaces to the system. In general, you can have more than one, for example in ACE we can define a client connector that allows us to talk to the ACE server. + +The protocol adapters expose interfaces through a specific protocol. A REST adapter provides access through the HTTP protocol, and specific UI adapters can be used to interact with a user using a specific web or desktop framework. In future extensions, adapters can also implement JMX management for example. + +Session management is responsible for managing authentication and authorization across all protocol adapters. This makes sessions a first class citizen in the design and abstracts them away from protocol specific sessions. + +Design +====== + +The design is based on the 4+1 View, which means we look at the logical view, process view, use case design, implementation view and deployment view to arrive at the full design. + +### Logical View + +<object data="remote-interfaces-components.svg" type="image/svg+xml" class="span12" height="344"></object> + +The design is split into three components: + +1. *Protocol Adapters.* For every protocol that we support, we have to create a protocol adapter, whose responsibility it is to expose connector interfaces. Connectors are discovered by leveraging the service registry using the whiteboard pattern. The protocol adapter talks to the session manager for managing sessions and making sure certain permissions have been given to the authenticated user. If the protocol itself also supports the notion of sessions, it's the adapters responsibility to ensure that the life cycle of the protocol specific session is aligned with our own sessions. + +2. *Connectors.* Each connector, which is implemented as a collection of one or more OSGi services, provides a specific API to talk to a system. An example is the client API. Which services make up a connector is specified in the design of a component. + +3. *Session Manager*. Responsible for managing sessions and the authentication and authorization of actors that use them. The session manager will be invoked by protocol adapters. It leverages the OSGi User Admin service to provide users with credentials, roles and permissions. The session manager is allowed to expire sessions if it needs to, so protocol adapters should always check if a session is still open. + +### Process View + +Incoming calls arrive at a protocol implementation. As soon as they reach one of our adapters, the adapter should synchronously talk to the session manager, because it needs to check for the proper authentication and authorization before it can continue. + +A session manager is allowed to spawn a background thread for session expiration purposes, but implementations can also decide to leverage Java features such as "weak" references, or use synchronous expiration as a side effect of incoming calls. + +### Use Case Design + +There are two major use cases that are designed explicitly here: + +1. UC-RI-1 Register Connector Services +2. UC-RI-2 Invoke Operation + +#### UC-RI-1 Register Connector Services + +##### Normal Flow + +1. A component registers a service in the OSGi service registry. +2. The protocol adapter, listening whiteboard style, is notified and validates the service. +3. The protocol adapter hands over the service to a protocol specific factory that creates an adapter. +4. The protocol adapter registers the created adapter with the protocol implementation. + +#### UC-RI-2 Invoke Operation + +##### Normal Flow + +1. An actor invokes an operation on a service. +2. The invocation arrives at the protocol adapter, who looks up or creates the session with the session manager. +3. The protocol adapter checks if the actor is authenticated and authorized to invoke this operation. +4. The protocol adapter invokes the operation on the connector associated with this adapter. + +### Implementation View + +The implementation views for the different connectors and protocol adapters can be found in the designs of these components. + +The implementation view for the session manager consists of: + +* o.a.a.client.repository.SessionFactory - the interface for a factory that can be used to create and destroy sessions; +* o.a.a.client.repository.impl.Activator - the implementation of the session factory. + +Note that this current implementation is specific to the client, because for now that is the only place where we have sessions (the other connectors are essentially stateless). + +### Deployment View + +The connectors and protocol adapters are deployed as different bundles: + +* o.a.a.client.repository.api - contains the API for the client connector and session manager; +* o.a.a.client.repository.impl- contains the implementation for the session manager and client connector +* o.a.a.client.repository.* - various bundles that also contain parts of the implementation of the client connector; +* o.a.a.client.rest - protocol adapter for the REST client API; +* o.a.a.webui.vaadin - protocol adapter for the Vaadin based web UI. Added: ace/site/trunk/content/docs/design/src/remote-interfaces-components.graffle URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/design/src/remote-interfaces-components.graffle?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/design/src/remote-interfaces-components.graffle ------------------------------------------------------------------------------ svn:mime-type = application/xml Added: ace/site/trunk/content/docs/design/src/remoteinterfaces-components.graffle URL: http://svn.apache.org/viewvc/ace/site/trunk/content/docs/design/src/remoteinterfaces-components.graffle?rev=1641501&view=auto ============================================================================== Binary file - no diff available. Propchange: ace/site/trunk/content/docs/design/src/remoteinterfaces-components.graffle ------------------------------------------------------------------------------ svn:mime-type = application/xml
