Added: vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext URL: http://svn.apache.org/viewvc/vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext?rev=1430372&view=auto ============================================================================== --- vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext (added) +++ vcl/site/trunk/content/confluence_export/vmware-configuration.mdtext Tue Jan 8 16:37:53 2013 @@ -0,0 +1,502 @@ +Title: VMware Configuration +<a name="VMwareConfiguration-Terminology"></a> +## Terminology + + +<a name="VMwareConfiguration-VMHost"></a> +#### VM Host + +* A VM host is a physical computer running a VMware hypervisor +* A VCL computer entry must be added for each VM +host ({color:#000000}{_}Manage Computers > Edit Computer +Information)_{color} +* {color:#000000}After the computer has been added to VCL, it is designated +as a VM host by changing the computer state to vmhostinuse{color} +{color:#000000}_(Manage Computers > Computer Utilities)_{color} + +<a name="VMwareConfiguration-VM"></a> +#### VM + +* A VM is a virtual machine managed by VCL +* A computer entry must be added to VCL for each VM +({color:#000000}{_}Manage Computers > Edit Computer Information)_{color} +* {color:#000000}Each VM must be assigned to a VM host{color} +{color:#000000}_(Virtual Hosts > VM Hosts tab > Configure Host)_{color} +* VMs do not need to be created manually in VMware, VCL automatically +creates and deletes VMs + +<a name="VMwareConfiguration-VMHostProfile"></a> +#### VM Host Profile + +* A VM host profile contains several parameters which describe how a +VM host is configured so that VCL knows how to manage it +* Each VM host is assigned a VM host profile +* A VM host profile may be assigned to multiple VM hosts if they are +configured identically +* VM host profiles may be added or modified via _Virtual Hosts > VM Host +Profiles tab_ + +<a name="VMwareConfiguration-VMwareProductsSupported"></a> +## VMware Products Supported + +* VMware Server 2.x +* VMware ESX 3.5 - 4.x +* VMware ESXi 4.x +* VMware ESXi 5.x + +<a name="VMwareConfiguration-VMHost ManagementOptions"></a> +### VM Host Management Options + + The VCL management node must be able to control the VM host and +the VMs running on it. VMware provides several different ways of +doing this. VCL currently supports the following methods for remote +VM host management: +* VMware vSphere SDK +* Use SSH to execute commands directly on the VM (_not officially +supported by VMware_) + +The vSphere SDK can only be used if management is not restricted +due to the VMware license key installed on the host. This mainly +affects hosts running the free version of ESXi. Remote management +using any of the methods supported by VMware is restricted once a +free license key is entered. + +If remote management is restricted, the VM host can be managed if SSH is +enabled on it. VCL will execute vim-cmd and other commands on the VM +host via SSH. + + +<a name="VMwareConfiguration-HowtoenableSSHontheVMhost:"></a> +#### How to enable SSH on the VM host: + +<a name="VMwareConfiguration-VMwareServer2.x"></a> +##### VMware Server 2.x + +Enable the SSH daemon and configure identity key authentication according +to the underlying VM host OS + +<a name="VMwareConfiguration-ESX/ESXi3.5&4.0"></a> +##### ESX/ESXi 3.5 & 4.0 + +* Connect to the console of the ESX/ESXi host +* Press *ALT-F1* \- you should see a black screen with the VMware product +name at the top +* Type the word *unsupported* and press *Enter* (you won't see the letters +appear as you type them) +* You should see a password prompt, type in the *root +password* and press *Enter* +* Edit the file: *vi /etc/inetd.conf* +* Uncomment the first line beginning with _\#ssh_ by deleting the # +character +* Save the file - press *Esc* and then *:wq* +* Kill the inetd process +** Determine the PID of the inetd process: *ps \| grep inetd* +You should see a line that looks like: _5065 5065 busybox inetd_ +** Kill the process (enter the PID from the output of the previous +command): *kill \-HUP 5065* + +<a name="VMwareConfiguration-ESXi4.1"></a> +##### ESXi 4.1 + +Beginning with *ESXi 4.1*, SSH can be enabled using the vSphere +Client: +* Select the ESXi host +* Select the *Configuration* tab +* Select *Security Profile* under Software +* Click *Properties* +* Select *Remote Tech Support (SSH)* +* Click *Options* +* Select *Start automatically* +* Click *Start* +* Click *OK* + +<a name="VMwareConfiguration-ESX5.0"></a> +##### ESX 5.0 + +In the case of *ESX 5.0*: +* Select the ESXi host +* Select the *Configuration* tab +* Select *Security Profile* under Software +* Click *Properties* +* Select *SSH Server* +* Click *Options* +* Confirm that *Start automatically* is selected +* Click *OK* + +<a name="VMwareConfiguration-HowtoconfigureESX/ESXitouseSSHidentitykeyauthentication:"></a> +#### How to configure ESX/ESXi to use SSH identity key authentication: + +SSH identity key authentication must be configured if SSH is used to +manage the VM host. +* Create an SSH key pair on the management node (or use a key you +previously created): +{tip} +ssh-keygen \-t rsa \-f /etc/vcl/vcl.key \-N '' \-b 1024 \-C 'VCL root +account' +{tip} +* Log into the ESX host via SSH (password authentication should work) and +create the directory: +{tip} +ssh <ESXi host> 'mkdir /.ssh' +{tip} +* Copy the public key to the ESXi host: +ESXi 4.x: +{tip} +scp /etc/vcl/vcl.key.pub <ESXi host>:/.ssh/authorized_keys +{tip} +ESXi 5.x: +{tip} +scp /etc/vcl/vcl.key.pub <ESXi host>:/etc/ssh/keys-root/authorized_keys +{tip} +* Test making an SSH connection using the key: +{tip} +ssh \-i /etc/vcl/vcl.key <ESXi host> +{tip} + +{color:#ff0000}{*}IMPORTANT:*{color} Under ESXi 4.x, the authorized_keys +file is erased when the ESXi VM host is rebooted. Complete the +following steps to make the authorized_keys file persistent: + +_Note: VCL will perform these steps automatically when the 1st reservation +assigned to the host is processed._ + + +* Create a compressed tarball file containing the /.ssh directory: +{tip} +tar \-C / \-czf bootbank/vcl.tgz .ssh +{tip} +* Edit the /bootbank/boot.cfg file and append *_' \--\- vcl.tgz'_* to +modules line as shown in the following example: +{panel} +kernel=b.z +kernelopt= +modules=k.z --- s.z --- c.z --- oem.tgz --- license.tgz --- m.z --- +state.tgz --- vcl.tgz +build=4.1.0-260247 +updated=2 +bootstate=0 +{panel} +{tip} +Optionally you can run the following two commands: +tar \-C / \-czf vcl.tgz .ssh +BootModuleConfig.sh \--add=vcl.tgz \--verbose +{tip} + +<a name="VMwareConfiguration-VMHostProfileParameters"></a> +## VM Host Profile Parameters + +<a name="VMwareConfiguration-GeneralParameters"></a> +### General Parameters + + +<a name="VMwareConfiguration-"></a> +#### + +<a name="VMwareConfiguration-*Name*"></a> +##### *Name* + +* Descriptive name of the VM host profile + + +<a name="VMwareConfiguration-*Type*_(deprecated)_"></a> +##### *Type* _(deprecated)_ + +* Removed in VCL 2.3 + +<a name="VMwareConfiguration-*Image*_(optional)_"></a> +##### *Image* _(optional)_ + +* VCL hypervisor image installed on VM host computers using xCAT +{info}{_}xCAT is not required. VM host computers may be installed +manually or by some other means._{info} +* If xCAT is not used, select "No Image" +* VCL has the ability to install a hypervisor image on bare-metal computers +using xCAT. If the image property is configured, the image is +installed when a computer's state is changed to vmhostinuse via _Manage +Computers > Computer Utilities_ + +<a name="VMwareConfiguration-*Username/Password*_(optional)_"></a> +##### *Username/Password* _(optional)_ + +* Name and password of the administrative or root user residing +on the VM host +* This account is used to manage the VM host and VMs assigned to the host +* The username and password are currently only used if the vSphere SDK is +used to manage the VM host and VMs + +<a name="VMwareConfiguration-StorageParameters"></a> +### Storage Parameters + +{anchor:resource} + +<a name="VMwareConfiguration-ResourcePath_(optional)_"></a> +##### Resource Path _(optional)_ + +Resource Path only needs to be configured if VMware vCenter is used. It +defines the location where VMs will be created in the vCenter inventory +tree. The inventory tree contains at least one Datacenter, and may also +contain Folders, Clusters, and Resource Pools. +Example: /DatacenterA/Folder1/Cluster2/ResourcePool3 + +{anchor:repository} + +<a name="VMwareConfiguration-*RepositoryPath*_(optional)_"></a> +##### *Repository Path* _(optional)_ + +* Path where master copies of images are stored which are used to +transfer images to VM host datastores or to other repositories: +** If a reservation is assigned to a host but the image does not exist in +that host's datastore, it is copied from the repository to the virtual disk +path when the VM is loaded +** If the VCL environment contains multiple management nodes and the image +does not exist in the repository or the host's datastore, the image will +be retrieved from another management node's repository by copying it via +SCP +* The Repository Path parameter does not need to be configured if the VCL +environment contains a single management node and all VM hosts share the +same Virtual Disk Path +* Example: _/vmfs/volumes/nfs-repository1_ +* VMs do not run directly off of the images stored in the repository +* Setting the Repository Path parameter determines whether or not an +additional copy of an image is created when an image is captured +** If repository path is not configured then only a single copy of the +image will exist in the virtual disk path after an image is captured +** If repository path is configured then two copies of the image will +exist after an image is captured - one in the virtual disk path and one in +the repository +* Repository Path location can refer to and be mounted on either the +management node or VM host +** It is highly recommended that the repository be mounted on the VM host +*** When mounted on the VM host, vmdk operations can be done directly on +the VM host in a single step +* Images in the repository are stored in the 2 GB sparse vmdk format +** The size of the vmdk files will approximately be equal to the amount of +actual data saved in the image regardless of the size of the VM's hard +drive +** Storing images in the 2 GB sparse format is necessary to allow images +to be transferred via SCP without having to transfer data equal to the +entire size of the VM's hard drive + +<a name="VMwareConfiguration-RepositoryImageType"></a> +##### Repository Image Type + +Virtual disk file format for images stored in the repository. +{anchor:datastore} + +<a name="VMwareConfiguration-*VirtualDiskPath(previouslyDatastorePath)*"></a> +##### *Virtual Disk Path (previously Datastore Path)* + +* Location where master copies of images are stored which are used by +running VMs +* Example: _/vmfs/volumes/nfs-datastore1_ +{info}For ESXi, the path configured in the profile may simply be the short +datastore name as it appears in the vSphere Client: nfs-datastore1{info} +* Storage location should be large enough to store all of the images which +may be loaded on the VM host _(from 100's of GB to several TB)_ +* VCL creates a directory for each image in the Virtual Disk Path +* Images are stored in the [vmfs thin vmdk format](http://www.sanbarrow.com/vmdk-basics.html) +* Virtual Disk Path may either reside on local or network storage +* Multiple VM hosts can share the same datastore if network storage is used +** A single datastore may be used by all VM hosts if performance is +adequate +** Multiple VMs on different hosts may access the same Virtual Disk Path +image at the same time +** It is recommended that datastores are shared among hosts so that fewer +copies of each image have to be stored +** The underlying storage hardware and network connectivity from the hosts +to the storage must be adequate +** Storage where the datastore is located should be optimized for read +performance +* VCL configures VMs to access images stored in the Virtual Disk Path in +read-only mode +** Changes made to the VM's hard drive are written to delta files located +in the VM Working Directory Path dedicated for the VM + +<a name="VMwareConfiguration-VirtualDiskImageType"></a> +##### Virtual Disk Image Type + +Virtual disk file format for images stored in the virtual disk path. + +<a name="VMwareConfiguration-VirtualDiskMode(previously*VMDisk)*"></a> +##### Virtual Disk Mode (previously *VM Disk)* + +* Defines whether the storage where the VM host's Virtual Disk Path +resides is dedicated to a single host or shared among multiple hosts: +** dedicated (previously localdisk) +*** The VM host's Virtual Disk Path is located on local disks or +dedicated network storage +*** The VM host is the only host which accesses the Virtual Disk Path +*** Repository Path must be configured +** shared (previously networkdisk) +*** The VM host's Virtual Disk Path is located on network storage +which is shared by other VM hosts +*** Repository Path is optional + +{note} +The Virtual Disk Mode (VM Disk) parameter does *not* determine whether or +not: +...images are copied from the datastore to the repository during image +capture +...images are copied from the repository to the datastore during image load +These are determined by whether or not Repository Path is configured in the +profile +{note} + + +{anchor:vmpath} + +<a name="VMwareConfiguration-VMWorkingDirectoryPath_(optional)_(previously*VMPath*)"></a> +##### VM Working Directory Path _(optional)_ (previously *VM Path*) + +* Defines path on VM host where VM working directories will reside +(contains .vmx, delta, .vswp, nvram files) +* If not configured, the Virtual Disk Path location will be used +* VCL creates a directory under the VM Working Directory Path for each VM +it creates +** Contains the .vmx file which defines the VM +** Contains delta vmdk files which are written to as changes are made to +the VM's hard drive +* VM Working Directory Path may either reside on local or network storage +* Location should be dedicated for each VM host +** Multiple VM hosts should not share the same VM Working Directory Path +location for performance and image safety reasons +** VM Working Directory Paths of multiple hosts may reside on the same +volume but a subdirectory should be created for each host +* Storage where the VM Working Directory Path is located should be +optimized for read-write performance + + + + +<a name="VMwareConfiguration-"></a> +##### + +<a name="VMwareConfiguration-NetworkingParameters"></a> +#### Networking Parameters + +<a name="VMwareConfiguration-*VMNetwork(previouslyVirtualSwitch)*"></a> +##### *VM Network (previously Virtual Switch)* + +* VM Network 0 (previously Virtual Switch 0) - private VCL +management network +* VM Network 1 (previously Virtual Switch 1) - public network used by user +making reservation to access the VMs + +* The VM Network parameters should match the network names configured on +the VM host +** For ESXi, the _VM Network_ parameters must match the _Virtual Machine +Port Group Network Labels_ configured in the vSphere Client, example: +*** VM Network 0: Public +*** VM Network 1: Private + !vmware-network-labels.gif|border=1! + +* For VMware Server 2.x, the _VM Network_ parameters must match the +_Network Names_ configured by running vmware-config.pl + +<a name="VMwareConfiguration-*Generateeth0/eth1MAC*"></a> +##### *Generate eth0/eth1 MAC* + +* New in VCL 2.3 +* Determines whether VMs are assigned MAC addresses defined in the VCL +database or if random MAC addresses should be assigned + +<a name="VMwareConfiguration-ConfigurationExamples"></a> +## Configuration Examples + + +<a name="VMwareConfiguration-LocalDiskOnly-RepositoryMountedviaNFS"></a> +### Local Disk Only - Repository Mounted via NFS + + +<a name="VMwareConfiguration-!local-only-nfs.gif!"></a> +### !local-only-nfs.gif! + + +The diagram above shows a simple VCL configuration with 1 management node +and 2 VMware ESXi hosts. Network storage is not used. +The local disks on the VM hosts are used to store all of the files used by +running VMs including the VM's working directory and the master vmdk image. + +A directory on the local disk on the management node is used to as the +image repository. This directory is exported via NFS. VM hosts +mount this directory as a datastore named "repository". Mounting the +repository directly on the VM hosts allows the vmkfstools utility to be +used on the VM hosts to copy and convert images directly from the +repository to the local datastore in a single step. + +If an image is to be loaded on a VM host and that image does not already +exist in the VM host's local datastore (Virtual Disk Path), it is +automatically copied from the repository to the VM host's local datastore +(Virtual Disk Path) at the beginning of the load process. + +During image capture, images are automatically copied to from the VM host's +local datastore (Virtual Disk Path) to the repository. This allows +images captured on a VM host to be loaded on any other VM host. + +The VM host profile Virtual Disk Mode parameter is set to dedicated. +This indicates to the load process that the VM host's Virtual Disk Path is +dedicated to the VM host and not shared by other VM hosts. This +allows images to be deleted from the VM host's local datastore (Virtual +Disk Path) if another image must be copied from the repository and not +enough space is available. + + +<a name="VMwareConfiguration-LocalDiskOnly-RepositoryNotAvailableviaNFS"></a> +### Local Disk Only - Repository Not Available via NFS + + +!local-only-scp.gif! + +This example is identical to the one above except that the repository +located on the management node's local disk is not exported via NFS. +Because of this, images must be transferred using SCP instead of +vmkfstools. This is less desirable than mounting the repository +directly on the VM hosts because images cannot be copied and converted in a +single step. Images are stored in the repository in the 2GB sparse +format. This allows the images to be copied via SCP while only +transferring the data stored in the image, not the entire size of the hard +drive stored in the image. VMware ESXi cannot run VMs using vmdk +images stored in the 2GB sparse format. Images are converted to the +vmfs thin format so that they can be loaded on VMware ESXi. This adds +extra time to the load process if an image does not exist in the VM's local +datastore (Virtual Disk Path) and must be copied from the repository. +It also requires additional space in the VM host's local datastore (Virtual +Disk Path) becuase 2 copies of the image exist while it is being converted. +Note that the VM host profile Repository Path parameter is set to the path +on the management node's hard drive. The code first checks if the +path exists on the VM host. If not, it assumes the repository is not +mounted directly on the VM host and the Repository Path value refers to a +location on the management node. + +<a name="VMwareConfiguration-NetworkStorageOnly-NoRepository"></a> +### Network Storage Only - No Repository + +!network-only-no-repo.gif! + +This is an example of a simple configuration where the network storage is +used. + +A repository is not used in this configuration. This implies that all +VM hosts which will ever be added to this VCL environment will need to be +able to connect to the network storage. + +A datastore to be used as the Virtual Disk Path named "datastore" is +mounted on every VM host. Each of these mounts points to the same +location on the network storage. The datastore will contain the +master vmdk images. VMs loaded on the VM hosts will read from these +master vmdk images. + +A datastore to be used as the VM Working Directory Path named "vmpath" is +also mounted on each VM host. However, the location to which each VM +host points should be different. In the example above, vmhost-a-01 +points to th the /vmpath01 directory on the network storage and vmhost-a-02 +points to the /vmpath02 directory. These locations may be different +network storage filesystems or may be different directories on the same +network filesystem. Even though the mounts on the VM hosts point to +different locations, the datastore names configured under ESXi are +identical. This allows you to use the same VCL VM host profile for +all of the VM hosts. + +The VM host profile Virtual Disk Mode parameter is set to shared. +This indicates to the load process that the VM host's Virtual Disk Path is +shared by other VM hosts.
Added: vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext URL: http://svn.apache.org/viewvc/vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext?rev=1430372&view=auto ============================================================================== --- vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext (added) +++ vcl/site/trunk/content/confluence_export/vmware-provisioning-module-information.mdtext Tue Jan 8 16:37:53 2013 @@ -0,0 +1,78 @@ +Title: VMware Provisioning Module Information +<a name="VMwareProvisioningModuleInformation-VMwareProvisioningModule"></a> +## VMware Provisioning Module + +* Source file: managementnode/lib/VCL/Module/Provisioning/VMware/VMware.pm + +<a name="VMwareProvisioningModuleInformation-SupportedVMwareProducts"></a> +#### Supported VMware Products + +* VMware Server 2.0 +* VMware ESX/ESXi 3.5 +* VMware ESX/ESXi 4.0 + +<a name="VMwareProvisioningModuleInformation-SupportedVMware APIs,SDKs,&CLIs"></a> +#### Supported VMware APIs, SDKs, & CLIs + +There are several different methods to control VMware hosts and virtual +machines. The main VMware.pm module should remain as +SDK/API/CLI-agnostic as possible. All code which interacts with a +VMware remote control method should reside in utility modules located in +the lib/VCL/Module/Provisioning/VMware directory. +* vSphere SDK - lib/VCL/Module/Provisioning/VMware/vSphere_SDK.pm +** Provides full VM and VM host remote access functionality +** Provides VM host datastore file access +** Can only be used if the VMware host is not running with a +restricted license +** SSH access is not required to the VM host +** Installed by default on ESX/ESXi hosts +** Can be installed on Server 2.0 hosts +* -VIX API - lib/VCL/Module/Provisioning/VMware/VIX_API.pm- +_Although a utility module has been written to use the VIX API, it is not +supported and will not be maintained. Everything that the VIX API can +do can be done by the vim-cmd CLI and SSH access to the VM host is required +for both methods. Therefore, there is no benefit to maintain the +VIX API module._ +** -Provides basic VM control functions- +** -Does not provide any file access- +** -SSH access is required to the VM host in order to manipulate the file +system- +** -Can only be used if the VMware host is not running with a +restricted license- +** -Installed by default on Server 2.0 hosts- +* vim-cmd CLI via SSH - lib/VCL/Module/Provisioning/VMware/VIM_SSH.pm +** Provides full VM host remote access functionality +** Does not provide file access +** SSH access is required to the VM host in order to manipulate the file +system +** Can be used if the the VMware host is running with a restricted license + +<a name="VMwareProvisioningModuleInformation-RequiredUtilityModuleSubroutines"></a> +### Required Utility Module Subroutines + +* get_registered_vms +* vm_register +* vm_unregister +* get_vm_power_state +* vm_power_on +* vm_power_off +* copy_virtual_disk +* move_virtual_disk +* get_virtual_disk_controller_type +* get_virtual_disk_hardware_version +* get_virtual_disk_type + +<a name="VMwareProvisioningModuleInformation-RequiredOSModuleSubroutines"></a> +### Required OS Module Subroutines + +* find_files +* copy_file +* copy_file_to +* copy_file_from +* delete_file +* move_file +* file_exists +* get_available_space +* create_directory +* get_file_contents +* get_file_size Added: vcl/site/trunk/content/confluence_export/web-code-installation.mdtext URL: http://svn.apache.org/viewvc/vcl/site/trunk/content/confluence_export/web-code-installation.mdtext?rev=1430372&view=auto ============================================================================== --- vcl/site/trunk/content/confluence_export/web-code-installation.mdtext (added) +++ vcl/site/trunk/content/confluence_export/web-code-installation.mdtext Tue Jan 8 16:37:53 2013 @@ -0,0 +1,159 @@ +Title: Web Code Installation +{excerpt}This page describes how to install and configure the frontend VCL +web code components including the web server prerequisites and frontend VCL +web code. It also describes how to add local web accounts, configure LDAP +authentication, and set the timezone correctly.{excerpt} + + +<a name="WebCodeInstallation-*Prerequisites*"></a> +## *Prerequisites* + +Your web server should meet the following criteria before installing the +frontend VCL code: +* Apache HTTP Server v1.3 or v2.x with SSL enabled - while VCL may run +under another webserver capable of running PHP code, it has only been +tested to work with Apache HTTP Server +* PHP 5 +* php modules that should be installed (depending on your Linux distro, +some of these may be compiled in to php instead of being a separate +module): +** php-gd +** php-json (if your PHP version is < 5.2, this is not required) +** php-mcrypt +** php-mysql +** php-openssl +** php-sysvsem +** php-xml +** php-xmlrpc +* useful to have the server set up to be able to send debugging emails +* php-mcrypt requires libmcrypt and mcrypt libraries as dependencies. +These may need to be installed first. + +<a name="WebCodeInstallation-*InstallingVCLFrontendWebCode*"></a> +## *Installing VCL Frontend Web Code* + +1. If you haven't already done so, download and extract a copy of the latest +release. There is a link to it under the Project Resources section on our +wiki [home page](apache-vcl.html) +. Look for "Current version". +1. copy the "web" directory to a location somewhere under the web root of +your web server: + + cp -r web/ /var/www/html/vcl + +1. modify vcl/.ht-inc/secrets.php +1. * set $vclhost, $vcldb, $vclusername, and $vclpassword to match your +database setup +1. * create random passwords for $mcryptkey, $mcryptiv, and $pemkey - +$mcryptiv must be 8 hex characters +1. run the genkeys.sh script from within vcl/.ht-inc and give it $pemkey +from secrets.php as the passphrase (3 times, copy/paste is a good idea +here) +1. modify vcl/.ht-inc/conf.php to match your site - COOKIEDOMAIN needs to be +the domain name your web server is using, or left blank if you are +accessing it by IP only. +*NOTE:* There is a misconfiguration in conf.php in VCL 2.1. To correct it, +change affiliationid for "Local Account" in the $authMechs array from 4 to +1. +1. \**NOTICE*\* JpGraph 2.x is no longer available. JpGraph 3.x is released +under a dual license. QPL 1.0 (Qt Free Licensee). Free for non-commercial, +open-source or educational use (JpGraph Professional License for commercial +use). If you are planning to use this for commercial use and don't want to +pay for JpGraph, you can safely skip this step with the only side effect of +not being able to display a few graphs on the statistics page. +Download JpGraph from [http://www.aditus.nu/jpgraph/jpdownload.php](http://www.aditus.nu/jpgraph/jpdownload.php) +1. * For PHP5, download the 3.x series, extract it, and copy the src +directory from it to vcl/.ht-inc/jpgraph +1. download version 0.4.0 of Dojo Toolkit: [http://download.dojotoolkit.org/release-0.4.0/dojo-0.4.0-ajax.tar.gz](http://download.dojotoolkit.org/release-0.4.0/dojo-0.4.0-ajax.tar.gz) +1. * extract it under the vcl directory and rename "dojo-0.4.0-ajax" to +"dojoAjax" +1. download version 1.1.0 of Dojo Toolkit: [http://download.dojotoolkit.org/release-1.1.0/dojo-release-1.1.0.tar.gz](http://download.dojotoolkit.org/release-1.1.0/dojo-release-1.1.0.tar.gz) +1. * extract it under the vcl directory and rename "dojo-release-1.1.0" to +"dojo" +1. go into the themes directory (vcl/themes) and run "./copydojocss.sh +default" to copy parts of dojo's css into the "default" theme +1. if you want to be able to edit any of the documentation that comes +bundled with the vcl web code, download fckeditor from [http://www.fckeditor.net/download](http://www.fckeditor.net/download) + (most people can skip this step) +1. * extract it under the vcl directory +1. open a browser and open the testsetup.php page +1. * i.e. if you set up your site to be [https://my.server.org/vcl/](https://my.server.org/vcl/) + open [https://my.server.org/vcl/testsetup.php] +1. debug any issues reported by testsetup.php +1. now, open the index.php page in your browser +1. select Local Account and use 'admin' as the user and 'adminVc1passw0rd' +as the password +1. click the "Management Nodes" link +1. enter the hostname and IP of your management node +1. click Add +1. fill in "Install Path" - this is parent directory under which image files +will be stored +1. enter "/etc/vcl/vcl.key" for "End Node SSH Identity Key Files" +1. click "Confirm Management Node" +1. click Submit +1. click the "Management Nodes" link +1. select "Edit Management Node Grouping" +1. click Submit +1. select the checkbox for your management node +1. click Submit +1. click "Manage Computers" +1. select the "Add Single Computer" radio button +1. click the Submit +1. fill in Hostname, IP Address, owner (admin@Local), RAM, Proc Speed, +Network Speed, select "blade" for Type, select "xCAT 1.x Provisioning" for +"Provisioning Engine", and click the checkbox under "allcomputers", and +"newimages" + Note: if using using vmware, select "virtualmachine" for +Type and "VMWare Server Provisioning" for "Provisioning Engine" +1. click Confirm Computer +1. click Submit (don't worry about the fact that the computer you just added +isn't listed after clicking Submit) +1. after you've configured your image library and your management node has +started checking in, you should be able to make a reservation + +<a name="WebCodeInstallation-*Addingextralocalaccounts*"></a> +## *Adding extra local accounts* + +There's not currently a tool for this. You will need to add entries +directly to the database. +1. add entry to user table + + INSERT INTO user (unityid, firstname, lastname, email, lastupdated) VALUES +('myusername', 'myfirstname', 'mylastname', 'myemailaddr', NOW()); + +1. find out the id generated for that user + + SELECT id, unityid FROM user WHERE unityid = 'myusername'; + +1. add entry to the localauth table + + INSERT INTO localauth (userid, salt, passhash, lastupdated) VALUES +('place1', 'place2', 'place3', NOW()) + +with place1 = id from step 2 +place2 = an 8 char random string +place3 = sha1sum( desired password with place2 stuck on the end ) +this can be generated under linux like this (using 'thedog' as the password +and 11111111 as place2): +echo \-n 'thedog11111111' \| sha1sum +Once a user has been added, the user can go to User Preferences to change +his/her password + +<a name="WebCodeInstallation-*AddingLDAPauthentication*"></a> +## *Adding LDAP authentication* + +1. fill in the necessary information in vcl/.ht-inc/conf.php +1. add an entry to the affiliation table and use the id for that entry as +'affiliationid' for your new entry in vcl/.ht-inc/conf.php +1. uncomment the 'require_once(".ht-inc/authmethods/ldapauth.php");' line in +in vcl/.ht-inc/conf.php + +<a name="WebCodeInstallation-*SettingTimeZone*"></a> +## *Setting Time Zone* + +1. Edit /var/www/html/vcl/.ht-inc/php5extras.php to indicate the correct +time zone: +date_default_timezone_set('America/Los_Angeles'); +1. Edit /var/www/html/vcl/.ht-inc/requests.php (line 3301 currently) to +indicate the correct time zone (purely cosmetic): +print "<small>(Pacific Time Zone)</small>"; Added: vcl/site/trunk/content/confluence_export/web-code-overview.mdtext URL: http://svn.apache.org/viewvc/vcl/site/trunk/content/confluence_export/web-code-overview.mdtext?rev=1430372&view=auto ============================================================================== --- vcl/site/trunk/content/confluence_export/web-code-overview.mdtext (added) +++ vcl/site/trunk/content/confluence_export/web-code-overview.mdtext Tue Jan 8 16:37:53 2013 @@ -0,0 +1,454 @@ +Title: Web Code Overview +<a name="WebCodeOverview-*GeneralOverview*"></a> +## *General Overview* + +The code is broken up into multiple files, based on sections of the site +(as divided up by the navigation area). There are also several general +purpose files: +* index.php - This is the only file used in any URLs. It includes other +files based on the passed in "mode". +* states.php - builds an array of modes, which function should be called +for each mode, and the part of the site each mode belongs to +* utils.php - contains many functions that are common to several sections +* errors.php - contains an array of all errors that can be reported by the +site. I'm not sure how useful it was do set it up this way, but it's where +things currently are. errors.php also contains the debug function that is +used if the current user has an adminlevel of ADMIN_DEVELOPER in the user +table. +* secrets.php - contains all passwords/passphrases and secret and shared +keys +* conf.php - contains all configuration variables that might need to be +modified for each install base +* php5extras.php - anything that needs to be done differently when using +php5 vs php4 + +<a name="WebCodeOverview-*SiteSecurity*"></a> +## *Site Security* + +The initial handling of form data within the code was quite insecure, and +several areas of the site are still this way. After learning more about web +security, I developed a security model based on "continuations". All of the +pages have been converted to using continuations. + +Deep linking into the site is only allowed for modes in the $actions\['entry'\]('entry'\.html) + array in states.php. Anything else requires the submission of a +continuation. For the most part, access to different parts of the site is +controlled by what privileges you have in the Privileges section of the +site. However, there are a few things controlled by a user's adminlevel +field in the user table. The very earliest form of authorization was +handled by the user's adminlevel field, and it has continued to be useful +in a few situations. + +All form data passed in to the site should be verified very strictly. +Unfortunately, that is not currently the case. All of the main pages +available to the average user should have been updated to have strict +validation, though other parts of the site have not made it yet. Most +sections of the site have a single function (or a very small number of +similar functions) that handle the processing of form data. This will make +it easier to add better checks throughout the site as the number of +locations needing to be modified is fairly small. + +<a name="WebCodeOverview-*GeneralProcessingFlow*"></a> +## *General Processing Flow* + +Every time someone views the site, it is through index.php. This file +defines several global variables and includes conf.php, states.php, +errors.php, and utils.php. It then creates a database connection and calls +initGlobals(), which among other things, determines $mode. Based on $mode, +index.php determines which function needs to be called and assigns it to +$actionFunction. Next, checkAccess() is called, followed by sendHeaders() +and printHTMLHeader() (which doesn't actually print the header if $mode is +in $noHTMLwrappers). set_error_handler is called if the current user has an +adminlevel of ADMIN_DEVELOPER. Next, $actionFunction is finally called, +followed by disconnecting from the database, a call to printHTMLFooter(), +and a call to semUnlock() to make sure the semaphore was unlocked if it was +locked somewhere in the process. + +One thing worth noting that initGlobals() does is to include other files +based on which mode is being processed. This helps to prevent the php +script engine from having to process unnecessary files. It also adds a +small layer of security because a section of code cannot be attacked if it +has not been included. + +<a name="WebCodeOverview-*Continuations*"></a> +## *Continuations* + +Continuations are how the site handles sequences of pages. It also helps +keep people from getting to parts of the site they aren't allowed to access +or shouldn't jump in to the middle of (i.e. by using the browser's back +button). Continuations are created by calling addContinuationsEntry, which +accepts up to 6 arguments: +* $nextmode - the mode that should be used when the continuation is +submitted +* $data - an array of any data that you would like to have available when +the continuation is submitted +* $duration - how long from "now" that the continuation should be valid (in +seconds) +* $deleteFromSelf - boolean - 1 if this is the start of a new sequence of +pages, 0 if it will be a continuation of a sequence. If it is set to 0, +then the "parent" continuation for this one is the continuation that the +site is currently processing. +* $multicall - boolean - whether or not the continuation may be submitted +more than once +* $repeatProtect - boolean - used in cases where a "sequence loop" can +occur + +addContinuationsEntry returns a long encrypted string to be used as an +identifier to be submitted (either as a hidden form field or in the URL for +a GET link). What gets encrypted is a salt value, the id of the +continuation that was created (or updated), the user's numeric id, and a +timestamp. If the string gets tampered with, it will not decrypt properly. +If someone tries to submit a continuation given to another user, their user +ids won't match; so, it will be considered invalid. + +When a continuation is submitted, some checks are run and, if everything +passes, whatever was submitted as $nextmode is the mode for which the site +functions. One of those checks is that $duration has not expired; if it +has, the user is given a notice that he has submitted expired data. Any +data submitted as $data can be accessed by calling getContinuationsData() +with a single argument being the index of the array that was passed to +addContinuationsEntry. Additionally, getContinuationsData can be called +with no arguments to get all of $data as a single array. If $multicall was +set to 0, then the continuation is deleted. If $deleteFromSelf was also set +to 0, then this continuation's parent will also be deleted. If +$deleteFromSelf was set to 0 for the parent, it's parent will be deleted, +and so on until a continuation is reached that had $deleteFromSelf set to +1. + +<a name="WebCodeOverview-*JavascriptandAJAX*"></a> +## *Javascript and AJAX* + +Efforts have been made to ensure the pages required for making and +connecting to a reservation work without requiring any javascript. However, +enhancements have been made to enrich those parts of the site if javascript +is enabled. For some of the administrative parts of the site, javascript +and AJAX have been used heavily, particularly the Privileges page. The [Dojo Toolkit](http://dojotoolkit.org) + is what is being used for javascript widgets and to handle some of the +AJAX. + +<a name="WebCodeOverview-*AFewExamples*"></a> +## *A Few Examples* + + +<a name="WebCodeOverview-*Addingalinktothenavigationarea*"></a> +### *Adding a link to the navigation area* + +Here are the steps that would need to be done to add a new section of the +site. + +First, modify states.php to add a new mode. +1. create a new $actions\['mode'\]('mode'\.html) + with the name of your mode set to the name of the function that should be +called +1. create a new $actions\['pages'\]('pages'\.html) + with the name of your mode set to the name of the section this mode +belongs to. This is only an internal identifier used to associate modes +together. + +So, if your mode is named "examplemode", you could end up with these two +lines being added: + + $actions['mode'] +['examplemode'] = "exampleFunc1"; + $actions['pages'] +['examplemode'] = "exampleSection"; + +While we're editing states.php, lets jump to the top and add our new mode to $actions\['entry'\]('entry'\.html) + so that it can be called directly without having to already be on the +site. Just add 'examplemode' as a new item at the end of the array. + +The next thing to do is to actually add the functions. Lets place them in a +new file called 'examples.php' in the .ht-inc directory. Our first function +can be really simple and just print out some text. So, create examples.php +with this in it: + + + <?php + function exampleFunc1() { + print "exampleFunc1 successfully called.<br>\n"; + } + ?> + + +There's one last thing we need to do before linking this in on the site. As +described in the "General Processing Flow" section, initGlobals includes +the required files based on the current mode's section. So, edit utils.php +and scroll toward the end of it where files are included (using +require_once) within a switch statement. In the switch statement, before +the "default" case, add + + + case 'exampleSection': + require_once('.ht-inc/examples.php'); + break; + + +Now, we're ready to actually add a link for this example function to the +navigation area (of course, not all modes are linked to from the navigation +area, but it is an easy example). To do this, edit utils.php and find the +getNavMenu function close to the bottom of the file. We'll add our new mode +to the end; so, find the "logout" part which should look something like +this: + + + if($inclogout) { + $rt .= menulistLI('authentication'); + $rt .= "<a href=\"" . BASEURL . SCRIPT . "?mode=logout\">"; + $rt .= "Logout</a></li>\n"; + } + + +We'll basically duplicate that (without the if conditional), changing a few +things so that we add this right below it: + + + $rt .= menulistLI('exampleSection'); + $rt .= "<a href=\"" . BASEURL . SCRIPT . "?mode=examplemode\">"; + $rt .= "Example Section</a></li>\n"; + + +Viewing the site should now show "Example Section" right under "Logout" in +the navigation area. Clicking "Example Section" should cause "exampleFunc1 +successfully called." to be displayed in the main content area of the site. + +<a name="WebCodeOverview-*Usingcontinuationswhensubmittingformdata*"></a> +### *Using continuations when submitting form data* + +Let's modify examplefunc1 so that it prints some form data that gets +submitted with a continuation. + +So, change the contents of examplefunc1 to be: + + + $options = array(0 => "option1", + 1 => "option2"); + print "<FORM action=\"" . BASEURL . SCRIPT . "\" method=post>\n"; + print "Select one of these options:"; + printSelectInput("theoption", $options); + $cont = addContinuationsEntry("submitFunc1Form", $options); + print "<INPUT type=hidden name=continuation value=\"$cont\">\n"; + print "<INPUT type=submit value=Submit>\n"; + print "</FORM>\n"; + + +Now, we have a form that gets displayed when "Example Section" is clicked. +Now, we need to add a function to process that form. Add this function to +examples.php: + + + function submitFunc1Form() { + $data = getContinuationVar(); + $theoption = processInputVar("theoption", ARG_NUMERIC); + if(! array_key_exists($theoption, $data)) { + print "invalid option submitted\n"; + return; + } + print "The option you selected was: "; + print "{$data\[$theoption\] +}<br>\n"; + } + + +Next, we add this function to states.php: + + + $actions['mode'] +['submitFunc1Form'] = "submitFunc1Form"; + $actions['pages'] +['submitFunc1Form'] = "exampleSection"; + + +Now, click the "Example Section" link, select one of the options, and click +Submit to see if it works. + +<a name="WebCodeOverview-*UsingAJAXtodynamicallyupdateapage*"></a> +### *Using AJAX to dynamically update a page* + +AJAX is very useful for dynamically updating page content. Let's add +something to examplefunc1 that can be updated with an AJAX call. + +Add this to the end of examplefunc1: + + + print "<br><br>\n"; + print "<div id=examplediv>\n"; + print "This will get dynamically changed.<br>\n"; + print "</div>\n"; + $cont = addContinuationsEntry("AJexample"); + print "<a onclick=\"JS_AJexample('$cont');\">Click to "; + print "update</a><br>\n"; + + +Next, we need to add the javascript function we just referenced to code.js +(in .ht-inc's parent directory) as well as a callback function that will be +called when the results of the AJAX call are returned: + + + function JS_AJexample(contid) { + dojo.xhrPost({ + url:'index.php', + load: JS_AJexampleCB, + error: errorHandler, + content: {continuation: contid}, + timeout: 15000 + }); + } + + function JS_AJexampleCB(data, ioArgs) { + eval(data); + } + + +Then, we need to add a few things to states.php: + +to the $actions array: + + + $actions['mode'] +['AJexample'] = "exampleFunc2"; + $actions['pages'] +['AJexample'] = "exampleSection"; + + +to the $noHTMLwrappers array: + + + 'AJexample' + + +Now, we need to create exampleFunc2 (in examples.php): + + + function exampleFunc2() { + print "document.getElementById('examplediv').innerHTML = 'Dynamic +update';"; + } + + +Then, we do something we haven't seen before. getDojoHTML (in utils.php) +must be modified so that the correct dojo requires get set when we are in +mode examplefunc1. For a simple AJAX update, we only need the dojo.parser +module to be loaded. Since this is already loaded for some of the Group +modes, just add another case statement under submitDeleteGroup so we have: + + + case 'viewGroups': + case 'submitEditGroup': + case 'submitAddGroup': + case 'submitDeleteGroup': + case 'examplemode': + $dojoRequires = array('dojo.parser'); + break; + + +We also have to add a case statement a little further down where the HTML +is actually generated. Find "case 'submitDeleteGroup':" in the switch +statement following the one we just modified, and add another case +statement for examplemode so we have: + + + case 'viewGroups': + case 'submitEditGroup': + case 'submitAddGroup': + case 'submitDeleteGroup': + case 'examplemode': + $rt .= "<style type=\"text/css\">\n"; + + +Since we modified code.js, to test this, you will need to hold Shift and +click the reload button in your browser to force your browser to reload +code.js. "Click to update" will not show up as a normal link, but it +should still be clickable. You can use CSS to make it look like a normal +link or give it an href="#" if you want. + +<a name="WebCodeOverview-*Awordaboutusingtabsforcodeindentation*"></a> +## *A word about using tabs for code indentation* + +All of the code has been indented using tabs (rather than spaces) except +where code wraps to more than one line, in which case a combination of tabs +and spaces are used. This allows someone editing the code to set his tab +width to whatever he prefers, but still allows code that wraps to multiple +lines to line up correctly. To do this, when you want to wrap code to +another line, put a newline at the end of the one you are on just as you +normally would. Next, tab over from the beginning of the line until you +are even with where the initial line of code started, then use spaces to +line up this line with something in the line above it that makes sense. +I'll give an example. Almost all of the queries are written on multiple +lines like this: + + + =====------------------------------ + $query = "SELECT id, " + . "name, " + . "prettyname " + . "FROM image " + . "WHERE id < 400 AND " + . "OSid = 3"; + =====------------------------------ + + +In this example, whitespace indicated by the = marks should be made of +tabs, whitespace indicated by the - marks should be made of spaces. + +<a name="WebCodeOverview-*InlineCodeDocumentation*"></a> +## *Inline Code Documentation* + +Online documentation of the code is generated using Doxygen. Each file +that should be parsed to generate docs needs to have the following toward +the top of it: + + + /** + * \file + */ + + +All functions should have a header above them that documents what it does. +The header should include the name of the function, a description of any +parameters passed in, a description of anything returned, and a brief +description of what the function does. The format of the header is: + + + ///////////////////////////////////////////////////////////////////// + /// + /// \fn nameOfFunction($param1, $param2) + /// + /// \param $param1 - description of param1 + /// \param $param2 - description of param2 + /// + /// \return description of datastructure returned + /// + /// \brief short description of what the function does + /// + ///////////////////////////////////////////////////////////////////// + + +If the function doesn't receive any parameters or doesn't return anything, +those sections can be omitted. + +<a name="WebCodeOverview-*Authentication*"></a> +## *Authentication* + +Authentication has been somewhat modularized. When a user loads the site, +the initGlobals function checks to see if the user is logged in. If not, +the mode in which the site should function gets set to "selectauth". Here, +the user is prompted to select an authentication method to use to log in. +Authentication methods are configured in $authMethods which is in conf.php. + There are currently three authentication types that can be handled: +redirect, ldap, and local. Redirect types send the user to another site to +handle their authentication, with the expectation that a cookie will be set +and that knowledge of how to interpret that cookie is handled in +initGlobals. If an ldap or local method is chosen, the user is directed to +a page displayed by printLoginPageWithSkin() (in authentication.php as are +most of the following functions). This function sets up the skin for the +page to match the affiliation defined in $authMethods, and then calls +printLoginPage(). printLoginPage() displays a form for the user to enter a +userid and password. The form is submitted and then processed by +submitLogin(). If the authentication method is ldap based, ldapLogin() is +called; if it is the local system, localLogin() is called. Each of these +functions tries to validate the user. If it succeeds, a cookie named +VCLAUTH is set, and the user is redirected to the main page of the site. +If it fails, the user is redirected back to the login page. _If you enter +valid credentials, but still get redirected back to the login page, the +first thing to check is whether or not the setcookie() call was reached, +and if so, was the cookie actually set in your browser._
