Author: pierre Date: Mon Mar 27 12:49:04 2017 New Revision: 3945 Log: Refactor two README files
Modified: jhalfs/trunk/README jhalfs/trunk/README.BLFS Modified: jhalfs/trunk/README ============================================================================== --- jhalfs/trunk/README Mon Mar 27 12:12:06 2017 (r3944) +++ jhalfs/trunk/README Mon Mar 27 12:49:04 2017 (r3945) @@ -2,146 +2,175 @@ 1. INTRODUCTION:: - This collection of scripts, known as jhalfs, strives to create - accurate makefiles from the Linux From Scratch book series XML files. - This software is an evolution of the original "jhalfs-0.2" code developed - by Jeremy Huntwork. - - The usage of this script assumes you have read and are familiar with - the book(s) and, therefore, the configuration variables found in menuconfig - interface will have meaning to you. - - The list of supported books can be found at - http://wiki.linuxfromscratch.org/alfs/wiki/SupportedBooks - - NOTES:: - *. The resulting Makefile takes considerable time to run to completion. - Lay in a supply of caffeine beverages. - - *. It is recommended that you temporarily unpack your linux kernel, - run <make menuconfig>, configure the kernel as per the book and save - the resulting .config file. + The scripts in this directory implement an automation of the building + of a GNU/LInux system, as described in the Linux From Scratch book series. + The name of the project is jhalfs: in that name, "alfs" stands for + "automated linux from scratch", and the initials "jh" have been kept since + the original "jhalfs-0.2" code developed by Jeremy Huntwork. + + The list of supported books can be found at + http://wiki.linuxfromscratch.org/alfs/wiki/SupportedBooks. + + The documentation is split among various README.* files. Here is a list + of what is in which: + - README (this file): instructions to use the LFS book. This should be + enough if you just want to build a base system as per the LFS book. It is + also a required reading for all the other projects. + - README.CLFS: supplementary instructions to use the CLFS book series. + - README.BLFS: instructions to install an automated build infrastructure + for the BLFS book. There are two ways to do so: (i) install the + tools at the end of an LFS build (CLFS is not supported in that case), or + (ii) install the tools on an already running system. Both methods are + described in that file. + - README.CUSTOM: instructions to run custom commands either during the xLFS + build, at the end of a xLFS build. Note that you will not find + instructions on how to write those commands, but some examples are + available. + - README.PACKAGE_MANAGEMENT: instructions to use package management during + the build (only for LFS, patches welcome for CLFS...) + - README.HLFS: very short file explaining why you cannot use HLFS with the + present tool version. - *. Read carefully this file and the other README.* files before using - this tool. + Other sources of information are the context help in the menu interface, + and the xLFS books themselves. 2. PREREQUISITES:: - To use this tool you MUST: - - - have experience building {c,h,b}LFS packages - - know how to edit and write shell scripts - - know how a Makefile works - - be able to trace build failures and to find what is causing them - (user error, package bug, {c,h,b}LFS command bug, or jhalfs code bug) - - If you do not have the above skills, please don't use this tool. + As said elsewhere, it is strongly advised that you first build manually + a complete system before attempting to automate the build. + Of course the "Host System Requirements" should be fulfilled. The needed + supplementary packages are detailed at the bottom of the page: + http://www.linuxfromscratch.org/alfs/download.html 3. INSTALLATION:: - No installation is required. You should just run <make> in this directory. + No installation is required. You may want to move the files in this + directory to a convenient location, and then follow the instructions below. 4. CONFIGURATION:: - Configuration is done through a menu based interface. See the section - RUNNING, for details. + 4.1. CONFIGURATION OF THE TOOLS: + There is no configuration of the tools themselves. The various + parameters for the build are set through a menu driven interface. See + the section RUNNING below for details. + + 4.2. PRELIMINARY TASKS: + This tool has no support at all for creating a partition and a mount + point for the built system. You should follow the book up to the section + "Mounting the new partition". Note that the default name for the + partition mount point is "/mnt/build_dir", instead of /mnt/{c,}lfs. + You can change that default to anything you'd like in the menu, so you + may name it /mnt/lfs, or whatever you like. One important point is that + the user you are logged in as (and not the (c)lfs user) should have write + permission to the mounted directory. We'll use the name /mnt/build_dir + in the sequel. + + The tool can download the needed packages for you, or you may download + them yourself. The tool may optionally use a package archive directory + where the downloaded packages are stored. That directory name may be made + available to the tool in two ways: (i) export the SRC_ARCHIVE variable, + for example SRC_ARCHIVE=/usr/src, (ii) enter the name at the "Package + Archive Directory" menu prompt. Note that the user should have write + permission to that directory. If a needed package is found in that + directory, it is copied to /mnt/build_dir/sources, if not, it is + downloaded to that directory and copied to /mnt/build_dir/sources, + except if found in /mnt/build_dir/sources, in which case, it is just + copied to $SRC_ARCHIVE. If you want the tool to download packages and you + do not want to archive them, just unset SRC_ARCHIVE, and keep the + default entry for "Package Archive Directory". If you choose to download + the packages by yourself, you should download (or copy) them to + /mnt/build_dir/sources directly. + + If you want to build the kernel as part of the automated build, select + "Build the kernel" in the menu. Then, a configuration file must be + provided. In order to do so, it is recommended to download the kernel + tarball, unpack it, run <make menuconfig>, configure the kernel as per + the book, and save the resulting .config file to a location where it can + be retrieved later on (a convenient location and name is + $SRC_ARCHIVE/config-<arch>-<kernel version>-<config details>). + + Another file you may provide is the fstab file. To use it, select + "Use a custom fstab file" in the menu interface, and enter the name of + the file where asked. As for the kernel configuration, this file has to + be prepared before running the menu. A convenient location and name is + $SRC_ARCHIVE/fstablfs. + + At a more advanced level, you may want to supply custom commands + to be run at the end of (C)LFS build. Scripts containing those commands + are located in the ./custom/config directory. Examples are given in + ./custom/examples. A template is provided as ./custom/template. See + README.CUSTOM for more details. 5. RUNNING:: - The command <make> will launch a menu based configuration program. You will - recognize the layout from building the kernel or uClibc/BusyBox. The - underlying menu code was borrowed from BusyBox and slightly modified for - our use. + The command <make> will launch a menu based configuration program. The + underlying menu code was borrowed from BusyBox and slightly modified for + our use. Help on parameter function is available from the on-line help. Please - make use of that feature: it may contain additional information not - duplicated in this file. + make use of that feature: it may contain additional information not + duplicated in this file. You should first choose which book and flavour you want to build. Note - that when you choose the BLFS book, the tool will just install the BLFS - tool to your system. You'll have to run that installed tool to build - packages in BLFS. See README.BLFS to know how. If you choose any other - book, you'll have to configure the settings and the build parameters - from the menu. Note that you may choose to install the blfs tools onto - the newly built system (see below). It is not the same thing as choosing - the BLFS book in the menu, which will install the blfs tools on the - currently running system. - + that when you choose the BLFS book, the tool will just install the BLFS + tool to your system. You'll have to run that installed tool to build + packages in BLFS. See README.BLFS to know how. If you choose any other + book, you'll have to configure the settings and the build parameters + from the menu. Note that you may choose to install the blfs tools onto + the newly built system. It is not the same thing as choosing + the BLFS book in the menu, which will install the blfs tools on the + currently running system. + + The "General Settings" menu is where the "Build Directory" name is to be + entered. Other entries in that menu select what the tool should do. The + "Run the Makefile" entry selects whether the tool will start the build + automatically after generating the needed files. The "Rebuild files" selects + whether to clean the build directory before doing anything else. To protect + against removing important files, this can only be done in an empty directory, + or a directory previously populated by the tool. + + The "Build Settings" menu is where various options for the build can be + selected. Two options, "Use a custom fstab file" and "Build the kernel", + have been described above. "Do not use/display progress_bar", if set, will + prevent a progress bar to be displayed during the build. That may be useful + on slow machine. The other options should be self explanatory, using either + the online help or book reading. + + The "Advanced Features" menu is for various maintenance tasks, like + testing the build instructions or reporting build statistics. One useful + option is "Optimization and parallelisation". It is not recommended to use + it for setting compiler optimization flags, although it is possible, but + if you select it, you'll be able to select the number of parallel `make' + jobs, which allows much faster builds on modern multicore CPUs. Once you have set the parameters and saved the configuration, the script - is launched. Its aim is to extract instructions from the selected book - to generate scripts, and to generate a Makefile, which allows running - the scripts in the right order. The script verifies first that the host - can run it and build the xLFS system, then validates the configuration - and lists the parameters. At this point, you may choose to quit or to - continue with the listed parameters. The script will then proceed to - generate the Makefile and the build scripts, optionally download - packages, and eventually verify the host prerequisite. If you have - selected "Run the makefile", the command make is launched in the - adequate directory, and the build begins. If not, you'll have to run - "make" manually, for example: "make -C /mnt/build_dir/jhalfs", if you - have used the default parameters (see the layout under $BUILDDIR in the - Q&A below). - - IMPORTANT:: - You must be logged as a normal user with sudo privileges to run - the Makefile. Furthermore, you are supposed to have enough privilege - to become any user. If you are not bothered about security issues, - the entry for the user "jhalfs_user" in /etc/sudoers could be - jhalfs_user ALL=(ALL) NOPASSWD:ALL - - NOTE:: - If you run the jhalfs script directly the only function you can select - is to display the version number running <./jhalfs -v> - -6. BLFS_TOOL SUPPORT:: - - For books that support it (only LFS for jhalfs version 2.4), - there is an option to install an automated framework for building BLFS - packages. It is called blfs-tool. When you tick `BOOK Settings/Add - blfs-tool support' in jhalfs configuration menu, the tools are - installed in $BLFS_ROOT (default /blfs_root) on the xLFS system, - and a few dependencies (which you may select) are built at the - end of the jhalfs run, before the custom tools. The instructions for - building the dependencies are taken from the BLFS book. - - (TODO: blfs-tools have not been tested with current (version 3.0) of CLFS, - and certainly need some adaptation to run) - WARNING:: If you add blfs-tool support on a CLFS Sysroot build - you MUST edit the scripts to fix the installation paths. - - After booting the new xLFS system some steps are needed to finish - the installation of the automated tools: - - - A user account must be created. You must be logged on that user - account to use blfs-tool. This is not strictly necessary, - since the packages can be built as root, too, but it is - never a good idea to build packages as root. - - - Move /blfs-root to that user's home and change ownership of the - directory and files to the user. - - - Give the user read and write privileges over the $TRACKING_DIR - directory and the files that it contains. - - - Configure sudo, adding the needed privileges for the user. For - newer sudo version, do not forget to add a line Defaults secure_path= - containing /sbin and /usr/sbin (in /etc/sudoers), otherwise some - executables are not found. - - - Although it is not strictly necessary, it is recommended to install - the bash shell startup files (as per `3.After LFS Configuration - Issues' of the BLFS book), as some instructions in BLFS rely on - their being present. - - We assume that blfs-tool will be used on a running fresh xLFS system. - To use it to build BLFS packages from the chroot jail is also possible, - but not supported. - - To know how to use blfs-tool, see README.BLFS. + is launched. Its aim is to extract instructions from the selected book + to generate scripts, and to generate a Makefile, which allows running + the scripts in the right order. The script verifies first that the host + can run itself and build the xLFS system, then validates the configuration + and lists the parameters. At this point, you may choose to quit or to + continue with the listed parameters. The script will then proceed to + generate the Makefile and the build scripts, optionally download + packages, and eventually verify the host prerequisite. If you have + selected "Run the makefile", the command <make> is launched in the + adequate directory, and the build begins. If not, you'll have to run + "make" manually, for example: "make -C /mnt/build_dir/jhalfs", if you + have used the default parameters (see the layout under $BUILDDIR in the + Q&A below). + + IMPORTANT:: + You must be logged as a normal user with sudo privileges to run + the Makefile. Furthermore, you are supposed to have enough privilege + to become any user. If you are not bothered about security issues, + the entry for the user "jhalfs_user" in /etc/sudoers could be + jhalfs_user ALL=(ALL) NOPASSWD:ALL + + NOTE:: + If you run the jhalfs script directly the only function you can select + is to display the version number by running <./jhalfs -v> -7. LAYOUT:: +6. LAYOUT:: /BLFS (see README.BLFS) @@ -204,21 +233,16 @@ jhalfs blfs-tool -8. FAQ:: - Q. "This 'help' file is very sparse" - A. Yes, it is. This tool, jhalfs, is for those who understand the LFS books - and wish to automate the build. 99% of any problems that arise can be - solved by reading the book(s). - - Q. "It doesn't work!" - A. Yes it does, try >> make - Remember you must have 'sudo' privileges. - - Q. "It still doesn't work" - A. jhalfs was designed to work against the development versions of the LFS - series of books. Consequently changes in a book(s) sometimes breaks older - versions of jhalfs. Before you start pulling out your hair download the - latest version of jhalfs to see if that solves your problem. +7. FAQ:: + Q. "It doesn't work" + A. There are several reasons why it may be so. One possibility is the + following:jhalfs was designed to work against the development versions + of the LFS series of books. Consequently changes in a book(s) sometimes + breaks older versions of jhalfs. Before you start pulling out your hair + download the latest version of jhalfs to see if that solves your + problem. Note that it may be the other way around. If you want to build + an old version of the book, you may have to downgrade you jahlfs + version. Q. "How do I specify the build location?" A. The original LFS document worked against the well known location Modified: jhalfs/trunk/README.BLFS ============================================================================== --- jhalfs/trunk/README.BLFS Mon Mar 27 12:12:06 2017 (r3944) +++ jhalfs/trunk/README.BLFS Mon Mar 27 12:49:04 2017 (r3945) @@ -2,10 +2,6 @@ 1. INTRODUCTION:: - If you want to add blfs-tool support into an xLFS base system build, - read the "BLFS_TOOL SUPPORT" section found in the README and be sure - to follow the after-booting installation instructions. - To automate package builds from the BLFS book instructions is a huge task. Some of the issues are: the BLFS book isn't linear; some package pages use a custom layout; there are circular dependencies; several @@ -20,55 +16,88 @@ 2. PREREQUISITES:: - To use this tool you MUST: - - - have experience building BLFS packages - - know how to edit and write shell scripts - - know how a Makefile works - - be able to trace build failures and to find what is causing it - (user error, package bug, BLFS command bug, or jhalfs code bug) - - If you do not have the above skills, please don't use this tool. - - -3. USAGE:: - - Due to the complexity of the BLFS book, the scripts and Makefile - generation is done in several steps: - - 3.1 INSTALLED PACKAGES TRACKING SYSTEM:: - - This tool includes a very simple tracking system to log which packages - have been installed using the tool. It is used to skip installed packages - from target selection menu and to test if an installed package has been - updated in the BLFS book. Do not rely on this feature as a package - management tool. + In addition to a full LFS system, the following packages and their + dependencies are needed by this tool: + - required: libxml2, libxslt, DocBook XML DTD + - recommended: wget (to download the package tarballs) and sudo (to build + as a user) + - optional: lynx (allows to read the generated linearized book), GPM (to + cut and paste commands from the book), subversion (to update the book + sources), openssl (used by wget for all https:// sites) + Note that the optional dependencies are recommended for ease of use of the + tool. + + You should also have the following personal skills: + - Ability to write and debug shell scripts: as said in the introduction, + not all the generated scripts can be used directly. They need to be + edited to produce an error free build. + - Ability to debug build failures, like missing dependencies or + installation directories not known to the system (when you install in + /opt for example). + - Ability to choose the tools you need to configure and administrate + your system: in the BLFS book, nothing is mandatory, nothing is + useless. You are on your own in choosing what to build, but wrong + decisions may lead to a non functional system... + +3. INSTALL:: + + There are two ways to install the BLFS tools on an LFS system, described + in paragraphs 3.1 and 3.2, respectively: + + 3.1 INSTALLATION ON A RUNNING SYSTEM + + Select "Use Book --> Beyond Linux From Scratch" in the jhalfs menu: + The tools are installed in $HOME$BLFS_ROOT (the default for $BLFS_ROOT + is /blfs_root). The BLFS book is downloaded or copied to its directory. + The tracking directory (see below) is initialized but not created: before + the installation, you should ensure the tracking directory (default location + /var/lib/jhalfs/BLFS) exists and is writable by the user. After the + intallation, you should perform the following additional steps: + + - Configure sudo, adding the needed privileges for the user. For + newer sudo version, do not forget to add a line `Defaults secure_path=' + containing /sbin and /usr/sbin (in /etc/sudoers), otherwise some + executables are not found. + + - Although it is not strictly necessary, it is recommended to install + the bash shell startup files (as per `3.After LFS Configuration + Issues' of the BLFS book), as some instructions in BLFS rely on + their being present. + + 3.2 INSTALLATION ON A JUST BUILT xLFS SYSTEM + + For books that support it (only LFS for jhalfs version 2.4), + there is an option to install the BLFS tools right after building + the xLFS system: just tick `BOOK Settings/Add blfs-tool support' in + jhalfs configuration menu. The tools are installed in $BLFS_ROOT + (default /blfs_root) on the xLFS system, and the dependencies are built + at the end of the jhalfs run, before the custom tools. + + (TODO: blfs-tools have not been tested with current (version 3.0) of CLFS, + and certainly need some adaptation to run) + + After booting the new xLFS system some steps are needed to finish + the installation of the automated tools: + + - A user account must be created. You must be logged on that user + account to use blfs-tool. This is not strictly necessary, + since the packages can be built as root, too, but it is + never a good idea to build packages as root. + + - Move /blfs-root to that user's home and change ownership of the + directory and files to the user. + + - Give the user read and write privileges over the $TRACKING_DIR + directory and the files that it contains. + + - Configure sudo and add the bash shell startup files, as described + above + + We assume that the BLFS tools will be used on a booted xLFS system. + Using them to build BLFS packages in a chroot jail is also possible, + but not supported. - The tracking system itself is an XML file: instpkg.xml. It is - initialized when <make> is first run in blfs_root. It resides in a - directory, which is created when needed during the process of building - custom tools or blfs dependencies, right after xLFS. You can specify - that directory location in the blfs-tools sub-menu of jhalfs. You may - need to update permissions and/or ownership of this directory before - using the blfs tool (see README in jhalfs). - - The default location of the tracking directory is /var/lib/jhalfs/BLFS. - NB : after the initial build, that directory is only used to contain - instpkg.xml, unless custom tools have been built. In the latter case, - it also contains empty files whose name are "$PKG-$VERSION" for each - versionned package built. The information about those packages is - included into instpkg.xml the next time the tool is run. - - 3.2 BLFS_TOOL INSTALLATION:: - - 3.2.1 Normal install - The tools are installed just after the building of xLFS, if the - appropriate options have been selected in the building menu, as per - jhalfs README. If you forgot to select the options and xLFS has been - built, it is possible to go back to selecting the appropriate - BLFS tools options in the jhalfs menu, then tick `Run makefile' - and not `Rebuild files'. You obtain a /blfs_root directory in the - root directory of the new xLFS system, which contains the followings: + 3.3 DIRECTORY LAYOUT IN THE $BLFS_ROOT DIRECTORY blfs-xml/* SVN tree of the selected BLFS book version lib/constants.inc functions libraries @@ -98,17 +127,7 @@ database and the tracking file. envars.conf envars needed when running the target build scripts - 3.2.2 Install to an already running LFS/BLFS system - If you forgot to install the tools when building xLFS, or want to try - the tools, you can select the BLFS book from the jhalfs menu. It will - run a script, which creates the above hierarchy in your home directory and - initialize the tracking file. You have first to make sure that the tracking - dir exists and is writable by the user. You may also populate it with - (empty) files whose names are of the form package-version, for installed - packages, so that they are included into the tracking file. - - 3.3.3 Working files - Several files are generated during the process: + Working files: several files are generated when first running the tool packages.xml auto-generated packages database Config.in input file for the menu driven choices @@ -118,13 +137,38 @@ book-html/* the linearized book rendered in html scripts/* the scriptlets + 3.4 INSTALLED PACKAGES TRACKING SYSTEM: + + This tool includes a very simple tracking system to log which packages + have been installed using the tool. It is used to skip installed packages + from target selection menu and to test if an installed package has been + updated in the BLFS book. Do not rely on this feature as a package + management tool. + + The tracking system itself is an XML file: instpkg.xml. It is + initialized when <make> is first run in blfs_root. It resides in a + directory, which is created when needed during the process of building + custom tools or blfs dependencies, right after xLFS. You can specify + that directory location in the blfs-tools sub-menu of jhalfs. You may + need to update permissions and/or ownership of this directory before + using the blfs tool (see README in jhalfs). + + The default location of the tracking directory is /var/lib/jhalfs/BLFS. + NB : after the initial build, that directory is only used to contain + instpkg.xml, unless custom tools have been built. In the latter case, + it also contains empty files whose name are "$PKG-$VERSION" for each + versionned package built. The information about those packages is + included into instpkg.xml the next time the tool is run. + +4. USAGE:: + From now on, all the work must be done from inside the installation root directory. - You may move that directory to the $HOME of a non root user, or build - as root from that directory. + Due to the complexity of the BLFS book, the scripts and Makefile + generation is done in several steps: - 3.3 UPDATING BOOK SOURCES:: + 4.1 UPDATING BOOK SOURCES:: If you are using the development book version and you want to update installed packages to the latest version found in that book, you need to @@ -142,7 +186,7 @@ with a new version in the book will be available for target selection and used to solve dependencies. - 3.4 CONFIGURING AND PARSING THE BOOK:: + 4.2 CONFIGURING AND PARSING THE BOOK:: The next step is to create a book and build scripts in dependency build order for one or several packages. @@ -180,7 +224,7 @@ There is also another directory, "dependencies" that contains files generated while resolving dependencies. - 3.5 EDITING BUILD SCRIPTS:: + 4.3 EDITING BUILD SCRIPTS:: Now it is time to review the generated book and scripts, making any changes to the scripts necessary to fix generation bugs or to suit your @@ -197,7 +241,7 @@ Also, review and edit envars.conf. This file is used to set global envars needed by the build scripts. - 3.6 CREATING THE MAKEFILE:: + 4.4 CREATING THE MAKEFILE:: When the build scripts are ready to be run, the Makefile can be created. Create an empty directory (for example "mkdir work") and cd @@ -206,7 +250,7 @@ Review the Makefile, and, if all looks sane, start the build by running "make". -4. GENERATED BUILD SCRIPTS ISSUES:: +5. GENERATED BUILD SCRIPTS ISSUES:: In this section, known issues with the generated build scripts are discussed. They are due to build procedures and/or BLFS layout @@ -222,14 +266,14 @@ to another name (using the xxx-a- fields), and possibly renaming the xxx-a- fields of each involved script. - 4.1 BLFS BOOTSCRIPTS:: + 5.1 BLFS BOOTSCRIPTS:: Normally, bootscript installation should work. On the other hand, the book does not give instruction for running them, so you might have to manually insert "/etc/init.d/rc.d/<initscript> start" at some place during the build. - 4.2 PACKAGE CONFIGURATION:: + 5.2 PACKAGE CONFIGURATION:: For those packages that have a "Configuration" section, you should edit the build script to fit the needs of your system. Sometimes, the @@ -237,7 +281,7 @@ line 'source /etc/profile', which ensures that the proper environment variables are used. - 4.3 PAGES WITH TWO OR MORE PACKAGES:: + 5.3 PAGES WITH TWO OR MORE PACKAGES:: For example: sane, poppler, audacious, freetts, which, etc. @@ -248,7 +292,7 @@ name (for example PKG1, PKG2, etc) after each download. The unpacking instructions may need to be repeated for each tarball in turn. - 4.4 XORG7 + 5.4 XORG7 The book has special page layouts for the Xorg7 packages. The tool breaks those pages into individual pages for each packages in the linear @@ -257,13 +301,13 @@ To build the whole Xorg7 chapter, select twm. The (recommended) dependency chain brings in the whole set of Xorg packages. - 4.5 PATCHES + 5.5 PATCHES Please, make sure that all scripts have the commands to download/apply the required patches. Due to book layout issues, some patches may be missing (as of BLFS 8.0, all the patches seem to be downloaded). - 4.6 ROOT COMMANDS + 5.6 ROOT COMMANDS If building as a normal user (the default setting), be sure that all commands that require root privileges are run using sudo. Also make sure @@ -292,7 +336,7 @@ Due to book layout issues, some sudo commands may be missing. - 4.7 OTHERS + 5.7 OTHERS There may be other issues that we are not aware of. If you find any, please report it to <alfs-disc...@linuxfromscratch.org>. -- http://lists.linuxfromscratch.org/listinfo/alfs-log Unsubscribe: See the above information page
