Repository: incubator-hawq-docs Updated Branches: refs/heads/develop af1299d7a -> acf2b91fb
hawq no longer embeds a python installation (closes #54) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/acf2b91f Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/acf2b91f Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/acf2b91f Branch: refs/heads/develop Commit: acf2b91fb64f502ea4a4823b89e82ce944c3be5d Parents: af1299d Author: David Yozie <[email protected]> Authored: Mon Nov 7 15:31:09 2016 -0800 Committer: David Yozie <[email protected]> Committed: Mon Nov 7 15:31:09 2016 -0800 ---------------------------------------------------------------------- plext/using_plpython.html.md.erb | 42 +++++++++-------------------------- 1 file changed, 11 insertions(+), 31 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/acf2b91f/plext/using_plpython.html.md.erb ---------------------------------------------------------------------- diff --git a/plext/using_plpython.html.md.erb b/plext/using_plpython.html.md.erb index 970f966..ff7b527 100644 --- a/plext/using_plpython.html.md.erb +++ b/plext/using_plpython.html.md.erb @@ -8,11 +8,7 @@ This section contains an overview of the HAWQ PL/Python language extension. PL/Python is a loadable procedural language. With the HAWQ PL/Python extension, you can write HAWQ user-defined functions in Python that take advantage of Python features and modules to quickly build robust database applications. -If PL/Python is enabled during HAWQ build time, HAWQ includes both a version of Python and PL/Python when deployed. HAWQ uses the following Python installation: - -```shell -$GPHOME/ext/python/ -``` +HAWQ uses the system Python installation. ### <a id="hawqlimitations"></a>HAWQ PL/Python Limitations @@ -21,11 +17,9 @@ $GPHOME/ext/python/ ## <a id="enableplpython"></a>Enabling and Removing PL/Python Support -If enabled as an option during HAWQ compilation, the PL/Python language is installed with HAWQ. - -**Note**: To use PL/Python in HAWQ, you must either use a pre-compiled version of HAWQ that includes PL/Python or specify PL/Python as a build option when compiling HAWQ. +To use PL/Python in HAWQ, you must either use a pre-compiled version of HAWQ that includes PL/Python or specify PL/Python as a build option when compiling HAWQ. -To create and run a PL/Python user-defined function (UDF) in a database, you must register the PL/Python language with the database. On every database where you want to install and enable PL/Python, connect to the database using the psql client. +To create and run a PL/Python user-defined function (UDF) in a database, you must register the PL/Python language with the database. On every database where you want to install and enable PL/Python, connect to the database using the `psql` client. ```shell $ psql -d <dbname> @@ -41,7 +35,7 @@ psql# CREATE LANGUAGE plpythonu; Note that `plpythonu` is installed as an âuntrustedâ language, meaning it does not offer any way of restricting what users can do in it. -To remove support for plpythonu from a database, run the following SQL command: +To remove support for `plpythonu` from a database, run the following SQL command: ```shell psql# DROP LANGUAGE plpythonu; @@ -51,7 +45,7 @@ psql# DROP LANGUAGE plpythonu; The body of a PL/Python user-defined function is a Python script. When the function is called, its arguments are passed as elements of the array `args[]`. Named arguments are also passed as ordinary variables to the Python script. The result is returned from the PL/Python function with return statement, or yield statement in case of a result-set statement. -The HAWQ PL/Python language module imports the Python module `plpy`. The module plpy implements these functions: +The HAWQ PL/Python language module imports the Python module `plpy`. The module `plpy` implements these functions: - Functions to execute SQL queries and prepare execution plans for queries. - `plpy.execute` @@ -144,32 +138,18 @@ In terms of performance, importing a Python module is an expensive operation and ```sql psql=# - CREATE FUNCTION pytest() RETRUNS text AS $$ + CREATE FUNCTION pytest() RETURNS text AS $$ if 'mymodule' not in GD: import mymodule GD['mymodule'] = mymodule return GD['mymodule'].sumd([1,2,3]) -$$; +$$ LANGUAGE plpythonu; ``` ## <a id="installpythonmodules"></a>Installing Python Modules -When you install a Python module on HAWQ, the HAWQ Python environment must have the module added to it across all segment hosts in the cluster. When expanding HAWQ, you must add the Python modules to the new segment hosts. You can use the HAWQ utilities `hawq ssh` and `hawq scp` run commands on HAWQ hosts and copy files to the hosts. For information about the utilities, see the [HAWQ Management Tools Reference](../reference/cli/management_tools.html). - -As part of the HAWQ installation, the `gpadmin` user environment is configured to use Python that is installed with HAWQ. - -To check which Python is being used in your environment, use the `which` command: - -```bash -$ which python -``` - -The command returns the location of the Python installation. The Python installed with HAWQ is in the HAWQ `ext/python` directory. - -```bash -/$GPHOME/ext/python/bin/python -``` - +HAWQ is configured to use the system Python. When you install a Python module on HAWQ, you must add the module to all segment hosts in the cluster. When expanding HAWQ, you must add Python modules to the new segment hosts. You can use the HAWQ utilities `hawq ssh` and `hawq scp` to run commands on HAWQ hosts and copy files to the hosts. For information about the utilities, see the [HAWQ Management Tools Reference](../reference/cli/management_tools.html). + If you are building a Python module, you must ensure that the build creates the correct executable. For example on a Linux system, the build should create a 64-bit executable. Before building a Python module prior to installation, ensure that the appropriate software to build the module is installed and properly configured. The build environment is required only on the host where you build the module. @@ -335,7 +315,7 @@ Before build and install the NumPy module, you install the OpenBLAS libraries. T 4. You can use the `hawq ssh` utility to build and install the OpenBLAS libraries on multiple hosts. - All HAWQ hosts (master and segment hosts) have identical configurations. You can copy the OpenBLAS libraries from the system where they were built instead of building the OpenBlas libraries on all the hosts. For example, these `hawq ssh` and `hawq scp commands copy and install the OpenBLAS libraries on the hosts listed in the hawq-hosts file. + All HAWQ hosts (master and segment hosts) have identical configurations. You can copy the OpenBLAS libraries from the system where they were built instead of building the OpenBlas libraries on all the hosts. For example, these `hawq ssh` and `hawq scp` commands copy and install the OpenBLAS libraries on the hosts listed in the `hawq-hosts` file. ```bash $ hawq ssh -f hawq-hosts -e 'sudo yum -y install gcc gcc-gfortran gcc-c++' @@ -354,7 +334,7 @@ $ hawq ssh -f hawq-hosts #### Build and Install NumPy <a name="buildinstallnumpy"></a> -After you have installed the OpenBLAS libraries, you can build and install NumPy module. These steps install the NumPy module on a single host. You can use the hawq ssh utility to build and install the NumPy module on multiple hosts. +After you have installed the OpenBLAS libraries, you can build and install NumPy module. These steps install the NumPy module on a single host. You can use the `hawq ssh` utility to build and install the NumPy module on multiple hosts. 1. Go to the packages subdirectory and get the NumPy module source and extract the files.
