rename to .html.md.erb
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/c73dab31 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/c73dab31 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/c73dab31 Branch: refs/heads/tutorial-proto Commit: c73dab31b0be8137432d9473a385762b008727a5 Parents: f9bb8a2 Author: Lisa Owen <[email protected]> Authored: Mon Oct 31 13:47:16 2016 -0700 Committer: Lisa Owen <[email protected]> Committed: Mon Oct 31 13:47:16 2016 -0700 ---------------------------------------------------------------------- tutorial/admin/admin_overview.html.md.erb | 21 +++ tutorial/admin/admin_overview.md | 21 --- .../datascientist/datasci_overview.html.md.erb | 21 +++ tutorial/datascientist/datasci_overview.md | 21 --- .../gettingstarted/basichawqadmin.html.md.erb | 181 +++++++++++++++++++ tutorial/gettingstarted/basichawqadmin.md | 181 ------------------- .../getstart_overview.html.md.erb | 21 +++ tutorial/gettingstarted/getstart_overview.md | 21 --- .../gettingstarted/introhawqtbls.html.md.erb | 169 +++++++++++++++++ tutorial/gettingstarted/introhawqtbls.md | 169 ----------------- 10 files changed, 413 insertions(+), 413 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/admin/admin_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/tutorial/admin/admin_overview.html.md.erb b/tutorial/admin/admin_overview.html.md.erb new file mode 100644 index 0000000..bc2a656 --- /dev/null +++ b/tutorial/admin/admin_overview.html.md.erb @@ -0,0 +1,21 @@ +--- +title: HAWQ for Administrators +--- + + +## <a id="tut_getstartov"></a>Overview + +These exercises present typical and advanced HAWQ cluster operation and administration activities. + + +## <a id="tut_getstartov_prereq"></a>Prerequisites + +Ensure that you have a running HAWQ installation and that you have downloaded the tutorial work files, including the Retail example data set and HAWQ scripts. + +## <a id="tut_hawqexlist"></a>Exercises + +Administration exercises include: + +LIST +HERE + http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/admin/admin_overview.md ---------------------------------------------------------------------- diff --git a/tutorial/admin/admin_overview.md b/tutorial/admin/admin_overview.md deleted file mode 100644 index bc2a656..0000000 --- a/tutorial/admin/admin_overview.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: HAWQ for Administrators ---- - - -## <a id="tut_getstartov"></a>Overview - -These exercises present typical and advanced HAWQ cluster operation and administration activities. - - -## <a id="tut_getstartov_prereq"></a>Prerequisites - -Ensure that you have a running HAWQ installation and that you have downloaded the tutorial work files, including the Retail example data set and HAWQ scripts. - -## <a id="tut_hawqexlist"></a>Exercises - -Administration exercises include: - -LIST -HERE - http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/datascientist/datasci_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/tutorial/datascientist/datasci_overview.html.md.erb b/tutorial/datascientist/datasci_overview.html.md.erb new file mode 100644 index 0000000..a6bc425 --- /dev/null +++ b/tutorial/datascientist/datasci_overview.html.md.erb @@ -0,0 +1,21 @@ +--- +title: HAWQ for Data Scientists +--- + + +## <a id="tut_getstartov"></a>Overview + +HAWQ provides advanced analytic capabilities for data scientists including X, Y, Z. + + +## <a id="tut_getstartov_prereq"></a>Prerequisites + +Ensure that you have a running HAWQ installation and that you have downloaded the tutorial work files, including the Retail example data set and HAWQ scripts. + +## <a id="tut_hawqexlist"></a>Exercises + +Analytics exercises include: + +LIST +HERE + http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/datascientist/datasci_overview.md ---------------------------------------------------------------------- diff --git a/tutorial/datascientist/datasci_overview.md b/tutorial/datascientist/datasci_overview.md deleted file mode 100644 index a6bc425..0000000 --- a/tutorial/datascientist/datasci_overview.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: HAWQ for Data Scientists ---- - - -## <a id="tut_getstartov"></a>Overview - -HAWQ provides advanced analytic capabilities for data scientists including X, Y, Z. - - -## <a id="tut_getstartov_prereq"></a>Prerequisites - -Ensure that you have a running HAWQ installation and that you have downloaded the tutorial work files, including the Retail example data set and HAWQ scripts. - -## <a id="tut_hawqexlist"></a>Exercises - -Analytics exercises include: - -LIST -HERE - http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/gettingstarted/basichawqadmin.html.md.erb ---------------------------------------------------------------------- diff --git a/tutorial/gettingstarted/basichawqadmin.html.md.erb b/tutorial/gettingstarted/basichawqadmin.html.md.erb new file mode 100644 index 0000000..32b354c --- /dev/null +++ b/tutorial/gettingstarted/basichawqadmin.html.md.erb @@ -0,0 +1,181 @@ +--- +title: HAWQ Administration Basics +--- + +You can manage your HAWQ cluster from either the command line or the Ambari web console. + +In this tutorial section, you will be introduced to basic HAWQ and database administration tasks. You will manage aspects of your HAWQ cluster from both the command line and Ambari. + +## <a id="tut_adminprereq"></a> Prerequisites +You must have a running HAWQ installation. + +## <a id="tut_adminsetupenv"></a> HAWQ Environment + +The `/usr/local/hawq/greenplum_path.sh` script sets up access to the HAWQ components on your system. This script sets your `$PATH` and other environment variables to find HAWQ files. Most importantly, it sets the `$GPHOME` environment variable to point to the root of the HAWQ installation. The HAWQ root directory is `/usr/local/hawq` in the sandbox VM. + +Source `greenplum_path.sh` before invoking any HAWQ commands. + +``` shell +$ . /usr/local/hawq/greenplum_path.sh +``` + +Add the command above to your `.bash_profile` to set up your HAWQ environment on every log in. + +## <a id="tut_adminsetupenv"></a> HAWQ Administrator + +`gpadmin` is the HAWQ administrator user name. The `gpadmin` user has super-user capabilities on all HAWQ databases and HAWQ cluster management commands. + +## <a id="tut_adminclustermgmt"></a>HAWQ Management - Command Line + +The following table identifies a subset of HAWQ management commands that are applicable to the tutorial exercises. For detailed information on HAWQ management commands, refer to the [HAWQ Management Tools Reference](../../reference/cli/management_tools.html). + +<caption><span class="tablecap">Table 1. Common HAWQ Management Commands</span></caption> + +<a id="topic_table_clustmgmtcmd"></a> + +| Action | Command | +|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| get HAWQ cluster status | `$ hawq state` | +| start/stop/restart HAWQ object (cluster, master, segment, standby, allsegments) | `$ hawq start object` <p> `$ hawq stop object` <p> `$ hawq restart object` | +| list HAWQ configuration parameters and values | `$ hawq config --list` | +| change value of HAWQ configuration parameter | `$ hawq config -c parameter_name -v new_value` | +| reload HAWQ configuration files | `$ hawq stop --reload` | +| restart PXF service | `$ sudo service pxf-service restart` | +| extract/dump database | `$ pg_dump -f output_file dbname` | + +### <a id="tut_adminclustermgmthstate"></a>HAWQ State + +Use the `hawq state` command to obtain the state of your HAWQ cluster: + +``` shell +$ hawq state +DATETIME hawq_state:sandbox:gpadmin-[INFO]:--HAWQ instance status summary +DATETIME hawq_state:sandbox:gpadmin-[INFO]:------------------------------------------------------ +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Master instance = Active +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- No Standby master defined +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segment instance count from config file = 1 +DATETIME hawq_state:sandbox:gpadmin-[INFO]:------------------------------------------------------ +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Segment Status +DATETIME hawq_state:sandbox:gpadmin-[INFO]:------------------------------------------------------ +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segments count from catalog = 1 +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segment valid (at master) = 1 +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segment failures (at master) = 0 +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total number of postmaster.pid files missing = 0 +DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total number of postmaster.pid files found = 1 +``` + +State information returned includes the status of the master node, standby master, number of segment instances, and for each segment, number valid and failed. + +### <a id="tut_adminclustermgmthcfgparam"></a>HAWQ Configuration Parameters + +HAWQ configuration parameters affect the behaviour of the HAWQ system. The collection of HAWQ configuration parameters and values are stored in the `$GPHOME/etc/hawq-site.xml` file. Use the `hawq config` command to view and set the values of HAWQ configuration parameters from the command line. + +Display the value of the segment temporary directory: + +``` shell +$ hawq config --show hawq_segment_temp_directory +GUC : hawq_segment_temp_directory +Value : /tmp/hawq/segment +``` + +Set the PL/Java classpath configuration parameter value: + +``` shell +$ hawq config -c pljava_classpath -v \'examples.jar\' +GUC pljava_classpath does not exist in hawq-site.xml +Try to add it with value: examples.jar +GUC : pljava_classpath +Value : examples.jar +``` + +As the message indicates, `pljava_classpath` was not previously set. + +HAWQ configuration must be reloaded after setting a configuration parameter. + +``` shell +$ hawq stop --reload +``` + +`hawq config` will propagate the configuration parameter value changes across the cluster. Never modify the `hawq-site.xml` file directly. + + +## <a id="tut_adminclustermgmt"></a>HAWQ Management - Ambari Web UI + +You can also use the Ambari Web UI to manage your HAWQ deployment. + +?? CHANGE RUNTIME CONFIG PARAM AND STOP --reload AND DISCUSS ?? + +1. Start the Ambari web UI by entering the the IP address of your sandbox VM followed by `:8080` in the URL of a browser window. If you do not remember, view the `/etc/hosts` file to get your VM IP address: + + ``` shell + vm_ipaddr:8080 + ``` + + Ambari runs on port 8080. + +2. Log in to Ambari; Ambari user credentials in the sandbox VM are `admin`/`admin`. + + After logging in, you are presented with the Ambari dashboard. The dashboard provides an at-a-glance status of the health of your HAWQ cluster. A list of running services and their status is displayed in the left panel. The main display area includes a set of configurable tiles providing specific information about your cluster, including HDFS disk usage and resource manager metrics. + +3. Select the *HAWQ* service from the service list. + + The HAWQ service page *Summary* tab is displayed. This page includes a *Summary* pane identifying the HAWQ master and all HAWQ segment nodes in your cluster. The *Metrics* pane includes a set of HAWQ-specific metrics tiles. + +4. View HAWQ-specific configuration settings by selecting the *Configs* tab. + + HAWQ general settings displayed include master and segment data and temp directories, and specific resource management parameters. + +5. Select the *Advanced* tab to view additional HAWQ parameter settings. + + The *General* drop down pane is opened. In addition to displaying master and segment port information, this pane is where the HAWQ administrator would (re)set the `gpadmin` password. + + *Advanced* and *Custom* drop down panes provide access to advanced configuration settings for HAWQ, HDFS, and YARN components. + +6. Select the *Advanced hawq-site* drop down. + + Specific configuration parameters and values are displayed in the pane. Hover the mouse cursor over the value field to display a tooltip description of a specific configuration parameter. + + +## <a id="tut_admindbaccess"></a>HAWQ Database + +The HAWQ PostgreSQL database XXXX + +??DESCRIBE HOW IT DOES/DOES NOT CONFORM TO STANDARD?? + +?? DESCRIBE HAWQ DATABASES/TABLES CREATED AT INSTALL TIME ?? + +### <a id="tut_admindbaccesscmds"></a>Database Commands + +HAWQ databases and roles can be created and deleted by authorized users via the command line. + +<caption><span class="tablecap">Table 2. Common HAWQ Database Commands</span></caption> + +<a id="topic_table_dbcmd"></a> + +| Action | Command | +|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| create database | `createdb` | +| created database role | `createuser` | +| remove database | `dropdb` | +| remove database role | `dropuser` | +| interactive HAWQ database front-end | `psql` | +| run SQL command directly | `psql -c SQL_COMMAND` | + + +### <a id="tut_admindbaccesspsql"></a>psql + +You can also manage and access HAWQ databases via the `psql` utility, an interactive front-end to the HAWQ PostgreSQL database. Commonly-used `psql` meta commands include: + +| Action | Command | +|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| list databases | `\l` | +| list tables in current database | `\dt` | +| get schema info for specific table | `\d tablename` | +| read SQL script | `\i scriptname` | +| quit | `\q` | + +For detailed information on `psql` commands, refer to the [psql](../../reference/cli/client_utilities/psql.html) reference page. + +### <a id="tut_admindbsql"></a>SQL + +For specific information on SQL command support in HAWQ databases, refer to the [SQL Command Reference](../../reference/SQLCommandReference.html). http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/gettingstarted/basichawqadmin.md ---------------------------------------------------------------------- diff --git a/tutorial/gettingstarted/basichawqadmin.md b/tutorial/gettingstarted/basichawqadmin.md deleted file mode 100644 index 32b354c..0000000 --- a/tutorial/gettingstarted/basichawqadmin.md +++ /dev/null @@ -1,181 +0,0 @@ ---- -title: HAWQ Administration Basics ---- - -You can manage your HAWQ cluster from either the command line or the Ambari web console. - -In this tutorial section, you will be introduced to basic HAWQ and database administration tasks. You will manage aspects of your HAWQ cluster from both the command line and Ambari. - -## <a id="tut_adminprereq"></a> Prerequisites -You must have a running HAWQ installation. - -## <a id="tut_adminsetupenv"></a> HAWQ Environment - -The `/usr/local/hawq/greenplum_path.sh` script sets up access to the HAWQ components on your system. This script sets your `$PATH` and other environment variables to find HAWQ files. Most importantly, it sets the `$GPHOME` environment variable to point to the root of the HAWQ installation. The HAWQ root directory is `/usr/local/hawq` in the sandbox VM. - -Source `greenplum_path.sh` before invoking any HAWQ commands. - -``` shell -$ . /usr/local/hawq/greenplum_path.sh -``` - -Add the command above to your `.bash_profile` to set up your HAWQ environment on every log in. - -## <a id="tut_adminsetupenv"></a> HAWQ Administrator - -`gpadmin` is the HAWQ administrator user name. The `gpadmin` user has super-user capabilities on all HAWQ databases and HAWQ cluster management commands. - -## <a id="tut_adminclustermgmt"></a>HAWQ Management - Command Line - -The following table identifies a subset of HAWQ management commands that are applicable to the tutorial exercises. For detailed information on HAWQ management commands, refer to the [HAWQ Management Tools Reference](../../reference/cli/management_tools.html). - -<caption><span class="tablecap">Table 1. Common HAWQ Management Commands</span></caption> - -<a id="topic_table_clustmgmtcmd"></a> - -| Action | Command | -|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| get HAWQ cluster status | `$ hawq state` | -| start/stop/restart HAWQ object (cluster, master, segment, standby, allsegments) | `$ hawq start object` <p> `$ hawq stop object` <p> `$ hawq restart object` | -| list HAWQ configuration parameters and values | `$ hawq config --list` | -| change value of HAWQ configuration parameter | `$ hawq config -c parameter_name -v new_value` | -| reload HAWQ configuration files | `$ hawq stop --reload` | -| restart PXF service | `$ sudo service pxf-service restart` | -| extract/dump database | `$ pg_dump -f output_file dbname` | - -### <a id="tut_adminclustermgmthstate"></a>HAWQ State - -Use the `hawq state` command to obtain the state of your HAWQ cluster: - -``` shell -$ hawq state -DATETIME hawq_state:sandbox:gpadmin-[INFO]:--HAWQ instance status summary -DATETIME hawq_state:sandbox:gpadmin-[INFO]:------------------------------------------------------ -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Master instance = Active -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- No Standby master defined -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segment instance count from config file = 1 -DATETIME hawq_state:sandbox:gpadmin-[INFO]:------------------------------------------------------ -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Segment Status -DATETIME hawq_state:sandbox:gpadmin-[INFO]:------------------------------------------------------ -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segments count from catalog = 1 -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segment valid (at master) = 1 -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total segment failures (at master) = 0 -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total number of postmaster.pid files missing = 0 -DATETIME hawq_state:sandbox:gpadmin-[INFO]:-- Total number of postmaster.pid files found = 1 -``` - -State information returned includes the status of the master node, standby master, number of segment instances, and for each segment, number valid and failed. - -### <a id="tut_adminclustermgmthcfgparam"></a>HAWQ Configuration Parameters - -HAWQ configuration parameters affect the behaviour of the HAWQ system. The collection of HAWQ configuration parameters and values are stored in the `$GPHOME/etc/hawq-site.xml` file. Use the `hawq config` command to view and set the values of HAWQ configuration parameters from the command line. - -Display the value of the segment temporary directory: - -``` shell -$ hawq config --show hawq_segment_temp_directory -GUC : hawq_segment_temp_directory -Value : /tmp/hawq/segment -``` - -Set the PL/Java classpath configuration parameter value: - -``` shell -$ hawq config -c pljava_classpath -v \'examples.jar\' -GUC pljava_classpath does not exist in hawq-site.xml -Try to add it with value: examples.jar -GUC : pljava_classpath -Value : examples.jar -``` - -As the message indicates, `pljava_classpath` was not previously set. - -HAWQ configuration must be reloaded after setting a configuration parameter. - -``` shell -$ hawq stop --reload -``` - -`hawq config` will propagate the configuration parameter value changes across the cluster. Never modify the `hawq-site.xml` file directly. - - -## <a id="tut_adminclustermgmt"></a>HAWQ Management - Ambari Web UI - -You can also use the Ambari Web UI to manage your HAWQ deployment. - -?? CHANGE RUNTIME CONFIG PARAM AND STOP --reload AND DISCUSS ?? - -1. Start the Ambari web UI by entering the the IP address of your sandbox VM followed by `:8080` in the URL of a browser window. If you do not remember, view the `/etc/hosts` file to get your VM IP address: - - ``` shell - vm_ipaddr:8080 - ``` - - Ambari runs on port 8080. - -2. Log in to Ambari; Ambari user credentials in the sandbox VM are `admin`/`admin`. - - After logging in, you are presented with the Ambari dashboard. The dashboard provides an at-a-glance status of the health of your HAWQ cluster. A list of running services and their status is displayed in the left panel. The main display area includes a set of configurable tiles providing specific information about your cluster, including HDFS disk usage and resource manager metrics. - -3. Select the *HAWQ* service from the service list. - - The HAWQ service page *Summary* tab is displayed. This page includes a *Summary* pane identifying the HAWQ master and all HAWQ segment nodes in your cluster. The *Metrics* pane includes a set of HAWQ-specific metrics tiles. - -4. View HAWQ-specific configuration settings by selecting the *Configs* tab. - - HAWQ general settings displayed include master and segment data and temp directories, and specific resource management parameters. - -5. Select the *Advanced* tab to view additional HAWQ parameter settings. - - The *General* drop down pane is opened. In addition to displaying master and segment port information, this pane is where the HAWQ administrator would (re)set the `gpadmin` password. - - *Advanced* and *Custom* drop down panes provide access to advanced configuration settings for HAWQ, HDFS, and YARN components. - -6. Select the *Advanced hawq-site* drop down. - - Specific configuration parameters and values are displayed in the pane. Hover the mouse cursor over the value field to display a tooltip description of a specific configuration parameter. - - -## <a id="tut_admindbaccess"></a>HAWQ Database - -The HAWQ PostgreSQL database XXXX - -??DESCRIBE HOW IT DOES/DOES NOT CONFORM TO STANDARD?? - -?? DESCRIBE HAWQ DATABASES/TABLES CREATED AT INSTALL TIME ?? - -### <a id="tut_admindbaccesscmds"></a>Database Commands - -HAWQ databases and roles can be created and deleted by authorized users via the command line. - -<caption><span class="tablecap">Table 2. Common HAWQ Database Commands</span></caption> - -<a id="topic_table_dbcmd"></a> - -| Action | Command | -|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| create database | `createdb` | -| created database role | `createuser` | -| remove database | `dropdb` | -| remove database role | `dropuser` | -| interactive HAWQ database front-end | `psql` | -| run SQL command directly | `psql -c SQL_COMMAND` | - - -### <a id="tut_admindbaccesspsql"></a>psql - -You can also manage and access HAWQ databases via the `psql` utility, an interactive front-end to the HAWQ PostgreSQL database. Commonly-used `psql` meta commands include: - -| Action | Command | -|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| list databases | `\l` | -| list tables in current database | `\dt` | -| get schema info for specific table | `\d tablename` | -| read SQL script | `\i scriptname` | -| quit | `\q` | - -For detailed information on `psql` commands, refer to the [psql](../../reference/cli/client_utilities/psql.html) reference page. - -### <a id="tut_admindbsql"></a>SQL - -For specific information on SQL command support in HAWQ databases, refer to the [SQL Command Reference](../../reference/SQLCommandReference.html). http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/gettingstarted/getstart_overview.html.md.erb ---------------------------------------------------------------------- diff --git a/tutorial/gettingstarted/getstart_overview.html.md.erb b/tutorial/gettingstarted/getstart_overview.html.md.erb new file mode 100644 index 0000000..c5168e7 --- /dev/null +++ b/tutorial/gettingstarted/getstart_overview.html.md.erb @@ -0,0 +1,21 @@ +--- +title: Getting Started with HAWQ +--- + + +## <a id="tut_getstartov"></a>Overview + +This tutorial provides a quick introduction to get you up and running with your HAWQ installation. You will be introduced to basic HAWQ functionality, including cluster management, database creation, and simple querying. An introduction to external data sources is also provided. + + +## <a id="tut_getstartov_prereq"></a>Prerequisites + +Ensure that you have a running HAWQ installation and that you have the tutorial work files, including the Retail sample data set and HAWQ scripts. + +## <a id="tut_hawqexlist"></a>Exercises + +This HAWQ introduction includes the following content and exercises: + +[HAWQ Administration Basics](basichawqadmin.html) +[Introduction to HAWQ Tables](introhawqtbls.html) +[Introduction to PXF](intropxfhdfs.html) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/gettingstarted/getstart_overview.md ---------------------------------------------------------------------- diff --git a/tutorial/gettingstarted/getstart_overview.md b/tutorial/gettingstarted/getstart_overview.md deleted file mode 100644 index c5168e7..0000000 --- a/tutorial/gettingstarted/getstart_overview.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -title: Getting Started with HAWQ ---- - - -## <a id="tut_getstartov"></a>Overview - -This tutorial provides a quick introduction to get you up and running with your HAWQ installation. You will be introduced to basic HAWQ functionality, including cluster management, database creation, and simple querying. An introduction to external data sources is also provided. - - -## <a id="tut_getstartov_prereq"></a>Prerequisites - -Ensure that you have a running HAWQ installation and that you have the tutorial work files, including the Retail sample data set and HAWQ scripts. - -## <a id="tut_hawqexlist"></a>Exercises - -This HAWQ introduction includes the following content and exercises: - -[HAWQ Administration Basics](basichawqadmin.html) -[Introduction to HAWQ Tables](introhawqtbls.html) -[Introduction to PXF](intropxfhdfs.html) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/gettingstarted/introhawqtbls.html.md.erb ---------------------------------------------------------------------- diff --git a/tutorial/gettingstarted/introhawqtbls.html.md.erb b/tutorial/gettingstarted/introhawqtbls.html.md.erb new file mode 100644 index 0000000..041ed3e --- /dev/null +++ b/tutorial/gettingstarted/introhawqtbls.html.md.erb @@ -0,0 +1,169 @@ +--- +title: Introduction to HAWQ Tables +--- + +## <a id="tut_introhawqtbldisc"></a>Discussion + +DISCUSS HAWQ TABLES HERE + +In this exercise, you will create a HAWQ table, add data to the table, and perform simple queries. You will also load the Retail sample data set into HAWQ tables and perform more complicated queries. + +## <a id="tut_introhawqtblprereq"></a>Prerequisites + +Ensure that you have a running HAWQ installation and have downloaded the tutorial work files, including the Retail example data set and HAWQ scripts. + +## <a id="tut_excreatehawqtblsteps"></a>Steps + +Perform the following steps to create a HAWQ database and associated tables to represent the Retail data set. + +1. In a terminal window, navigate to the tutorial work files base directory. + + ``` shell + $ cd $HAWQTUTBASE + ``` + +2. Run the `createdb` command at the shell prompt to create a database named `hawqintro_tbl1`: + + ``` shell + $ createdb hawqintro_tbl1 + ``` + +3. Run the PostgreSQL interpreter `psql` and connect to database `hawqintro_tbl1`: + + ``` shell + $ psql -d hawqintro_tbl1 + ``` + + You will enter the `psql` interpreter: + + ``` sql + hawqintro_tbl1=# + ``` + + Notice that the `psql` prompt is the database name followed by `=#`. + +4. Create the table `id_tbl1` with a single column named `id` of type `integer`: + + ``` sql + hawqintro_tbl1=# create table id_tbl1 (id int); + ``` + +5. Add some data to `id_tbl1`: + + ``` sql + hawqintro_tbl1=# insert into id_tbl1 select generate_series(1,100); + ``` + + This `insert` command adds 100 rows to `id_tbl1`, incrementing and writing the `id` for each row. + +6. Query the table: + + ``` sql + hawqintro_tbl1=# select * from id_tbl1 where id > 97; + ``` + ``` shell + ** id + ----- + 98 + 99 + 100 + (3 rows)** + ``` + + The query returns all rows in the table where the `id` is greater than 97. The query also returns the number of rows returned. + +7. Quit and restart `psql`. + + ``` sql + hawqintro_tbl1=#\q + ``` + + ``` shell + $ cd $HAWQTUTBASE/sample_data + $ psql + ``` + +8. Create a schema named `retail_demo` to represent the Retail data set: + + ``` sql + gpadmin=# CREATE SCHEMA retail_demo; + ``` + + A schema is a namespace for the database. It contains named objects like tables, data types, functions, and operators. These named objects are accessed by qualifying their name with `schemaname.` (or `retail_demo.` in this exercise) as a prefix. You may also set a search path that includes schema name(s). + +7. Use the provided scripts to create tables for, load, and verify the Retail demo data set: + + ``` sql + gpadmin=#\i ../hawq/hawq_tables/create_hawq_tables.sql + gpadmin=#\q + ``` + The `create_hawq_tables.sql` script deletes each table before creating. If this is your first time performing this exercise, ignore the `psql` "table does not exist, skipping" messages. + + View the `create_hawq_tables.sql` script. Notice the use of the `retail_demo.` prefix to the table name: + + ``` sql + CREATE TABLE retail_demo.categories_dim_hawq +( + category_id integer NOT NULL, + category_name character varying(400) NOT NULL +) +WITH (appendonly=true, compresstype=zlib) DISTRIBUTED RANDOMLY; + ``` + + Load data into the tables and verify the load succeeded. You will run a script that displays the row count of tables in the `retail_demo` schema: + + ``` shell + $ ../hawq/hawq_tables/load_hawq_tables.sh + $ cd ../hawq/hawq_tables + $ ./verify_load_hawq_tables.sh + ``` + + The output of the script should match the following: + + ``` shell + Table Name | Count + -----------------------------+------------------------ + customers_dim_hawq | 401430 + categories_dim_hawq | 56 + customer_addresses_dim_hawq | 1130639 + email_addresses_dim_hawq | 401430 + order_lineitems_hawq | 1024158 + orders_hawq | 512071 + payment_methods_hawq | 5 + products_dim_hawq | 698911 + -----------------------------+------------------------ + ``` + +8. Determine the top ten postal codes by order revenue by running the following query on the `retail_demo.orders_hawq` table: + + ``` shell + $ psql + ``` + + ``` sql + gpadmin=# select billing_address_postal_code, + sum(total_paid_amount::float8) as total, + sum(total_tax_amount::float8) as tax + from retail_demo.orders_hawq + group by billing_address_postal_code + order by total desc limit 10; + ``` + + Compare your output to the following: + + ``` shell + billing_address_postal_code | total | tax +----------------------------+-----------+----------- + 48001 | 111868.32 | 6712.0992 + 15329 | 107958.24 | 6477.4944 + 42714 | 103244.58 | 6194.6748 + 41030 | 101365.5 | 6081.93 + 50223 | 100511.64 | 6030.6984 + 03106 | 83566.41 | 0 + 57104 | 77383.63 | 3095.3452 + 23002 | 73673.66 | 3683.683 + 25703 | 68282.12 | 4096.9272 + 26178 | 66836.4 | 4010.184 +(10 rows) +Time: 2120.214 ms + ``` \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c73dab31/tutorial/gettingstarted/introhawqtbls.md ---------------------------------------------------------------------- diff --git a/tutorial/gettingstarted/introhawqtbls.md b/tutorial/gettingstarted/introhawqtbls.md deleted file mode 100644 index 041ed3e..0000000 --- a/tutorial/gettingstarted/introhawqtbls.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -title: Introduction to HAWQ Tables ---- - -## <a id="tut_introhawqtbldisc"></a>Discussion - -DISCUSS HAWQ TABLES HERE - -In this exercise, you will create a HAWQ table, add data to the table, and perform simple queries. You will also load the Retail sample data set into HAWQ tables and perform more complicated queries. - -## <a id="tut_introhawqtblprereq"></a>Prerequisites - -Ensure that you have a running HAWQ installation and have downloaded the tutorial work files, including the Retail example data set and HAWQ scripts. - -## <a id="tut_excreatehawqtblsteps"></a>Steps - -Perform the following steps to create a HAWQ database and associated tables to represent the Retail data set. - -1. In a terminal window, navigate to the tutorial work files base directory. - - ``` shell - $ cd $HAWQTUTBASE - ``` - -2. Run the `createdb` command at the shell prompt to create a database named `hawqintro_tbl1`: - - ``` shell - $ createdb hawqintro_tbl1 - ``` - -3. Run the PostgreSQL interpreter `psql` and connect to database `hawqintro_tbl1`: - - ``` shell - $ psql -d hawqintro_tbl1 - ``` - - You will enter the `psql` interpreter: - - ``` sql - hawqintro_tbl1=# - ``` - - Notice that the `psql` prompt is the database name followed by `=#`. - -4. Create the table `id_tbl1` with a single column named `id` of type `integer`: - - ``` sql - hawqintro_tbl1=# create table id_tbl1 (id int); - ``` - -5. Add some data to `id_tbl1`: - - ``` sql - hawqintro_tbl1=# insert into id_tbl1 select generate_series(1,100); - ``` - - This `insert` command adds 100 rows to `id_tbl1`, incrementing and writing the `id` for each row. - -6. Query the table: - - ``` sql - hawqintro_tbl1=# select * from id_tbl1 where id > 97; - ``` - ``` shell - ** id - ----- - 98 - 99 - 100 - (3 rows)** - ``` - - The query returns all rows in the table where the `id` is greater than 97. The query also returns the number of rows returned. - -7. Quit and restart `psql`. - - ``` sql - hawqintro_tbl1=#\q - ``` - - ``` shell - $ cd $HAWQTUTBASE/sample_data - $ psql - ``` - -8. Create a schema named `retail_demo` to represent the Retail data set: - - ``` sql - gpadmin=# CREATE SCHEMA retail_demo; - ``` - - A schema is a namespace for the database. It contains named objects like tables, data types, functions, and operators. These named objects are accessed by qualifying their name with `schemaname.` (or `retail_demo.` in this exercise) as a prefix. You may also set a search path that includes schema name(s). - -7. Use the provided scripts to create tables for, load, and verify the Retail demo data set: - - ``` sql - gpadmin=#\i ../hawq/hawq_tables/create_hawq_tables.sql - gpadmin=#\q - ``` - The `create_hawq_tables.sql` script deletes each table before creating. If this is your first time performing this exercise, ignore the `psql` "table does not exist, skipping" messages. - - View the `create_hawq_tables.sql` script. Notice the use of the `retail_demo.` prefix to the table name: - - ``` sql - CREATE TABLE retail_demo.categories_dim_hawq -( - category_id integer NOT NULL, - category_name character varying(400) NOT NULL -) -WITH (appendonly=true, compresstype=zlib) DISTRIBUTED RANDOMLY; - ``` - - Load data into the tables and verify the load succeeded. You will run a script that displays the row count of tables in the `retail_demo` schema: - - ``` shell - $ ../hawq/hawq_tables/load_hawq_tables.sh - $ cd ../hawq/hawq_tables - $ ./verify_load_hawq_tables.sh - ``` - - The output of the script should match the following: - - ``` shell - Table Name | Count - -----------------------------+------------------------ - customers_dim_hawq | 401430 - categories_dim_hawq | 56 - customer_addresses_dim_hawq | 1130639 - email_addresses_dim_hawq | 401430 - order_lineitems_hawq | 1024158 - orders_hawq | 512071 - payment_methods_hawq | 5 - products_dim_hawq | 698911 - -----------------------------+------------------------ - ``` - -8. Determine the top ten postal codes by order revenue by running the following query on the `retail_demo.orders_hawq` table: - - ``` shell - $ psql - ``` - - ``` sql - gpadmin=# select billing_address_postal_code, - sum(total_paid_amount::float8) as total, - sum(total_tax_amount::float8) as tax - from retail_demo.orders_hawq - group by billing_address_postal_code - order by total desc limit 10; - ``` - - Compare your output to the following: - - ``` shell - billing_address_postal_code | total | tax -----------------------------+-----------+----------- - 48001 | 111868.32 | 6712.0992 - 15329 | 107958.24 | 6477.4944 - 42714 | 103244.58 | 6194.6748 - 41030 | 101365.5 | 6081.93 - 50223 | 100511.64 | 6030.6984 - 03106 | 83566.41 | 0 - 57104 | 77383.63 | 3095.3452 - 23002 | 73673.66 | 3683.683 - 25703 | 68282.12 | 4096.9272 - 26178 | 66836.4 | 4010.184 -(10 rows) -Time: 2120.214 ms - ``` \ No newline at end of file
