subject:"\[GitHub\] incubator\-hawq\-docs pull request #46\: HAWQ\-1119 \- create doc content for PXF..."

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-18 Thread lisakowen

Github user lisakowen closed the pull request at:

https://github.com/apache/incubator-hawq-docs/pull/46


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-18 Thread lisakowen

Github user lisakowen commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96705982
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -396,7 +394,7 @@ The field type of `recordkey` must correspond to the 
key type, much as the other
 
 ### Example: Using Record Keys
 
-Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (We did not identify a record key when creating 
the writable table.
+Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (You did not identify a record key when you 
created the writable table.
--- End diff --

those two sentences are not directly related.  the type of recordkey will 
always be int8.  i am going to just remove the "You did not identify a record 
key..." sentence as a variant is used after the example.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-18 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96702964
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -413,4 +411,4 @@ gpadmin=# SELECT * FROM read_pxf_tbl_seqwrit_RECKEY;
 (2 rows)
 ```
 
-The `recordkey` is displayed as `0`; a record key was not identified when 
you inserted entries in the writable table.
+The `recordkey` is displayed as `0`; you did not identify a record key 
when you inserted entries in the writable table.
--- End diff --

Edit:  The `recordkey` value is `0` because you did not identify a record 
key when you inserted entries into the writable table.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-18 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96702640
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -136,7 +136,7 @@ Perform the following operations to use the PXF 
`HdfsTextSimple` profile to crea
FORMAT 'CSV';
 ```
 
-The table is created with the `'CSV'` `FORMAT` because the delimiter 
character used when creating the writable table was a comma `,`.
+The table is created with the `'CSV'` `FORMAT` because the delimiter 
character with which you created the writable table was a comma `,`.
--- End diff --

This sounds a little awkward.  Edit to:

"Specify the the `'CSV'` `FORMAT` for the readable table, because you 
created the writable table with a comma ',' as the delimiter character.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-18 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96702781
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -396,7 +394,7 @@ The field type of `recordkey` must correspond to the 
key type, much as the other
 
 ### Example: Using Record Keys
 
-Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (We did not identify a record key when creating 
the writable table.
+Create an external readable table to access the record keys from the 
writable table `pxf_tbl_seqwrit` that you created in [Example: Writing Data 
Using the SequenceWritable Profile](#profile_hdfsseqwritable_writing).  The 
`recordkey` is of type `int8`. (You did not identify a record key when you 
created the writable table.
--- End diff --

Edit last sentence:  The `recordkey` is of type `int8` because you did not 
identify a record key when you created the writable table.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-13 Thread kavinderd

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96090205
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF classpath. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-13 Thread kavinderd

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96086612
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF classpath. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
--- End diff --

I

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2017-01-13 Thread kavinderd

Github user kavinderd commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r96086235
  
--- Diff: markdown/pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
--- End diff --

Is the formatting of this correct for 
`org.apache.hadoop.io.compress.GzipCode`? It seems like it should be a part of 
the row above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-02 Thread lisakowen

Github user lisakowen commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86175112
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86050201
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86051029
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86049842
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86052740
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86050016
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86049010
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,416 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to binary or text HDFS files.
+
+Use the `HdfsTextSimple` profile when writing text data. Use the 
`SequenceWritable` profile when dealing with binary data.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: Tables that you create with writable profiles can only be used 
for INSERT operations.  If you want to query inserted data, you must define a 
separate external readable table that references the new HDFS file using the 
equivalent readable profile.  ??You can also create a Hive table to access the 
HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services.
+-   HDFS write permissions are provided to a restricted set of users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports two writable profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE WRITABLE EXTERNAL TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
to create a plain-text-delimited file at the location specified by 
\. The `HdfsTextSimple` '`TEXT`' `FORMAT` supports only the 
built-in `(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with the `HdfsTextSimple` profile to 
create a comma-separated-value file at the location specified by 
\.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
custom options:
+
+| Option  | Value Description | Profile |
+|---|-||
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec` and 
`org.apache.hadoop.io.compress.BZip2Codec` | HdfsTextSimple, SequenceWritable |
+||  `org.apache.hadoop.io.compress.GzipCodec` | HdfsTextSimple |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. | HdfsTextSimple, SequenceWritable |
+| DATA-SCHEMA| The name of the writer serialization/deserialization 
class. The jar file in which this class resides must be in the PXF class path. 
This option is required for the `SequenceWritable` profile and has no default 
value. | SequenceWritable|
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-threaded mode. The default value is `TRUE`. Set this option to `FALSE` to 
handle all requests in a single thread for operations that are not thread-safe 
(for example, compression). | HdfsTextSimple, SequenceWritable| 
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86042974
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
--- End diff --

Thanks - that sounds good to me.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-11-01 Thread lisakowen

Github user lisakowen commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r86042160
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
--- End diff --

profile-options is a better name, i agree.  all of the plug-in pages are 
currently using custom-options.  i will make the change to profile-options for 
all plug-ins in a separate PR after this is accepted.  sound ok?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85814629
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
+
+- org.apache.hadoop.io.compress.DefaultCodec
+- org.apache.hadoop.io.compress.GzipCodec
+- org.apache.hadoop.io.compress.BZip2Codec
+

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85794041
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
--- End diff --

Change "Keyword" to "Option"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85791234
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
--- End diff --

Break the last requirement into a separate bullet:  Write permissions 
should be provided to a restricted set of user.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85813059
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
--- End diff --

Small edit:  Writable tables created using the HdfsTextSimple profile can 
optionally use `record` or `block` compression. The following

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85813812
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
+
+- org.apache.hadoop.io.compress.DefaultCodec
+- org.apache.hadoop.io.compress.GzipCodec
+- org.apache.hadoop.io.compress.BZip2Codec
+

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85790837
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
--- End diff --

This note could use some editing.  Something like:  Tables that you create 
with the SequenceWritable profile can only be used for INSERT operations. If 
you want to query the inserted data, you must define a separate external table 
that references the new HDFS file, using the  or ??? profiles.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85845526
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
+
+- org.apache.hadoop.io.compress.DefaultCodec
+- org.apache.hadoop.io.compress.GzipCodec
+- org.apache.hadoop.io.compress.BZip2Codec
+

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85789703
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
--- End diff --

Seems like this last sentence should say something other than "to HDFS", 
since internal tables are already stored in HDFS but in HAWQ proprietary 
format.  Not sure what the best wording is - HDFS text files/flat files? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85793349
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
--- End diff --

Change the wording of this row and the next to something like:  Use 
'`TEXT`' `FORMAT` with the `HdfsTextSimple` profile to create a 
plain-text-delimited file at the specified \. 

(same for CSV)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85812589
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
--- End diff --

multi-thread -> multi-threaded.  Also, the rest  some edits:

The default value is true. Set this option to `FALSE` to handle all 
requests in a single thread for operations that are not thread-safe (for 
example, compression).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85845000
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
+
+- org.apache.hadoop.io.compress.DefaultCodec
+- org.apache.hadoop.io.compress.GzipCodec
+- org.apache.hadoop.io.compress.BZip2Codec
+

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85814125
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
+
+- org.apache.hadoop.io.compress.DefaultCodec
+- org.apache.hadoop.io.compress.GzipCodec
+- org.apache.hadoop.io.compress.BZip2Codec
+

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85794241
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
--- End diff --

Change  to " values"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85792675
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
--- End diff --

Maybe change this to  ?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85792301
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
--- End diff --

writable two -> two writable

Also, seems like there should be some mention of the difference between 
these profiles by now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85845296
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
+| THREAD-SAFE | Boolean value determining if a table query can run in 
multi-thread mode. Default value is `TRUE`, requests run in multi-threaded 
mode. When set to `FALSE`, requests will be handled in a single thread.  
`THREAD-SAFE` should be set appropriately when operations that are not 
thread-safe are performed (i.e. compression). |
+
+## HdfsTextSimple Profile
+
+Use the `HdfsTextSimple` profile when writing delimited data to a plain 
text file where each row is a single record.
+
+Writable tables created using the `HdfsTextSimple` profile can use no, 
record, or block compression. When compression is used, the default, gzip, and 
bzip2 Hadoop compression codecs are supported:
+
+- org.apache.hadoop.io.compress.DefaultCodec
+- org.apache.hadoop.io.compress.GzipCodec
+- org.apache.hadoop.io.compress.BZip2Codec
+

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85791950
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
--- End diff --

That syntax is unfortunate.  GPDB uses CREATE WRITABLE EXTERNAL instead of 
CREATE EXTERNAL WRITABLE :(


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85794550
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
+| COMPRESSION_TYPE| The compression type to employ; supported values 
are `RECORD` (the default) or `BLOCK`. |
+| DATA-SCHEMA| (`SequenceWritable` profile only) The name of the 
writer serialization/deserialization class. The jar file in which this class 
resides must be in the PXF class path. This option has no default value. |
--- End diff --

Is DATA-SCHEMA an option, or is it required?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-31 Thread dyozie

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/46#discussion_r85793887
  
--- Diff: pxf/HDFSWritablePXF.html.md.erb ---
@@ -0,0 +1,410 @@
+---
+title: Writing Data to HDFS
+---
+
+The PXF HDFS plug-in supports writable external tables using the 
`HdfsTextSimple` and `SequenceWritable` profiles.  You might create a writable 
table to export data from a HAWQ internal table to HDFS.
+
+This section describes how to use these PXF profiles to create writable 
external tables.
+
+**Note**: You cannot directly query data in a HAWQ writable table.  After 
creating the external writable table, you must create a HAWQ readable external 
table accessing the HDFS file, then query that table. ??You can also create a 
Hive table to access the HDFS file.??
+
+## Prerequisites
+
+Before working with HDFS file data using HAWQ and PXF, ensure that:
+
+-   The HDFS plug-in is installed on all cluster nodes. See [Installing 
PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+-   All HDFS users have read permissions to HDFS services and that write 
permissions have been restricted to specific users.
+
+## Writing to PXF External Tables
+The PXF HDFS plug-in supports writable two profiles: `HdfsTextSimple` and 
`SequenceWritable`.
+
+Use the following syntax to create a HAWQ external writable table 
representing HDFS data:Â 
+
+``` sql
+CREATE EXTERNAL WRITABLE TABLE  
+(   [, ...] | LIKE  )
+LOCATION ('pxf://[:]/
+
?PROFILE=HdfsTextSimple|SequenceWritable[&=[...]]')
+FORMAT '[TEXT|CSV|CUSTOM]' ();
+```
+
+HDFS-plug-in-specific keywords and values used in the [CREATE EXTERNAL 
TABLE](../reference/sql/CREATE-EXTERNAL-TABLE.html) call are described in the 
table below.
+
+| Keyword  | Value |
+|---|-|
+| \[:\]| The HDFS NameNode and port. |
+| \| The path to the file in the HDFS data store. |
+| PROFILE| The `PROFILE` keyword must specify one of the values 
`HdfsTextSimple` or `SequenceWritable`. |
+| \  | \ is profile-specific. These 
options are discussed in the next topic.|
+| FORMAT 'TEXT' | Use '`TEXT`' `FORMAT` with the `HdfsTextSimple` profile 
when \ will reference a plain text delimited file. The 
`HdfsTextSimple` '`TEXT`' `FORMAT` supports only the built-in 
`(delimiter=)` \. |
+| FORMAT 'CSV' | Use '`CSV`' `FORMAT` with `HdfsTextSimple` when 
\ will reference a comma-separated value file.  |
+| FORMAT 'CUSTOM' | Use the `'CUSTOM'` `FORMAT` with the 
`SequenceWritable` profile. The `SequenceWritable` '`CUSTOM`' `FORMAT` supports 
only the built-in `(formatter='pxfwritable_export)` (write) and 
`(formatter='pxfwritable_import)` (read) \.
+
+**Note**: When creating PXF external tables, you cannot use the `HEADER` 
option in your `FORMAT` specification.
+
+## Custom Options
+
+The `HdfsTextSimple` and `SequenceWritable` profiles support the following 
\:
+
+| Keyword  | Value Description |
+|---|-|
+| COMPRESSION_CODEC| The compression codec Java class name. If this 
option is not provided, no data compression is performed. Supported compression 
codecs include: `org.apache.hadoop.io.compress.DefaultCodec`, 
`org.apache.hadoop.io.compress.BZip2Codec`, and 
`org.apache.hadoop.io.compress.GzipCodec` (`HdfsTextSimple` profile only) |
--- End diff --

Instead of including parentheticals here (`HdfsTextSimple` profile only), 
add a third column to indicate which profile(s) the option applies to.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #46: HAWQ-1119 - create doc content for PXF...

2016-10-28 Thread lisakowen

GitHub user lisakowen opened a pull request:

https://github.com/apache/incubator-hawq-docs/pull/46

HAWQ-1119 - create doc content for PXF writable profiles

created a new section for PXF writable profiles (HDFS plug in 
HdfsTextSimple and SequenceWritable).  included examples and discussions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lisakowen/incubator-hawq-docs 
feature/pxfhdfs-writable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq-docs/pull/46.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #46


commit f2304ce06b0529177efee1912c6c3c3b9aaf5b1f
Author: Lisa Owen 
Date:   2016-10-25T20:14:42Z

add file for PXF HDFS writable profile topic

commit 80dc3dbe33397ef872265afe50551957cd773bef
Author: Lisa Owen 
Date:   2016-10-26T19:32:04Z

adding more content

commit fddb5b8817fdf3a800eb7099d9417bd05735abab
Author: Lisa Owen 
Date:   2016-10-28T21:58:01Z

flesh out sequencewritable profile section

commit a88c167ead7135a6f43d11cb6c4921fe680e60b9
Author: Lisa Owen 
Date:   2016-10-28T22:00:17Z

change section title

commit a3cbdcf804d936c512f99e1b15905328f4f835f1
Author: Lisa Owen 
Date:   2016-10-28T22:13:54Z

add link to writing to HDFS in pxf overview page




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

37 matches

Mail list logo