[GitHub] incubator-hawq pull request #1321: HAWQ-1565. Include Pluggable Storage Form...

[chiyang10000]

Github user chiyang1 closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1321


---

[GitHub] incubator-hawq pull request #1321: HAWQ-1565. Include Pluggable Storage Form...

[chiyang10000]

Github user chiyang1 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1321#discussion_r157335240
  
--- Diff: src/backend/executor/nodeExternalscan.c ---
@@ -80,8 +83,61 @@ ExternalNext(ExternalScanState *node)
/*
 * get the next tuple from the file access methods
 */
-   externalSelectDesc = external_getnext_init(&(node->ss.ps), node);
-   tuple = external_getnext(scandesc, direction, externalSelectDesc);
+   if (scandesc->fs_formatter_type == ExternalTableType_Invalid)
+   {
+   elog(ERROR, "invalid formatter type for external table: %s", 
__func__);
+   }
+   else if (scandesc->fs_formatter_type != ExternalTableType_PLUG)
+   {
+   externalSelectDesc = external_getnext_init(&(node->ss.ps), 
node);
+
+   returnTuple = external_getnext(scandesc, direction, 
externalSelectDesc,
+  &(node->ss), slot);
+   }
+   else
+   {
+   Assert(scandesc->fs_formatter_name);
+
+   FmgrInfo *getnextInitFunc = 
scandesc->fs_ps_scan_funcs.getnext_init;
+
+   if (getnextInitFunc)
+   {
+   /*
+* pg_strncasecmp(scandesc->fs_formatter_name, "orc", 
strlen("orc"))
+* Performance improvement for string comparison.
+*/
+   const char *formatter_name = "orc";
+   if (*(int *)(scandesc->fs_formatter_name) != *(int 
*)formatter_name)
--- End diff --

Thanks for your advise. I will update this PR with a simple demo in the 
near days.


---

[GitHub] incubator-hawq pull request #1321: HAWQ-1565. Include Pluggable Storage Form...

[interma]

Github user interma commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1321#discussion_r157174345
  
--- Diff: src/backend/executor/nodeExternalscan.c ---
@@ -80,8 +83,61 @@ ExternalNext(ExternalScanState *node)
/*
 * get the next tuple from the file access methods
 */
-   externalSelectDesc = external_getnext_init(&(node->ss.ps), node);
-   tuple = external_getnext(scandesc, direction, externalSelectDesc);
+   if (scandesc->fs_formatter_type == ExternalTableType_Invalid)
+   {
+   elog(ERROR, "invalid formatter type for external table: %s", 
__func__);
+   }
+   else if (scandesc->fs_formatter_type != ExternalTableType_PLUG)
+   {
+   externalSelectDesc = external_getnext_init(&(node->ss.ps), 
node);
+
+   returnTuple = external_getnext(scandesc, direction, 
externalSelectDesc,
+  &(node->ss), slot);
+   }
+   else
+   {
+   Assert(scandesc->fs_formatter_name);
+
+   FmgrInfo *getnextInitFunc = 
scandesc->fs_ps_scan_funcs.getnext_init;
+
+   if (getnextInitFunc)
+   {
+   /*
+* pg_strncasecmp(scandesc->fs_formatter_name, "orc", 
strlen("orc"))
+* Performance improvement for string comparison.
+*/
+   const char *formatter_name = "orc";
+   if (*(int *)(scandesc->fs_formatter_name) != *(int 
*)formatter_name)
--- End diff --

I think pg_strncasecmp() or other some string compare function is easier to 
understand.
Just a advise, do not change is also ok.


---

[GitHub] incubator-hawq pull request #1321: HAWQ-1565. Include Pluggable Storage Form...

[chiyang10000]

GitHub user chiyang1 opened a pull request:

https://github.com/apache/incubator-hawq/pull/1321

HAWQ-1565. Include Pluggable Storage Format Framework in External Table Scan

Rewrite the tuple construct and consume working flow in the external table, 
which leads to data copy cost.

@huor 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chiyang1/incubator-hawq HAWQ-1565

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1321.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1321


commit 223c8cb4738724b3c42d316a987ab506ca6eda08
Author: Chiyang Wan 
Date:   2017-12-07T15:05:40Z

HAWQ-1565. Include Pluggable Storage Format Framework in External Table Scan

Rewrite the tuple construct and consume working flow in the external table, 
which leads to data copy cost.




---