[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[zhangjackey]

Github user zhangjackey closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1351


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[zhangjackey]

Github user zhangjackey commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r181942632
  
--- Diff: contrib/vexecutor/vexecutor.c ---
@@ -163,24 +165,35 @@ static PlanState* VExecInitNode(PlanState 
*node,EState *eState,int eflags,Memory
case T_AppendOnlyScan:
case T_ParquetScan:
case T_TableScanState:
-   START_MEMORY_ACCOUNT(curMemoryAccount);
+   //START_MEMORY_ACCOUNT(curMemoryAccount);
--- End diff --

if the plan->memoryAccount is not NULL, we use it directly, otherwise, we 
do not enable it.


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[zhangjackey]

Github user zhangjackey commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r181942311
  
--- Diff: contrib/vexecutor/execVQual.c ---
@@ -123,3 +123,431 @@ VirtualNodeProc(ScanState* state,TupleTableSlot 
*slot){
 ExecStoreVirtualTuple(slot);
 return true;
 }
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalScalarVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+   TupleBatch tb;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   /* isNull is a single value, it can not be used when data is vectorized 
*/
+   *isNull = false;
+
+   /* Fetch the value from the slot */
+   tb = (TupleBatch )slot->PRIVATE_tb;
+
+   Assert(NULL != tb);
+
+   return PointerGetDatum(tb->datagroup[attnum]);
--- End diff --

yes, change it to attnum - 1.


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[zhangjackey]

Github user zhangjackey commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r181933851
  
--- Diff: contrib/vexecutor/execVQual.c ---
@@ -123,3 +123,431 @@ VirtualNodeProc(ScanState* state,TupleTableSlot 
*slot){
 ExecStoreVirtualTuple(slot);
 return true;
 }
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalScalarVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+   TupleBatch tb;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   /* isNull is a single value, it can not be used when data is vectorized 
*/
+   *isNull = false;
+
--- End diff --

add Assert in next commit.


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[zhangjackey]

Github user zhangjackey commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r181933267
  
--- Diff: contrib/vexecutor/execVQual.c ---
@@ -123,3 +123,431 @@ VirtualNodeProc(ScanState* state,TupleTableSlot 
*slot){
 ExecStoreVirtualTuple(slot);
 return true;
 }
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalScalarVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+   TupleBatch tb;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   /* isNull is a single value, it can not be used when data is vectorized 
*/
+   *isNull = false;
+
+   /* Fetch the value from the slot */
+   tb = (TupleBatch )slot->PRIVATE_tb;
+
+   Assert(NULL != tb);
+
+   return PointerGetDatum(tb->datagroup[attnum]);
+}
+
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   if (attnum != InvalidAttrNumber)
+   {
+   TupleBatch tb;
+   /*
+* Scalar variable case.
+*
+* If it's a user attribute, check validity (bogus system 
attnums will
+* be caught inside slot_getattr).  What we have to check for 
here
+* is the possibility of an attribute having been changed in 
type
+* since the plan tree was created.  Ideally the plan would get
+* invalidated and not re-used, but until that day arrives, we 
need
+* defenses.  Fortunately it's sufficient to check once on the 
first
+* time through.
+

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[wengyanqing]

Github user wengyanqing commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r180627154
  
--- Diff: contrib/vexecutor/execVQual.c ---
@@ -123,3 +123,431 @@ VirtualNodeProc(ScanState* state,TupleTableSlot 
*slot){
 ExecStoreVirtualTuple(slot);
 return true;
 }
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalScalarVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+   TupleBatch tb;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   /* isNull is a single value, it can not be used when data is vectorized 
*/
+   *isNull = false;
+
--- End diff --

It's better to check or assert to make sure slot is not null here.


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[wengyanqing]

Github user wengyanqing commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r180626694
  
--- Diff: contrib/vexecutor/execVQual.c ---
@@ -123,3 +123,431 @@ VirtualNodeProc(ScanState* state,TupleTableSlot 
*slot){
 ExecStoreVirtualTuple(slot);
 return true;
 }
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalScalarVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+   TupleBatch tb;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   /* isNull is a single value, it can not be used when data is vectorized 
*/
+   *isNull = false;
+
+   /* Fetch the value from the slot */
+   tb = (TupleBatch )slot->PRIVATE_tb;
+
+   Assert(NULL != tb);
+
+   return PointerGetDatum(tb->datagroup[attnum]);
+}
+
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   if (attnum != InvalidAttrNumber)
+   {
+   TupleBatch tb;
+   /*
+* Scalar variable case.
+*
+* If it's a user attribute, check validity (bogus system 
attnums will
+* be caught inside slot_getattr).  What we have to check for 
here
+* is the possibility of an attribute having been changed in 
type
+* since the plan tree was created.  Ideally the plan would get
+* invalidated and not re-used, but until that day arrives, we 
need
+* defenses.  Fortunately it's sufficient to check once on the 
first
+* time through.
+

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[wengyanqing]

Github user wengyanqing commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r180626789
  
--- Diff: contrib/vexecutor/vexecutor.c ---
@@ -163,24 +165,35 @@ static PlanState* VExecInitNode(PlanState 
*node,EState *eState,int eflags,Memory
case T_AppendOnlyScan:
case T_ParquetScan:
case T_TableScanState:
-   START_MEMORY_ACCOUNT(curMemoryAccount);
+   //START_MEMORY_ACCOUNT(curMemoryAccount);
--- End diff --

Any reason that comment this code line ?


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[wengyanqing]

Github user wengyanqing commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1351#discussion_r180625706
  
--- Diff: contrib/vexecutor/execVQual.c ---
@@ -123,3 +123,431 @@ VirtualNodeProc(ScanState* state,TupleTableSlot 
*slot){
 ExecStoreVirtualTuple(slot);
 return true;
 }
+
+/*
+ * get values from vectorized tuple slot
+ * copy from src/backend/executor/execQual.c
+ */
+static Datum
+VExecEvalScalarVar(ExprState *exprstate, ExprContext *econtext,
+   bool *isNull, ExprDoneCond *isDone)
+{
+   Var*variable = (Var *) exprstate->expr;
+   TupleTableSlot *slot;
+   AttrNumber  attnum;
+   TupleBatch tb;
+
+   if (isDone)
+   *isDone = ExprSingleResult;
+
+   Assert(econtext->ecxt_scantuple != NULL || econtext->ecxt_innertuple != 
NULL || econtext->ecxt_outertuple != NULL);
+   /*
+* Get the input slot and attribute number we want
+*
+* The asserts check that references to system attributes only appear at
+* the level of a relation scan; at higher levels, system attributes 
must
+* be treated as ordinary variables (since we no longer have access to 
the
+* original tuple).
+*/
+   attnum = variable->varattno;
+
+   switch (variable->varno)
+   {
+   case INNER: /* get the tuple from 
the inner node */
+   slot = econtext->ecxt_innertuple;
+   Assert(attnum > 0);
+   break;
+
+   case OUTER: /* get the tuple from 
the outer node */
+   slot = econtext->ecxt_outertuple;
+   Assert(attnum > 0);
+   break;
+
+   default:/* get the tuple from 
the relation being
+* scanned */
+   slot = econtext->ecxt_scantuple;
+   break;
+   }
+
+   /* isNull is a single value, it can not be used when data is vectorized 
*/
+   *isNull = false;
+
+   /* Fetch the value from the slot */
+   tb = (TupleBatch )slot->PRIVATE_tb;
+
+   Assert(NULL != tb);
+
+   return PointerGetDatum(tb->datagroup[attnum]);
--- End diff --

attnum's value starts with 1, it should be attnum -1 as the index of 
tb->datagroup ?


---

[GitHub] incubator-hawq pull request #1351: HAWQ-1603.add new hook API

[zhangjackey]

GitHub user zhangjackey opened a pull request:

https://github.com/apache/incubator-hawq/pull/1351

HAWQ-1603.add new hook API

add two hook API.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhangjackey/incubator-hawq v_expr

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1351.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1351


commit 6a44fd37be021c083004619e74834078bf967c0c
Author: Shujie Zhang 
Date:   2018-04-04T08:10:07Z

add new hook API




---