[GitHub] incubator-hawq issue #1262: HAWQ-1493. Integrate Ranger lookup JAAS configur...

[interma]

Github user interma commented on the issue:

https://github.com/apache/incubator-hawq/pull/1262
  
I was going to use `setProperty("java.security.auth.login.config", 
conf_file)`, but it not works.

After refer some ranger code, it seems ranger do some hacks by 
org.apache.ranger.audit.utils.InMemoryJAASConfiguration, so I use the same 
class as Ranger does.

Tested on my test vagrant cluster, passed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1262: HAWQ-1493. Integrate Ranger lookup JAAS c...

[interma]

GitHub user interma opened a pull request:

https://github.com/apache/incubator-hawq/pull/1262

HAWQ-1493. Integrate Ranger lookup JAAS configuration in ranger-admin…

… plugin jar



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/interma/interma-hawq hawq-1493

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1262.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1262


commit 8c8d55d0d29fb716aed1e40b7fccf4a2a09fab7e
Author: interma 
Date:   2017-06-30T05:12:17Z

HAWQ-1493. Integrate Ranger lookup JAAS configuration in ranger-admin 
plugin jar




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (HAWQ-1493) Integrate Ranger lookup JAAS configuration in ranger-admin plugin jar

[Hongxu Ma (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069401#comment-16069401
 ] 

Hongxu Ma commented on HAWQ-1493:
-

Only do `setProperty("java.security.auth.login.config")` doesn't work.
Because Ranger do some hacks by 
org.apache.ranger.audit.utils.InMemoryJAASConfiguration, we should also use 
this class as Ranger.


> Integrate Ranger lookup JAAS configuration in ranger-admin plugin jar
> -
>
> Key: HAWQ-1493
> URL: https://issues.apache.org/jira/browse/HAWQ-1493
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: 2.3.0.0-incubating
>
>
> For support ranger lookup a kerberized HAWQ, we modify java.security file and 
> add a .java.login.config file into ranger account now.
> But both of the two files are global, have influenced other program, so we 
> need integrate the JAAS configuration in a private scope.
> After investigation, `setProperty("java.security.auth.login.config")` in 
> ranger-admin plugin code is a good solution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HAWQ-1493) Integrate Ranger lookup JAAS configuration in ranger-admin plugin jar

[Hongxu Ma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongxu Ma reassigned HAWQ-1493:
---

Assignee: Hongxu Ma  (was: Radar Lei)

> Integrate Ranger lookup JAAS configuration in ranger-admin plugin jar
> -
>
> Key: HAWQ-1493
> URL: https://issues.apache.org/jira/browse/HAWQ-1493
> Project: Apache HAWQ
>  Issue Type: Sub-task
>  Components: Security
>Reporter: Hongxu Ma
>Assignee: Hongxu Ma
> Fix For: 2.3.0.0-incubating
>
>
> For support ranger lookup a kerberized HAWQ, we modify java.security file and 
> add a .java.login.config file into ranger account now.
> But both of the two files are global, have influenced other program, so we 
> need integrate the JAAS configuration in a private scope.
> After investigation, `setProperty("java.security.auth.login.config")` in 
> ranger-admin plugin code is a good solution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HAWQ-1492) Package jdbc-plugin RPM with PXF installation

[Oleksandr Diachenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Diachenko resolved HAWQ-1492.
---
Resolution: Fixed

> Package jdbc-plugin RPM with PXF installation
> -
>
> Key: HAWQ-1492
> URL: https://issues.apache.org/jira/browse/HAWQ-1492
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: PXF
>Reporter: Lav Jain
>Assignee: Lav Jain
> Fix For: 2.3.0.0-incubating
>
>
> JDBC PXF RPM is currently not being installed when PXF is installed in an 
> ambari cluster. PXF JDBC profile needs to be enabled as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HAWQ-1492) Package jdbc-plugin RPM with PXF installation

[Oleksandr Diachenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069296#comment-16069296
 ] 

Oleksandr Diachenko commented on HAWQ-1492:
---

Pushed to master.

> Package jdbc-plugin RPM with PXF installation
> -
>
> Key: HAWQ-1492
> URL: https://issues.apache.org/jira/browse/HAWQ-1492
> Project: Apache HAWQ
>  Issue Type: Improvement
>  Components: PXF
>Reporter: Lav Jain
>Assignee: Lav Jain
> Fix For: 2.3.0.0-incubating
>
>
> JDBC PXF RPM is currently not being installed when PXF is installed in an 
> ambari cluster. PXF JDBC profile needs to be enabled as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-hawq issue #1260: Hawq-1492. Enable packaging for PXF JDBC plugin

[sansanichfb]

Github user sansanichfb commented on the issue:

https://github.com/apache/incubator-hawq/pull/1260
  
Pushed to master, please feel free to close a PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124941368
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
--- End diff --

At this stage, we're just trying to test simple demo functionality. We can 
introduce dynamic types later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124941296
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
--- End diff --

Will update.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124941281
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
+private static final Log LOG = LogFactory.getLog(DummyAccessor.class);
+private int rowNumber;
+private int fragmentNumber;
+public DummyAccessor(InputData metaData) {
+super(metaData);
+}
+@Override
+public boolean openForRead() throws Exception {
+/* fopen or similar */
+return true;
+}
+@Override
+public OneRow readNextObject() throws Exception {
+/* return next row , 

[jira] [Commented] (HAWQ-1490) PXF dummy text profile for non-hadoop testing.

[John Gaskin (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069239#comment-16069239
 ] 

John Gaskin commented on HAWQ-1490:
---

We will be introducing a demo profile which returns text and a profile for a 
custom format.

> PXF dummy text profile for non-hadoop testing.
> --
>
> Key: HAWQ-1490
> URL: https://issues.apache.org/jira/browse/HAWQ-1490
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Tests
>Reporter: John Gaskin
>Assignee: Shivram Mani
> Fix For: 2.3.0.0-incubating
>
>
> A user should be able to create a PXF ext table with Demo profile that allows 
> her to check whether PXF service is functional in a given system without 
> relying on any backend data source.
> This would serve as a profile to test PXF service/framework. The demo profile 
> could return some configured static data while invoking all the necessary 
> APIs/functions for the smoke test.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124934930
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
+ * returns the data fragments - identifiers of data and a list of 
available hosts
+ */
+@Override
+public List getFragments() throws Exception {
+String localhostname = 
java.net.InetAddress.getLocalHost().getHostName();
+String[] localHosts = new String[]{localhostname, localhostname};
+fragments.add(new Fragment(inputData.getDataSource() + ".1" /* 
source name */,
+localHosts /* available hosts list */,
+"fragment1".getBytes()));
--- End diff --

Thanks, removed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124934892
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
--- End diff --

Thanks! changed to "Static"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124924601
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
--- End diff --

Thanks, will update.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124922961
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
--- End diff --

Not sure if "Dummy" is the best way to describe it, maybe Echo or smth less 
dummy :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124923619
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
+ * returns the data fragments - identifiers of data and a list of 
available hosts
+ */
+@Override
+public List getFragments() throws Exception {
+String localhostname = 
java.net.InetAddress.getLocalHost().getHostName();
+String[] localHosts = new String[]{localhostname, localhostname};
+fragments.add(new Fragment(inputData.getDataSource() + ".1" /* 
source name */,
+localHosts /* available hosts list */,
+"fragment1".getBytes()));
--- End diff --

I personally don't find the style of including comments inside function 
call signature all that useful. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124924223
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
+private static final Log LOG = LogFactory.getLog(DummyAccessor.class);
+private int rowNumber;
+private int fragmentNumber;
+public DummyAccessor(InputData metaData) {
+super(metaData);
+}
+@Override
+public boolean openForRead() throws Exception {
+/* fopen or similar */
+return true;
+}
+@Override
+public OneRow readNextObject() throws Exception {
+/* return next row , 

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124920725
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
+/* break up the row into fields */
+List output = new LinkedList();
+String[] fields = ((String) row.getData()).split(",");
--- End diff --

row can never be null here as the ReadBridge specifically check for null. 
None of the other resolvers have this check as well. Will add validation for 
other things you mentioned


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124919185
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
--- End diff --

will remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124917078
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
+private static final Log LOG = LogFactory.getLog(DummyAccessor.class);
+private int rowNumber;
+private int fragmentNumber;
+public DummyAccessor(InputData metaData) {
+super(metaData);
+}
+@Override
+public boolean openForRead() throws Exception {
+/* fopen or similar */
--- End diff --

It's an example of what function might go there. Will update the comment to 
reflect what the function currently does.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124916656
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
--- End diff --

Thanks, will add.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124894489
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
--- End diff --

This resolver returns static schema with two columns, let's return dynamic 
schema, which comes as a BridgeApi parameters. Please refer to 
InputData#getColumn(int index) and InputData#getColumns() method to get 
requested column's metadata.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124893900
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
+ * returns the data fragments - identifiers of data and a list of 
available hosts
+ */
+@Override
+public List getFragments() throws Exception {
+String localhostname = 
java.net.InetAddress.getLocalHost().getHostName();
+String[] localHosts = new String[]{localhostname, localhostname};
+fragments.add(new Fragment(inputData.getDataSource() + ".1" /* 
source name */,
+localHosts /* available hosts list */,
+"fragment1".getBytes()));
--- End diff --

/* fragment's metadata */


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124893752
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
--- End diff --

Please follow Javadoc format of comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124893371
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
--- End diff --

According to logic, it's not really a local filesystem fragmenter, so 
please update comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124891123
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
+/* break up the row into fields */
+List output = new LinkedList();
+String[] fields = ((String) row.getData()).split(",");
--- End diff --

I would add validations here, whether a row is not null, whether it has 
data, and if it has two tokens separated by a comma.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124889896
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
--- End diff --

This will fail rat check, please add Apache license header.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

GitHub user Quikling opened a pull request:

https://github.com/apache/incubator-hawq/pull/1261

HAWQ-1490. Added new dummy plugin for offline pxf testing.

This plugin allows for pxf testing without any underlying file system or 
filestore.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Quikling/incubator-hawq HAWQ-1490

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1261.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1261


commit 3bb594dafe035f15a44578d2dc614609f5232d65
Author: John Gaskin 
Date:   2017-06-26T23:36:37Z

added dummy profile, doesn't read from local file yet

commit 9132efc007942383fe1b88d060f22131a02a4fac
Author: shivzone 
Date:   2017-06-28T21:18:04Z

HAWQ-1490. Refactored Dummy profile into pxf-api project




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #1260: Hawq 1492. Enable packaging for PXF JDBC plugin

[shivzone]

Github user shivzone commented on the issue:

https://github.com/apache/incubator-hawq/pull/1260
  
You would have to use HAWQ-1492.  format for this pull request to 
show up with the jira


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1260: Hawq 1492. Enable packaging for PXF JDBC ...

[lavjain]

GitHub user lavjain opened a pull request:

https://github.com/apache/incubator-hawq/pull/1260

Hawq 1492. Enable packaging for PXF JDBC plugin



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lavjain/incubator-hawq HAWQ-1492

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1260.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1260


commit d0bdc0d82e80da77a46609f17003a53c5c802f00
Author: lavjain 
Date:   2017-06-27T21:28:21Z

HAWQ-1492. Enable packaging for PXF JDBC plugin

commit 9b60c5ef51941e9b64b91c0a2adba8186423f1fa
Author: lavjain 
Date:   2017-06-29T17:30:50Z

HAWQ-1492. Enable packaging for PXF JDBC plugin




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---