[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[kdunn926]

Github user kdunn926 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r127317467
  
--- Diff: 
pxf/pxf-api/src/test/java/org/apache/hawq/pxf/api/DemoAccessorTest.java ---
@@ -0,0 +1,79 @@
+package org.apache.hawq.pxf.api;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+
+
+
+import org.apache.hawq.pxf.api.examples.DemoAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.Mock;
+import org.powermock.core.classloader.annotations.PrepareForTest;
+import org.powermock.modules.junit4.PowerMockRunner;
+
+import static org.junit.Assert.*;
--- End diff --

IMO it would be better to explicitly identify each import, rather than a 
glob import.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[shivzone]

Github user shivzone commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r126821382
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DemoResolver.java ---
@@ -0,0 +1,59 @@
+package org.apache.hawq.pxf.api.examples;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+/**
+ * Class that defines the deserializtion of one record brought from the 
external input data.
+ *
+ * Dummy implementation
+ */
+public class DemoResolver extends Plugin implements ReadResolver {
+/**
+ * Constructs the DemoResolver
+ *
+ * @param metaData
+ */
+public DemoResolver(InputData metaData) {
+super(metaData);
+}
+
+@Override
+public List getFields(OneRow row) throws Exception {
+List output = new LinkedList();
+Object data = row.getData();
+
+/* break up the row into fields */
+String[] fields = ((String) data).split(",");
--- End diff --

Yes, it has now changed to make it flexible


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[shivzone]

Github user shivzone commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r126821163
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DemoTextResolver.java
 ---
@@ -0,0 +1,55 @@
+package org.apache.hawq.pxf.api.examples;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+
+import java.util.LinkedList;
+import java.util.List;
+
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+/**
+ * Class that defines the deserializtion of one record brought from the 
external input data.
+ *
+ * Dummy implementation
+ */
+public class DemoTextResolver extends Plugin implements ReadResolver {
+/**
+ * Constructs the DemoResolver
+ *
+ * @param metaData
+ */
+public DemoTextResolver(InputData metaData) {
+super(metaData);
+}
+@Override
+public List getFields(OneRow row) throws Exception {
+List output = new LinkedList();
+Object data = row.getData();
+output.add(new OneField(VARCHAR.getOID(), data));
--- End diff --

Text resolver only sends data as one field.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r125977351
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DemoTextResolver.java
 ---
@@ -0,0 +1,55 @@
+package org.apache.hawq.pxf.api.examples;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+
+import java.util.LinkedList;
+import java.util.List;
+
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+/**
+ * Class that defines the deserializtion of one record brought from the 
external input data.
+ *
+ * Dummy implementation
+ */
+public class DemoTextResolver extends Plugin implements ReadResolver {
+/**
+ * Constructs the DemoResolver
+ *
+ * @param metaData
+ */
+public DemoTextResolver(InputData metaData) {
+super(metaData);
+}
+@Override
+public List getFields(OneRow row) throws Exception {
+List output = new LinkedList();
+Object data = row.getData();
+output.add(new OneField(VARCHAR.getOID(), data));
--- End diff --

Instead of adding one field we can just call inputData.getColumns() and add 
that many fields to output how many was requested. In this way, it will be 
dynamic and work with any table definition.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r125976886
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DemoResolver.java ---
@@ -0,0 +1,59 @@
+package org.apache.hawq.pxf.api.examples;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+/**
+ * Class that defines the deserializtion of one record brought from the 
external input data.
+ *
+ * Dummy implementation
+ */
+public class DemoResolver extends Plugin implements ReadResolver {
+/**
+ * Constructs the DemoResolver
+ *
+ * @param metaData
+ */
+public DemoResolver(InputData metaData) {
+super(metaData);
+}
+
+@Override
+public List getFields(OneRow row) throws Exception {
+List output = new LinkedList();
+Object data = row.getData();
+
+/* break up the row into fields */
+String[] fields = ((String) data).split(",");
--- End diff --

Why are we not handling a case when data doesn't have 3 tokens? It would be 
useful for a demo profile to show how to gracefully handle errors for real 
profile.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124941368
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
--- End diff --

At this stage, we're just trying to test simple demo functionality. We can 
introduce dynamic types later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124941296
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
--- End diff --

Will update.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124941281
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
+private static final Log LOG = LogFactory.getLog(DummyAccessor.class);
+private int rowNumber;
+private int fragmentNumber;
+public DummyAccessor(InputData metaData) {
+super(metaData);
+}
+@Override
+public boolean openForRead() throws Exception {
+/* fopen or similar */
+return true;
+}
+@Override
+public OneRow readNextObject() throws Exception {
+/* return next row , 

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124934930
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
+ * returns the data fragments - identifiers of data and a list of 
available hosts
+ */
+@Override
+public List getFragments() throws Exception {
+String localhostname = 
java.net.InetAddress.getLocalHost().getHostName();
+String[] localHosts = new String[]{localhostname, localhostname};
+fragments.add(new Fragment(inputData.getDataSource() + ".1" /* 
source name */,
+localHosts /* available hosts list */,
+"fragment1".getBytes()));
--- End diff --

Thanks, removed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124934892
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
--- End diff --

Thanks! changed to "Static"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124924601
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
--- End diff --

Thanks, will update.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124922961
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
--- End diff --

Not sure if "Dummy" is the best way to describe it, maybe Echo or smth less 
dummy :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124923619
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
+ * returns the data fragments - identifiers of data and a list of 
available hosts
+ */
+@Override
+public List getFragments() throws Exception {
+String localhostname = 
java.net.InetAddress.getLocalHost().getHostName();
+String[] localHosts = new String[]{localhostname, localhostname};
+fragments.add(new Fragment(inputData.getDataSource() + ".1" /* 
source name */,
+localHosts /* available hosts list */,
+"fragment1".getBytes()));
--- End diff --

I personally don't find the style of including comments inside function 
call signature all that useful. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124924223
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
+private static final Log LOG = LogFactory.getLog(DummyAccessor.class);
+private int rowNumber;
+private int fragmentNumber;
+public DummyAccessor(InputData metaData) {
+super(metaData);
+}
+@Override
+public boolean openForRead() throws Exception {
+/* fopen or similar */
+return true;
+}
+@Override
+public OneRow readNextObject() throws Exception {
+/* return next row , 

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124920725
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
+/* break up the row into fields */
+List output = new LinkedList();
+String[] fields = ((String) row.getData()).split(",");
--- End diff --

row can never be null here as the ReadBridge specifically check for null. 
None of the other resolvers have this check as well. Will add validation for 
other things you mentioned


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124919185
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
--- End diff --

will remove


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124917078
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadAccessor;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+public class DummyAccessor extends Plugin implements ReadAccessor {
+private static final Log LOG = LogFactory.getLog(DummyAccessor.class);
+private int rowNumber;
+private int fragmentNumber;
+public DummyAccessor(InputData metaData) {
+super(metaData);
+}
+@Override
+public boolean openForRead() throws Exception {
+/* fopen or similar */
--- End diff --

It's an example of what function might go there. Will update the comment to 
reflect what the function currently does.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

Github user Quikling commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124916656
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
--- End diff --

Thanks, will add.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124894489
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
--- End diff --

This resolver returns static schema with two columns, let's return dynamic 
schema, which comes as a BridgeApi parameters. Please refer to 
InputData#getColumn(int index) and InputData#getColumns() method to get 
requested column's metadata.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124893900
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
+ * returns the data fragments - identifiers of data and a list of 
available hosts
+ */
+@Override
+public List getFragments() throws Exception {
+String localhostname = 
java.net.InetAddress.getLocalHost().getHostName();
+String[] localHosts = new String[]{localhostname, localhostname};
+fragments.add(new Fragment(inputData.getDataSource() + ".1" /* 
source name */,
+localHosts /* available hosts list */,
+"fragment1".getBytes()));
--- End diff --

/* fragment's metadata */


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124893752
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
+ *
+ * Given an text file on local filesystem(a file, directory, or wild card 
pattern),divide
+ * the data into fragments and return a list of them along with a list of
+ * host:port locations for each.
+ */
+public class DummyFragmenter extends Fragmenter{
+public DummyFragmenter(InputData metaData) {
+super(metaData);
+}
+/*
+ * path is a data source URI that can appear as a file name, a 
directory name or a wildcard
--- End diff --

Please follow Javadoc format of comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124893371
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyFragmenter.java 
---
@@ -0,0 +1,39 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import java.util.List;
+
+/**
+ * Fragmenter class for text data on local filesystem.
--- End diff --

According to logic, it's not really a local filesystem fragmenter, so 
please update comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124891123
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyResolver.java 
---
@@ -0,0 +1,29 @@
+package org.apache.hawq.pxf.api.examples;
+
+import org.apache.hawq.pxf.api.OneField;
+import org.apache.hawq.pxf.api.OneRow;
+import org.apache.hawq.pxf.api.ReadResolver;
+import org.apache.hawq.pxf.api.utilities.InputData;
+import org.apache.hawq.pxf.api.utilities.Plugin;
+import java.util.LinkedList;
+import java.util.List;
+import static org.apache.hawq.pxf.api.io.DataType.INTEGER;
+import static org.apache.hawq.pxf.api.io.DataType.VARCHAR;
+
+public class DummyResolver extends Plugin implements ReadResolver {
+private int rowNumber;
+public DummyResolver(InputData metaData) {
+super(metaData);
+rowNumber = 0;
+}
+@Override
+public List getFields(OneRow row) throws Exception {
+/* break up the row into fields */
+List output = new LinkedList();
+String[] fields = ((String) row.getData()).split(",");
--- End diff --

I would add validations here, whether a row is not null, whether it has 
data, and if it has two tokens separated by a comma.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1261#discussion_r124889896
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/examples/DummyAccessor.java 
---
@@ -0,0 +1,46 @@
+package org.apache.hawq.pxf.api.examples;
--- End diff --

This will fail rat check, please add Apache license header.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #1261: HAWQ-1490. Added new dummy plugin for off...

[Quikling]

GitHub user Quikling opened a pull request:

https://github.com/apache/incubator-hawq/pull/1261

HAWQ-1490. Added new dummy plugin for offline pxf testing.

This plugin allows for pxf testing without any underlying file system or 
filestore.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Quikling/incubator-hawq HAWQ-1490

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1261.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1261


commit 3bb594dafe035f15a44578d2dc614609f5232d65
Author: John Gaskin 
Date:   2017-06-26T23:36:37Z

added dummy profile, doesn't read from local file yet

commit 9132efc007942383fe1b88d060f22131a02a4fac
Author: shivzone 
Date:   2017-06-28T21:18:04Z

HAWQ-1490. Refactored Dummy profile into pxf-api project




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---