[GitHub] incubator-hawq-docs pull request #94: HAWQ-1304 - multiple doc changes for P...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/94#discussion_r100136630 --- Diff: markdown/pxf/ReadWritePXF.html.md.erb --- @@ -131,6 +149,8 @@ Note: The DELIMITER parameter is mandatory. +**Notes**: Metadata identifies the Java class that provides field definitions in the relation. OutputFormat identifies the file format for which a specific profile is optimized. While the built-in `Hive*` profiles provide Metadata and OutputFormat classes, most profiles will have no need to implement or specify these classes. --- End diff -- @lisakowen you are right. outputFormat is a property intended for developers creating custom plugins/profiles. End users who only wish to use any of the existing profiles don't have to know about this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq-docs pull request #94: HAWQ-1304 - multiple doc changes for P...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/94#discussion_r99964101 --- Diff: markdown/pxf/ReadWritePXF.html.md.erb --- @@ -131,6 +149,8 @@ Note: The DELIMITER parameter is mandatory. +**Notes**: Metadata identifies the Java class that provides field definitions in the relation. OutputFormat identifies the file format for which a specific profile is optimized. While the built-in `Hive*` profiles provide Metadata and OutputFormat classes, most profiles will have no need to implement or specify these classes. --- End diff -- I don't think this description about OutputFormat is the most accurate one. Oleks can you provide a more accurate description for this property --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-703. Serialize HCatalog Complex ...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/633#issuecomment-218238748 nice job with the refactoring of hawq types ! +! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-705. Fixed aggregation on psql f...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/628#issuecomment-214920718 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-683. Updates required for Javado...
Github user shivzone closed the pull request at: https://github.com/apache/incubator-hawq/pull/625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq-site pull request: Add link for HAWQ Extension Fram...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq-site/pull/6#issuecomment-213588552 I think its preferable to have the "docs" on top to point to a separate section. Including PXF javadocs link in that section makes more sense as opposed to adding pxf javadocs link under Contribute section --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq-site pull request: Updating doc links to point to h...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq-site/pull/5#issuecomment-213057563 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq-site pull request: Create README.md
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq-site/pull/3#issuecomment-213055799 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq-site pull request: HAWQ-683. Publish pxf javadoc ap...
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq-site/pull/4 HAWQ-683. Publish pxf javadoc api Added docs directory containing pxf api resources. HAWQ html file needs to be updated to include link to the PXF javadoc page. You can merge this pull request into a Git repository by running: $ git pull https://github.com/shivzone/incubator-hawq-site HAWQ-683 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq-site/pull/4.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4 commit ab8cf62a32e8c7904318356964a0d38bd433c921 Author: Shivram Mani <shivram.m...@gmail.com> Date: 2016-04-21T15:54:34Z HAWQ-683. Publish pxf javadoc api --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-683. Fix param name for Protocol...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/625#discussion_r60322236 --- Diff: pxf/README.md --- @@ -20,7 +20,7 @@ Package Contents PXF is distributed as a set of RPMs - - +git dif --- End diff -- my bad. removed it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-681. Removed hcatalog_enable GUC...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/621#discussion_r60271390
--- Diff: src/test/regress/input/hcatalog_lookup.source ---
@@ -2,16 +2,9 @@
-- test hcatalog lookup
-- --
--- Negative test with GUC disabled
-SET hcatalog_enable = false;
-SELECT * from hcatalog.db.t;
-
SELECT * FROM pxf_get_item_fields('Hive', '*');
--- End diff --
for each of the higher level use case. please add a one line comment on the
type of test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/604#discussion_r59805511
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -779,21 +779,24 @@ static void
RebuildNamespace(QueryContextInfo *cxt)
{
+ MemoryContext oldContext;
int len;
char buffer[4], *binary;
+ oldContext = MemoryContextSwitchTo(MessageContext);
ReadData(cxt, buffer, sizeof(buffer), TRUE);
len = (int) ntohl(*(uint32 *) buffer);
binary = palloc(len);
if(ReadData(cxt, binary, len, TRUE))
{
- StringInfoData buffer;
- initStringInfoOfString(, binary, len);
- dfs_address = strdup(buffer.data);
+ dfs_address = pstrdup(binary);
--- End diff --
identation needs to be 4 spaces
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-615. Handle incomptible tables w...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/551#issuecomment-210174733 @hornn the reason we went with returning an empty response was we wanted to be consistent with the way we treat the metadata api based on the given pattern. If the pattern has wildcards in it, we will never throw an exception in any case based on table incompatibilities. If the pattern corresponds to a specific table, we will throw the appropriate exception. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Adding dfs_address from pgf...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/412#issuecomment-210170271 This pull request is closed as it is outdated. Refer to the below 2 pull requests which solves it more generically for both readable and writable tables. https://github.com/apache/incubator-hawq/pull/503 https://github.com/apache/incubator-hawq/pull/604 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Adding dfs_address from pgf...
Github user shivzone closed the pull request at: https://github.com/apache/incubator-hawq/pull/412 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-644. Account for '\0' when dispa...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/604#discussion_r59638357 --- Diff: src/backend/cdb/cdbquerycontextdispatching.c --- @@ -3031,7 +3031,7 @@ prepareDfsAddressForDispatch(QueryContextInfo* cxt) if (!enable_secure_filesystem) return; const char *namespace = cxt->sharedPath; - int size = strlen(namespace); + int size = strlen(namespace) + 1; --- End diff -- please add a comment mentioning about null terminator character --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-628. Return -1 instead of error.
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/595#issuecomment-209102367 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-628. Return -1 instead of error.
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/595#discussion_r59450434
--- Diff: src/backend/utils/adt/dbsize.c ---
@@ -180,9 +180,7 @@ calculate_database_size(Oid dbOid)
Assert(Gp_role != GP_ROLE_EXECUTE);
if (dbOid == HcatalogDbOid)
- ereport(ERROR,
- (ERRCODE_UNDEFINED_DATABASE,
- errmsg("database hcatalog (OID 6120) is reserved")));
+ return -1;
--- End diff --
Should we return -1 ? For other non existent tables we return ERROR:
database does not exist.
Shouldn't we return an actual error instead of -1 ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request:
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/commit/59ebfa7072621117827ae3d9464c971a61919672#commitcomment-17053770 In src/backend/cdb/cdbquerycontextdispatching.c: In src/backend/cdb/cdbquerycontextdispatching.c on line 791: Thats true ! Just spoke to Kavinder, he is fixing this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request:
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/commit/59ebfa7072621117827ae3d9464c971a61919672#commitcomment-17050433 In src/backend/cdb/cdbquerycontextdispatching.c: In src/backend/cdb/cdbquerycontextdispatching.c on line 791: Good catch. This pull request is no longer valid though as we found a more generic way to pass the dfs_address for both readable and writable table flow via the query context https://github.com/apache/incubator-hawq/pull/503 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-178: Add JSON plugin support in ...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/302#issuecomment-206031571 @scottcode the appropriate forum for having questions/discussions such as the one you have is hawq user mailing list. You can find the available mailing lists here http://hawq.incubator.apache.org/#mailing-lists --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request:
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/commit/92d5acd341ee67c1d926882ee091ec7bde8e9ad2#commitcomment-16976473 +1 Thanks for fixing this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-615. Handle incomptible tables w...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/551#discussion_r58466610
--- Diff:
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/HiveMetadataFetcher.java
---
@@ -50,9 +50,21 @@ public HiveMetadataFetcher(InputData md) {
client = HiveUtilities.initHiveClient();
}
+/**
+ * Fetches metadata of hive tables corresponding to the given pattern
+ * For patterns matching more than one table, the tables are skipped.
--- End diff --
will update
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Hawq 546
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/529#discussion_r57979044
--- Diff: src/bin/psql/describe.c ---
@@ -4213,3 +4236,113 @@ printACLColumn(PQExpBuffer buf, const char *colname)
"pg_catalog.array_to_string(%s, '\\n') AS \"%s\"",
colname, gettext_noop("Access
privileges"));
}
+
+/*
+ * parsePxfPattern
+ *
+ * Splits user_pattern by "." and writes second part to pattern.
+ */
+static void
+parsePxfPattern(const char *user_pattern, char **pattern)
+{
+ strtok(user_pattern, ".");
+ *pattern = strtok(NULL, "/0");
+}
+
+/*
+ * describePxfTable
+ *
+ * Describes external PXF table.
+ */
+static bool
+describePxfTable(const char *profile, const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PQExpBufferData title;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ printTableContent cont;
+ int cols = 0;
+ int total_numrows = 0;
+ char *headers[2];
+ boolprintTableInitialized = false;
+
+ initPQExpBuffer();
+
+ printfPQExpBuffer(, "SELECT * FROM\n"
+ "pxf_get_item_fields('%s', '%s') \n", profile, pattern);
+
+ res = PSQLexec(buf.data, false);
+ total_numrows = PQntuples(res);
+ termPQExpBuffer();
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of Hive tables");
+ myopt.translate_header = true;
+
+
+ /* Header */
+ headers[0] = gettext_noop("Column");
+ headers[1] = gettext_noop("Type");
+ cols = 2;
+
+ char *previous_path = NULL;
+ char *previous_itemname = NULL;
+
+
--- End diff --
extra newline
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Hawq 546
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/529#discussion_r57973687
--- Diff: src/bin/psql/describe.c ---
@@ -4213,3 +4227,100 @@ printACLColumn(PQExpBuffer buf, const char *colname)
"pg_catalog.array_to_string(%s, '\\n') AS \"%s\"",
colname, gettext_noop("Access
privileges"));
}
+
+
+static void
+parsePxfPattern(const char *user_pattern, char **pattern)
+{
+ strtok(strdup(user_pattern), ".");
--- End diff --
will this work if the first strtok doesn't find the token ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Hawq 546
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/529#discussion_r57962901 --- Diff: src/bin/psql/describe.c --- @@ -1152,6 +1157,15 @@ describeTableDetails(const char *pattern, bool verbose, bool showSystem) PGresult *res; int i; + //Hive hook in this method --- End diff -- pattern null check required --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Hawq 546
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/529#discussion_r57962589
--- Diff: src/bin/psql/describe.c ---
@@ -4213,3 +4227,100 @@ printACLColumn(PQExpBuffer buf, const char *colname)
"pg_catalog.array_to_string(%s, '\\n') AS \"%s\"",
colname, gettext_noop("Access
privileges"));
}
+
+
+static void
--- End diff --
function level comment required
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Hawq 546
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/529#discussion_r57962186
--- Diff: src/bin/psql/describe.c ---
@@ -4213,3 +4227,100 @@ printACLColumn(PQExpBuffer buf, const char *colname)
"pg_catalog.array_to_string(%s, '\\n') AS \"%s\"",
colname, gettext_noop("Access
privileges"));
}
+
+
+static void
+parsePxfPattern(const char *user_pattern, char **pattern)
+{
+ strtok(strdup(user_pattern), ".");
+ *pattern = strtok(NULL, "/0");
+}
+
+static bool
--- End diff --
add comments describing the function
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Hawq 546
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/529#issuecomment-203544239 Can you please add a brief title/note/description about this change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Dispatch dfs_address from m...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/503#issuecomment-203543666 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Dispatch dfs_address from m...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/503#discussion_r57808336 --- Diff: src/backend/cdb/cdbquerycontextdispatching.c --- @@ -801,6 +825,9 @@ RebuildQueryContext(QueryContextInfo *cxt, HTAB **currentFilesystemCredentials, RebuildFilesystemCredentials(cxt, currentFilesystemCredentials, currentFilesystemCredentialsMemoryContext); break; +case Namespace: +RebuildNamespace(cxt); --- End diff -- Will this case be invoked in the non secure case where you don't set the namespace --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-577. Updated PXF metadata api to...
Github user shivzone closed the pull request at: https://github.com/apache/incubator-hawq/pull/522 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-577. Updated PXF metadata api to...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/522#discussion_r57661884
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/MetadataResponse.java
---
@@ -0,0 +1,94 @@
+package org.apache.hawq.pxf.service;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.List;
+
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.StreamingOutput;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hawq.pxf.api.Metadata;
+
+import org.codehaus.jackson.map.ObjectMapper;
+import org.codehaus.jackson.map.annotate.JsonSerialize.Inclusion;
+
+
+/**
+ * Class for serializing metadata in JSON format. The class implements
+ * {@link StreamingOutput} so the serialization will be done in a stream
and not
+ * in one bulk, this in order to avoid running out of memory when
processing a
+ * lot of items.
+ */
+public class MetadataResponse implements StreamingOutput {
+
+private static final Log Log =
LogFactory.getLog(MetadataResponse.class);
+private static final String METADATA_DEFAULT_RESPONSE =
"{\"PXFMetadata\":[]}";
+
+private List metadataList;
+
+/**
+ * Constructs metadata response out of a metadata list
+ *
+ * @param metadataList metadata list
+ */
+public MetadataResponse(List metadataList) {
+this.metadataList = metadataList;
+}
+
+/**
+ * Serializes the metadata list in JSON, To be used as the result
string for HAWQ.
+ */
+@Override
+public void write(OutputStream output) throws IOException,
+WebApplicationException {
+DataOutputStream dos = new DataOutputStream(output);
+ObjectMapper mapper = new ObjectMapper();
+mapper.setSerializationInclusion(Inclusion.NON_EMPTY); // ignore
empty fields
+
+if(metadataList == null || metadataList.isEmpty()) {
+dos.write(METADATA_DEFAULT_RESPONSE.getBytes());
+return;
+}
+
+dos.write("{\"PXFMetadata\":[".getBytes());
--- End diff --
What i meant was I wouldn't want to have any coorelation between
metadata_default_response and the header string used in the actual response. I
wouldn't mind introducing a separate constant for the string used in the acutal
response but wouldn't use that as part of the default response
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-577. Updated PXF metadata api to...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/522#discussion_r57657152
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/MetadataResponse.java
---
@@ -0,0 +1,94 @@
+package org.apache.hawq.pxf.service;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.List;
+
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.StreamingOutput;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.hawq.pxf.api.Metadata;
+
+import org.codehaus.jackson.map.ObjectMapper;
+import org.codehaus.jackson.map.annotate.JsonSerialize.Inclusion;
+
+
+/**
+ * Class for serializing metadata in JSON format. The class implements
+ * {@link StreamingOutput} so the serialization will be done in a stream
and not
+ * in one bulk, this in order to avoid running out of memory when
processing a
+ * lot of items.
+ */
+public class MetadataResponse implements StreamingOutput {
+
+private static final Log Log =
LogFactory.getLog(MetadataResponse.class);
+private static final String METADATA_DEFAULT_RESPONSE =
"{\"PXFMetadata\":[]}";
+
+private List metadataList;
+
+/**
+ * Constructs metadata response out of a metadata list
+ *
+ * @param metadataList metadata list
+ */
+public MetadataResponse(List metadataList) {
+this.metadataList = metadataList;
+}
+
+/**
+ * Serializes the metadata list in JSON, To be used as the result
string for HAWQ.
+ */
+@Override
+public void write(OutputStream output) throws IOException,
+WebApplicationException {
+DataOutputStream dos = new DataOutputStream(output);
+ObjectMapper mapper = new ObjectMapper();
+mapper.setSerializationInclusion(Inclusion.NON_EMPTY); // ignore
empty fields
+
+if(metadataList == null || metadataList.isEmpty()) {
+dos.write(METADATA_DEFAULT_RESPONSE.getBytes());
+return;
+}
+
+dos.write("{\"PXFMetadata\":[".getBytes());
--- End diff --
I didn't want to do that. The default response can possibly change tomorrow
and wanted to decouple that from a valid response string.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-577. Updated PXF metadata api to...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/522#discussion_r57655549 --- Diff: pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/rest/MetadataResource.java --- @@ -41,6 +41,7 @@ import org.apache.hawq.pxf.api.MetadataFetcher; import org.apache.hawq.pxf.api.utilities.InputData; import org.apache.hawq.pxf.service.MetadataFetcherFactory; +import org.apache.hawq.pxf.service.MetadataResponse; --- End diff -- yes, it is. I missed adding it and i've just updated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-577. Updated PXF metadata api to...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/522#discussion_r57654925
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/MetadataResponseFormatter.java
---
@@ -46,76 +45,50 @@
* @return JSON formatted response
* @throws IOException if converting the data to JSON fails
*/
-public static String formatResponseString(List metadataList)
throws IOException {
-return MetadataResponseFormatter.metadataToJSON(metadataList);
-}
-
-/**
- * Serializes a metadata in JSON,
- * To be used as the result string for HAWQ.
- * An example result is as follows:
- *
- *
{"PXFMetadata":[{"item":{"path":"default","name":"t1"},"fields":[{"name":"a","type":"int"},{"name":"b","type":"float"}]}]}
- */
-private static String metadataToJSON(List metadataList)
throws IOException {
-
-if (metadataList == null || metadataList.isEmpty()) {
- return METADATA_DEFAULT_RESPONSE;
+public static MetadataResponse formatResponse(List
metadataList, String path) throws IOException {
+/* print the fragment list to log when in debug level */
+if (LOG.isDebugEnabled()) {
+MetadataResponseFormatter.printMetadata(metadataList, path);
}
-StringBuilder result = null;
-
-for(Metadata metadata: metadataList) {
-if(metadata == null) {
-throw new IllegalArgumentException("metadata object is
null - cannot serialize");
-}
-if ((metadata.getFields() == null) ||
metadata.getFields().isEmpty()) {
-throw new IllegalArgumentException("metadata for " +
metadata.getItem() + " contains no fields - cannot serialize");
-}
-if (result == null) {
-result = new StringBuilder("{\"PXFMetadata\":["); /*
prefix info */
-} else {
-result.append(",");
-}
-
-ObjectMapper mapper = new ObjectMapper();
-mapper.setSerializationInclusion(Inclusion.NON_EMPTY); //
ignore empty fields
-result.append(mapper.writeValueAsString(metadata));
-}
-
-return result.append("]}").toString(); /* append suffix info */
-
+return new MetadataResponse(metadataList);
}
/**
* Converts metadata list to a readable string.
* Intended for debugging purposes only.
*/
-private static String metadataToString(List metadataList) {
-StringBuilder result = new StringBuilder("Metadata:");
+private static void printMetadata(List metadataList, String
path) {
+LOG.debug("Metadata List for path " + path + ": ");
+
+if (null == metadataList || metadataList.isEmpty()) {
+LOG.debug("No metadata");
+return;
+}
for(Metadata metadata: metadataList) {
-result.append(" Metadata for item \"");
+StringBuilder result = new StringBuilder();
+result.append("Metadata for item \"");
--- End diff --
good catch
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-577. Updated PXF metadata api to...
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq/pull/522 HAWQ-577. Updated PXF metadata api to stream response You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-hawq HAWQ-577 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/522.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #522 commit 68e30075976aff3e4b79dca08d5033e99c901b07 Author: Shivram Mani <shivram.m...@gmail.com> Date: 2016-03-28T23:31:34Z HAWQ-577. Updated PXF metadata api to stream response --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone closed the pull request at: https://github.com/apache/incubator-hawq/pull/477 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-465. Implement stored procedure ...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/479#issuecomment-200494858 +1 over all. good work with using the stored procedure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r57192932
--- Diff:
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java
---
@@ -203,19 +205,37 @@ private static boolean verifyModifers(String[]
modifiers) {
* It can be either table_name or
db_name.table_name.
*
* @param qualifiedName Hive table name
- * @return {@link org.apache.hawq.pxf.api.Metadata.Table} object
holding the full table name
+ * @return {@link Metadata.Item} object holding the full table name
*/
-public static Metadata.Table parseTableQualifiedName(String
qualifiedName) {
+public static Metadata.Item extractTableFromName(String qualifiedName)
{
+List items = extractTablesFromPattern(null,
qualifiedName);
+if(items.isEmpty()) {
+throw new IllegalArgumentException("No tables found");
--- End diff --
We don't use JSON response for any of the other errors we throw for the
other resources. We will have to be consistent across all the apis.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-465. Implement stored procedure ...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/479#issuecomment-199961634 @hornn the reason we didn't want to bump up the pxf api version was we haven't quite made a release on hawq git based on v14 and wanted to stick to this version until then --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r57041391
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/MetadataResponseFormatter.java
---
@@ -55,50 +54,65 @@ public static String formatResponseString(Metadata
metadata) throws IOException
* To be used as the result string for HAWQ.
* An example result is as follows:
*
- *
{"PXFMetadata":[{"table":{"dbName":"default","tableName":"t1"},"fields":[{"name":"a","type":"int"},{"name":"b","type":"float"}]}]}
+ *
{"PXFMetadata":[{"item":{"path":"default","name":"t1"},"fields":[{"name":"a","type":"int"},{"name":"b","type":"float"}]}]}
*/
-private static String metadataToJSON(Metadata metadata) throws
IOException {
+private static String metadataToJSON(List metadataList)
throws IOException {
--- End diff --
Very valid point. Will handle this in a separate pull request/jira
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/477#issuecomment-199948894 @sansanichfb is working on the related HAWQ changes to work with the new api --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r57039209
--- Diff:
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java
---
@@ -225,15 +245,47 @@ private static boolean verifyModifers(String[]
modifiers) {
}
if (toks.size() == 1) {
-dbName = HIVE_DEFAULT_DBNAME;
-tableName = toks.get(0);
+dbPattern = HIVE_DEFAULT_DBNAME;
+tablePattern = toks.get(0);
} else if (toks.size() == 2) {
-dbName = toks.get(0);
-tableName = toks.get(1);
+dbPattern = toks.get(0);
+tablePattern = toks.get(1);
} else {
-throw new IllegalArgumentException("\"" + qualifiedName + "\""
+ errorMsg);
+throw new IllegalArgumentException("\"" + pattern + "\"" +
errorMsg);
+}
+
+return getTablesFromPattern(client, dbPattern, tablePattern);
+
+}
+
+private static List
getTablesFromPattern(HiveMetaStoreClient client, String dbPattern, String
tablePattern) {
+
+List databases = null;
+List itemList = new ArrayList();
+List tables = new ArrayList();
+
+if(client == null || (!dbPattern.contains(WILDCARD) &&
!tablePattern.contains(WILDCARD)) ) {
+/* This case occurs when the call is invoked as part of the
fragmenter api or when metadata is requested for a specific table name */
+itemList.add(new Metadata.Item(dbPattern, tablePattern));
+return itemList;
}
-return new Metadata.Table(dbName, tableName);
+try {
+databases = client.getDatabases(dbPattern);
+if(databases.isEmpty()) {
+throw new IllegalArgumentException("No database found for
the given pattern");
+}
+for(String dbName: databases) {
+for(String tableName: client.getTables(dbName,
tablePattern)) {
+if (!tableName.isEmpty()) {
--- End diff --
will update, since we would anyway get a nosuchobjectexception if there are
not tables
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r57037899
--- Diff:
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java
---
@@ -225,15 +245,53 @@ private static boolean verifyModifers(String[]
modifiers) {
}
if (toks.size() == 1) {
-dbName = HIVE_DEFAULT_DBNAME;
-tableName = toks.get(0);
+dbPattern = HIVE_DEFAULT_DBNAME;
+tablePattern = toks.get(0);
} else if (toks.size() == 2) {
-dbName = toks.get(0);
-tableName = toks.get(1);
+dbPattern = toks.get(0);
+tablePattern = toks.get(1);
} else {
-throw new IllegalArgumentException("\"" + qualifiedName + "\""
+ errorMsg);
+throw new IllegalArgumentException("\"" + pattern + "\"" +
errorMsg);
}
-return new Metadata.Table(dbName, tableName);
+return getTablesFromPattern(client, dbPattern, tablePattern);
+
+}
+
+private static List
getTablesFromPattern(HiveMetaStoreClient client, String dbPattern, String
tablePattern) {
+
+List databases = null;
+List itemList = new ArrayList();
+List tables = new ArrayList();
+
+if(client == null || (!dbPattern.contains(WILDCARD) &&
!tablePattern.contains(WILDCARD)) ) {
+/* This case occurs when the call is invoked as part of the
fragmenter api or when metadata is requested for a specific table name */
+itemList.add(new Metadata.Item(dbPattern, tablePattern));
+return itemList;
+}
+
+try {
+/*if(dbPattern.contains(WILDCARD)) {
+databases.addAll(client.getAllDatabases());
+}*/
+databases = client.getDatabases(dbPattern);
+if(databases.isEmpty()) {
+throw new IllegalArgumentException("no database found for
the given pattern");
+}
+for(String dbName: databases) {
+for(String tableName: client.getTables(dbName,
tablePattern)) {
+if (!tableName.isEmpty()) {
+itemList.add(new Metadata.Item(dbName, tableName));
+}
+}
+}
+if(itemList.isEmpty()) {
+throw new IllegalArgumentException("no tables found");
+}
+return itemList;
+
+} catch (MetaException cause) {
+throw new RuntimeException("Failed connecting to Hive
MetaStore service: " + cause.getMessage(), cause);
--- End diff --
Updated case. Software engineers are not required to use perfect grammar :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-465. Implement stored procedure ...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/479#discussion_r57036285
--- Diff: src/backend/catalog/external/externalmd.c ---
@@ -438,18 +444,18 @@ void LoadHCatalogColumns(Oid relid, List *columns)
AttrNumber attno = 1;
foreach(lc, columns)
{
- HCatalogColumn *hcatCol = lfirst(lc);
+ PxfField *hcatCol = lfirst(lc);
--- End diff --
change from hcatCol to fieldCol ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r56933812
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/utilities/ProtocolData.java
---
@@ -95,6 +95,7 @@ public ProtocolData(Map<String, String> paramsMap) {
accessor = getProperty("ACCESSOR");
resolver = getProperty("RESOLVER");
fragmenter = getOptionalProperty("FRAGMENTER");
+metadata = getOptionalProperty("METADATA");
--- End diff --
The first one i had in mind was DESIGNER :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r56920623
--- Diff:
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java
---
@@ -203,19 +205,37 @@ private static boolean verifyModifers(String[]
modifiers) {
* It can be either table_name or
db_name.table_name.
*
* @param qualifiedName Hive table name
- * @return {@link org.apache.hawq.pxf.api.Metadata.Table} object
holding the full table name
+ * @return {@link Metadata.Item} object holding the full table name
*/
-public static Metadata.Table parseTableQualifiedName(String
qualifiedName) {
+public static Metadata.Item extractTableFromName(String qualifiedName)
{
+List items = extractTablesFromPattern(null,
qualifiedName);
+if(items.isEmpty()) {
+return null;
+}
+return items.get(0);
+}
+
+/**
+ * Extracts the db_name and table_name from the qualifiedName.
+ * qualifiedName is the Hive table name or pattern that the user
enters in the CREATE EXTERNAL TABLE statement
+ * or when querying HCatalog table.
+ * It can be either table_name_pattern or
db_name_pattern.table_name_pattern.
+ *
+ * @param client Hivemetastore client
+ * @param pattern Hive table name or pattern
+ * @return {@link Metadata.Item} object holding the full table name
+ */
+public static List
extractTablesFromPattern(HiveMetaStoreClient client, String pattern) {
-String dbName, tableName;
+String dbPattern, tablePattern;
String errorMsg = " is not a valid Hive table name. "
+ "Should be either or ";
-if (StringUtils.isBlank(qualifiedName)) {
+if (StringUtils.isBlank(pattern)) {
throw new IllegalArgumentException("empty string" + errorMsg);
}
-String[] rawToks = qualifiedName.split("[.]");
+String[] rawToks = pattern.split("[.]");
ArrayList toks = new ArrayList();
for (String tok: rawToks) {
if (StringUtils.isBlank(tok)) {
--- End diff --
Thats true but doesn't seem right. I'm not quite sure why we had this
earlier
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/477#discussion_r56919278 --- Diff: pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/MetadataFetcher.java --- @@ -1,5 +1,10 @@ package org.apache.hawq.pxf.api; +import java.util.List; + +import org.apache.hawq.pxf.api.utilities.InputData; +import org.apache.hawq.pxf.api.utilities.Plugin; + --- End diff -- you are right --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r56919152
--- Diff:
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/MetadataFetcher.java ---
@@ -21,25 +26,26 @@
/**
- * Abstract class that defines getting metadata of a table.
+ * Abstract class that defines getting metadata of an item.
*/
-public abstract class MetadataFetcher {
-protected Metadata metadata;
+public abstract class MetadataFetcher extends Plugin {
+protected List metadata;
/**
* Constructs a MetadataFetcher.
*
+ * @param metaData the input data
*/
-public MetadataFetcher() {
-
+public MetadataFetcher(InputData metaData) {
+super(metaData);
}
/**
* Gets a metadata of a given table
--- End diff --
Will change to item
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ 459 Enhanced metadata api to sup...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/477#discussion_r56919089
--- Diff: pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
---
@@ -26,51 +26,52 @@
import org.apache.commons.lang.StringUtils;
/**
- * Metadata holds a table's metadata information.
- * {@link MetadataFetcher#getTableMetadata} returns the table's metadata.
+ * Metadata holds an item's metadata information.
+ * {@link MetadataFetcher#getMetadata} returns the item's metadata.
*/
public class Metadata {
/**
- * Class representing table name - db (schema) name and table name.
+ * Class representing item name - db/schema/path name and table/file
name.
*/
-public static class Table {
-private String dbName;
-private String tableName;
+public static class Item {
+private String path;
+private String name;
-public Table(String dbName, String tableName) {
+public Item(String path, String itemName) {
-if (StringUtils.isBlank(dbName) ||
StringUtils.isBlank(tableName)) {
-throw new IllegalArgumentException("Table name cannot be
empty");
+if (StringUtils.isBlank(path) ||
StringUtils.isBlank(itemName)) {
+throw new IllegalArgumentException("Item name cannot be
empty");
--- End diff --
this checks for both
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-462. Adding dfs_address from pgf...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/412#discussion_r55436107 --- Diff: src/backend/access/external/hd_work_mgr.c --- @@ -768,6 +769,12 @@ make_allocation_output_string(List *segment_fragments) initStringInfo(); appendStringInfoString(, SEGWORK_PREFIX); + /* Add dfs_address from pg_filespace to the segment data. Fixes HAWQ-462 *//* dfs_address from pg_filespace entry */ + char* dfs_address = NULL; + get_hdfs_location_from_filespace(_address); --- End diff -- Updated to check for secure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Adding dfs_address from pgf...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/412#discussion_r55416677 --- Diff: src/backend/access/external/hd_work_mgr.c --- @@ -783,6 +786,10 @@ make_allocation_output_string(List *segment_fragments) appendStringInfoChar(_str, SEGWORK_IN_PAIR_DELIM); if (frag->fragment_md) appendStringInfo(_str, "%s", frag->fragment_md); + /* Adding dfs_address from pg_filespace entry required for HAWQ-462 */ + appendStringInfoChar(_str, SEGWORK_IN_PAIR_DELIM); + if (dfs_address) --- End diff -- I've updated the pull request to move the dfs address to seg level metadata instead of fragment level metadata --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Adding dfs_address from pgf...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/412#issuecomment-192482294 The dfs_address will not be required in the case of non-secure cluster. Will handle this once we update this to be passed via segment data as opposed to fragment data --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-462. Adding dfs_address from pgf...
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq/pull/412 HAWQ-462. Adding dfs_address from pgfilespace along with fragment dat… HAWQ-317 attempts to use the dfs_address from pg_filespace_entry. The segments (gpbridge) do not have catalog access. Adding the dfs_address along with the fragment data passed from the master to the segments. This fixes the pxf handling of secure isilon and hcatalog with or without HA You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-hawq HAWQ-462 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/412.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #412 commit 50926e7470e4d5a6c6857d57063f8b8f5634d6f8 Author: Shivram Mani <shivram.m...@gmail.com> Date: 2016-03-04T18:49:13Z HAWQ-462. Adding dfs_address from pgfilespace along with fragment data sent to segments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-432. Memory leaks in pg_regress.
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/368#issuecomment-187453971 good to know. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-432. Memory leaks in pg_regress.
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/368#issuecomment-187422604 Can you post the link to the updated coverity build to verify if there are no more leaks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-423. Updated gradle and download...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/356#discussion_r53423441
--- Diff: pxf/build.gradle ---
@@ -434,7 +435,6 @@ task tomcatRpm(type: Rpm) {
packageName 'apache-tomcat'
summary = 'Apache Tomcat RPM'
vendor = project.vendor
-release = buildNumber()
--- End diff --
The HAWQ pulse build number is not relevant for the apache tomcat rpm.
Besides we are not releasing internal custom builds of apache tomcat
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-423. Updated gradle and download...
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq/pull/356 HAWQ-423. Updated gradle and download plugin version You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-hawq HAWQ-423 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/356.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #356 commit 4883f80136e0a76d2e7211a2aaaf764afcc05dd5 Author: Shivram Mani <shivram.m...@gmail.com> Date: 2016-02-18T21:44:17Z HAWQ-423. Updated gradle and download plugin version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-400. Support expected exit codes...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/347#issuecomment-185460070 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-400. Support expected exit codes...
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/347#discussion_r53107862 --- Diff: src/test/regress/GNUmakefile --- @@ -143,7 +143,7 @@ installcheck-parallel: all upg2-setup ugpart-setup $(pg_regress_call) --psqldir=$(PSQLDIR) --schedule=$(srcdir)/parallel_schedule --srcdir=$(abs_srcdir) installcheck-good: all ./current_good_schedule upg2-setup ugpart-setup - $(pg_regress_call) --psqldir=$(PSQLDIR) --schedule=./current_good_schedule --srcdir=$(abs_srcdir) + $(pg_regress_call) --psqldir=$(PSQLDIR) --schedule=./current_good_schedule --srcdir=$(abs_srcdir) --expected-statuses-file=expected_statuses --- End diff -- can we instead call it expected-status-file. it will be implicit that this file will contain one status for each test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-400. Support expected exit codes...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/347#discussion_r53107793
--- Diff: src/test/regress/pg_regress.c ---
@@ -41,6 +41,14 @@ typedef struct _resultmap
struct _resultmap *next;
} _resultmap;
+/*linked list of statuses for tests*/
+typedef struct _statuslist
+{
+ char*test;
+ int status;
--- End diff --
fix indentation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-405. Fix for handling of pg_temp...
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq/pull/338 HAWQ-405. Fix for handling of pg_temp schema You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-hawq HAWQ-405 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/338.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #338 commit d431585ecd451549419081316eab7308249e4284 Author: Shivram Mani <shivram.m...@gmail.com> Date: 2016-02-10T21:43:04Z HAWQ-405. Fix for handling of pg_temp schema --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-340. Make getVersion API return ...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/290#discussion_r50771928
--- Diff:
pxf/pxf-service/src/test/java/org/apache/hawq/pxf/service/rest/VersionResourceTest.java
---
@@ -22,21 +22,26 @@
import static org.junit.Assert.assertEquals;
+import javax.ws.rs.core.HttpHeaders;
+import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.junit.Test;
public class VersionResourceTest {
-@Test
-public void getProtocolVersion() throws Exception {
+ @Test
+ public void getProtocolVersion() throws Exception {
-VersionResource resource = new VersionResource();
-Response result = resource.getProtocolVersion();
+ VersionResource resource = new VersionResource();
--- End diff --
Fix your indentation to 4 space instead of 8 space
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-340. Make getVersion API return ...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/290#issuecomment-174725634 +1 Nice fix --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Hawq 317
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/279#issuecomment-172989559 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Hawq 317
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/279#discussion_r50160999
--- Diff: src/backend/access/external/pxfutils.c ---
@@ -47,6 +52,24 @@ void port_to_str(char **port, int new_port)
}
/*
+ * get_hdfs_location_from_filespace
+ *
+ * Get hdfs location from pg_filespace_entry
+ * The returned path needs to be pfreed by the caller.
+ */
+void get_hdfs_location_from_filespace(char** path)
+{
+ Assert(NULL != path);
+ Oid dtsoid = get_database_dts(MyDatabaseId);
--- End diff --
what is MyDatabaseId and Where is it set ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Hawq 317
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/279#discussion_r50160316 --- Diff: src/backend/access/external/hd_work_mgr.c --- @@ -900,36 +900,30 @@ static void init_client_context(ClientContext *client_context) * is created. We cannot use that token because hd_work_mgr.c code gets * executed before a portal is created. * - * The function uses a hdfs uri in the form of hdfs://host:port/ - * where port is 8020. In the case of HA the function uses the form - * hdfs://nameservice/ - * - * TODO 8020 is hard-coded. Must find the NameNode port somehow. + * The function uses a hdfs uri in the form of hdfs://host:port/path. --- End diff -- hdfs://host:port/path or hdfs://nameservice/path --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Hawq 317
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/279#discussion_r50161526
--- Diff: src/backend/cdb/cdbquerycontextdispatching.c ---
@@ -2872,31 +2872,27 @@ static char* GetExtTableFirstLocation(Datum *array)
* Using host from uri, dispatch HDFS credentials to
* segments.
*
- * The function uses a hdfs uri in the form hdfs://host:port/ where
- * port is hard-coded 8020. For HA the function uses hdfs://nameservice/
+ * The function uses a hdfs uri in the form hdfs://host:port/path where
+ * this value is taken from pg_filespace_entry which is populated
+ * based on hawq-site.xml's hawq_dfs_url entry.
*
* prepareDispatchedCatalogFileSystemCredential will store the token
- * using port == 0 in HA case (otherwise the supplied port)
- *
- * TODO Get HDFS port from someplace else, currently hard coded
+ * using port == 0 in HA case (otherwise the supplied port).
*/
static void AddFileSystemCredentialForPxfTable(char *uri)
{
- StringInfoData hdfs_uri;
+ char* dfs_address = NULL;
- initStringInfo(_uri);
- GPHDUri *gphd_uri = parseGPHDUri(uri);
-if (gphd_uri->ha_nodes)
-appendStringInfo(_uri, "hdfs://%s/",
gphd_uri->ha_nodes->nameservice);
-else
-appendStringInfo(_uri, "hdfs://%s:8020/", gphd_uri->host);
+ if (!enable_secure_filesystem)
--- End diff --
we didn't have this check earlier ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-103. Build PXF RPMs with license...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/245#issuecomment-169165561 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-277. Remove gradle-wrapper.jar
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/216#issuecomment-167855175 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-191. Remove Analyzer plugin from...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/174#discussion_r47581115
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/rest/InvalidPathResource.java
---
@@ -153,9 +167,39 @@ private String parseVersion(String path) {
}
/**
+ * Parses the version part from the path.
--- End diff --
Fixed this directly on master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-191. Remove Analyzer plugin from...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/174#discussion_r47579068
--- Diff: pxf/build.gradle ---
@@ -218,7 +218,7 @@ project('pxf-service') {
project('pxf-hdfs') {
dependencies {
-compile(project(':pxf-service')) //Yikes, HdfsAnalyzer is directly
accessing the bridge
+compile(project(':pxf-service'))
--- End diff --
Yes, separate jira makes more sense
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-191. Remove Analyzer plugin from...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/174#discussion_r47532428
--- Diff: pxf/build.gradle ---
@@ -218,7 +218,7 @@ project('pxf-service') {
project('pxf-hdfs') {
dependencies {
-compile(project(':pxf-service')) //Yikes, HdfsAnalyzer is directly
accessing the bridge
+compile(project(':pxf-service'))
--- End diff --
There are other compile time dependancies unrelated to the analyzer
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-126. Switch PXF to use 3 digit s...
Github user shivzone closed the pull request at: https://github.com/apache/incubator-hawq/pull/80 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-205. PXF Throws NullPointerExcep...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/150#issuecomment-162679602 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-179. New PXF resource to return ...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/143#issuecomment-161011982 +1. Make sure doc is updated with this api. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-175. Update PXF version to 3.1.0
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/128#issuecomment-159017200 Would be useful if we can mention the higher level changes in the jira HAWQ-175 for future reference. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/92#discussion_r45493712
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/ReadBridge.java ---
@@ -14,58 +14,81 @@
import java.io.*;
import java.nio.charset.CharacterCodingException;
+import java.util.LinkedList;
import java.util.zip.ZipException;
-/*
- * ReadBridge class creates appropriate accessor and resolver.
- * It will then create the correct output conversion
- * class (e.g. Text or GPDBWritable) and get records from accessor,
- * let resolver deserialize them and reserialize them using the
- * output conversion class.
- *
- * The class handles BadRecordException and other exception type
- * and marks the record as invalid for GPDB.
+/**
+ * ReadBridge class creates appropriate accessor and resolver. It will then
+ * create the correct output conversion class (e.g. Text or GPDBWritable)
and
+ * get records from accessor, let resolver deserialize them and
reserialize them
+ * using the output conversion class.
+ * The class handles BadRecordException and other exception type and marks
the
+ * record as invalid for HAWQ.
*/
public class ReadBridge implements Bridge {
ReadAccessor fileAccessor = null;
ReadResolver fieldsResolver = null;
BridgeOutputBuilder outputBuilder = null;
+LinkedList outputQueue = null;
-private Log Log;
+static private Log Log = LogFactory.getLog(ReadBridge.class);
-/*
- * C'tor - set the implementation of the bridge
+/**
+ * C'tor - set the implementation of the bridge.
+ *
+ * @param protData input containing accessor and resolver names
+ * @throws Exception if accessor or resolver can't be instantiated
*/
public ReadBridge(ProtocolData protData) throws Exception {
outputBuilder = new BridgeOutputBuilder(protData);
-Log = LogFactory.getLog(ReadBridge.class);
+outputQueue = new LinkedList();
fileAccessor = getFileAccessor(protData);
fieldsResolver = getFieldsResolver(protData);
}
-/*
- * Accesses the underlying HDFS file
+/**
+ * Accesses the underlying HDFS file.
*/
@Override
public boolean beginIteration() throws Exception {
return fileAccessor.openForRead();
}
-/*
- * Fetch next object from file and turn it into a record that the GPDB
backend can process
+/**
+ * Fetches next object from file and turn it into a record that the
HAWQ
+ * backend can process.
*/
@Override
public Writable getNext() throws Exception {
-Writable output;
+Writable output = null;
--- End diff --
Aah ... you are right
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/92#issuecomment-158465037 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-56. Add format option to externa...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/121#issuecomment-158464458 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/92#discussion_r44827404
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/ReadBridge.java ---
@@ -14,58 +14,81 @@
import java.io.*;
import java.nio.charset.CharacterCodingException;
+import java.util.LinkedList;
import java.util.zip.ZipException;
-/*
- * ReadBridge class creates appropriate accessor and resolver.
- * It will then create the correct output conversion
- * class (e.g. Text or GPDBWritable) and get records from accessor,
- * let resolver deserialize them and reserialize them using the
- * output conversion class.
- *
- * The class handles BadRecordException and other exception type
- * and marks the record as invalid for GPDB.
+/**
+ * ReadBridge class creates appropriate accessor and resolver. It will then
+ * create the correct output conversion class (e.g. Text or GPDBWritable)
and
+ * get records from accessor, let resolver deserialize them and
reserialize them
+ * using the output conversion class.
+ * The class handles BadRecordException and other exception type and marks
the
+ * record as invalid for HAWQ.
*/
public class ReadBridge implements Bridge {
ReadAccessor fileAccessor = null;
ReadResolver fieldsResolver = null;
BridgeOutputBuilder outputBuilder = null;
+LinkedList outputQueue = null;
-private Log Log;
+static private Log Log = LogFactory.getLog(ReadBridge.class);
--- End diff --
best practice is to define log objects as private static final when you
initialize it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/92#discussion_r44704661
--- Diff:
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/FragmentsStats.java ---
@@ -0,0 +1,226 @@
+package org.apache.hawq.pxf.api;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.codehaus.jackson.map.ObjectMapper;
+
+import java.io.IOException;
+
+/**
+ * FragmentsStats holds statistics for a given path.
+ */
+public class FragmentsStats {
+
+/**
+ * Default fragment size. Assuming a fragment is equivalent to a block
in
+ * HDFS, we guess a full fragment size is 64MB.
+ */
+public static final long DEFAULT_FRAGMENT_SIZE = 67108864L;
--- End diff --
This should eventually be exposed as a configuration parameter
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/92#discussion_r44705929
--- Diff:
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/FragmentsStats.java ---
@@ -0,0 +1,226 @@
+package org.apache.hawq.pxf.api;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.codehaus.jackson.map.ObjectMapper;
+
+import java.io.IOException;
+
+/**
+ * FragmentsStats holds statistics for a given path.
+ */
+public class FragmentsStats {
+
+/**
+ * Default fragment size. Assuming a fragment is equivalent to a block
in
+ * HDFS, we guess a full fragment size is 64MB.
+ */
+public static final long DEFAULT_FRAGMENT_SIZE = 67108864L;
+
+private static Log Log = LogFactory.getLog(FragmentsStats.class);
+
+// number of fragments
+private long fragmentsNumber;
+// first fragment size
+private SizeAndUnit firstFragmentSize;
+// total fragments size
+private SizeAndUnit totalSize;
+
+/**
+ * Enum to represent unit (Bytes/MB/GB/TB)
+ */
+public enum SizeUnit {
--- End diff --
Good job with the handling of large size
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/92#discussion_r44705379
--- Diff:
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/FragmentsStats.java ---
@@ -0,0 +1,226 @@
+package org.apache.hawq.pxf.api;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.codehaus.jackson.map.ObjectMapper;
+
+import java.io.IOException;
+
+/**
+ * FragmentsStats holds statistics for a given path.
+ */
+public class FragmentsStats {
+
+/**
+ * Default fragment size. Assuming a fragment is equivalent to a block
in
+ * HDFS, we guess a full fragment size is 64MB.
+ */
+public static final long DEFAULT_FRAGMENT_SIZE = 67108864L;
+
+private static Log Log = LogFactory.getLog(FragmentsStats.class);
+
+// number of fragments
+private long fragmentsNumber;
+// first fragment size
+private SizeAndUnit firstFragmentSize;
+// total fragments size
+private SizeAndUnit totalSize;
+
+/**
+ * Enum to represent unit (Bytes/MB/GB/TB)
+ */
+public enum SizeUnit {
+/**
+ * Byte
+ */
+B,
+/**
+ * KB
+ */
+KB,
+/**
+ * MB
+ */
+MB,
+/**
+ * GB
+ */
+GB,
+/**
+ * TB
+ */
+TB;
+};
+
+/**
+ * Container for size and unit
+ */
+public class SizeAndUnit {
+long size;
+SizeUnit unit;
+
+/**
+ * Default constructor.
+ */
+public SizeAndUnit() {
+this.size = 0;
+this.unit = SizeUnit.B;
+}
+
+/**
+ * Constructor.
+ *
+ * @param size size
+ * @param unit unit
+ */
+public SizeAndUnit(long size, SizeUnit unit) {
+this.size = size;
+this.unit = unit;
+}
+
+/**
+ * Returns size.
+ *
+ * @return size
+ */
+public long getSize() {
+return this.size;
+}
+
+/**
+ * Returns unit (Byte/KB/MB/etc.).
+ *
+ * @return unit
+ */
+public SizeUnit getUnit() {
+return this.unit;
+}
+
+@Override
+public String toString() {
+return size + "" + unit;
+}
+}
+
+/**
+ * Constructs an FragmentsStats.
+ *
+ * @param fragmentsNumber number of fragments
+ * @param firstFragmentSize first fragment size (in bytes)
+ * @param totalSize total size (in bytes)
+ */
+public FragmentsStats(long fragmentsNumber, long firstFragmentSize,
+ long totalSize) {
+this.setFragmentsNumber(fragmentsNumber);
+this.setFirstFragmentSize(firstFragmentSize);
+this.setTotalSize(totalSize);
+}
+
+/**
+ * Given a {@link FragmentsStats}, serialize it in JSON to be used as
the
+ * result string for HAWQ. An example result is as follows:
+ * {"PXFFragmentsStats":{"fragmentsNumber"
+ * :3,"firstFragmentSize":67108864,"totalSize":2}}
+ *
+ * @param stats the data to be serialized
+ * @return the result in json format
+ * @throws IOException if converting to JSON format failed
+ */
+public static String dataToJSON(FragmentsStats stats) throws
IOException {
+ObjectMapper mapper = new ObjectMapper();
+// mapper serializes all members of the class by default
+return "{\"PXFFragmentsStats\":" + mapper.writeValueAsString(stats)
++ "}";
+}
+
+/**
+ * Given a stats structure, convert it to be readable. Intended for
+ * debugging purposes only.
+ *
+ * @param stats the data to be stringify
+ * @param datapath the data path part of the original URI (e.g., table
name,
+ **.csv, etc.)
+ * @return the stringified data
+ */
+public static String dataToString(FragmentsStats stats, String
datapath) {
+return "Statistics information for \"" + datapath + "\" "
++ " Number of Fragments: " + stats.fragmentsNumber
++ ", first Fragment size: " + st
[GitHub] incubator-hawq pull request: Analyze HAWQ-44
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/92#discussion_r44731388
--- Diff:
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/ReadBridge.java ---
@@ -14,58 +14,81 @@
import java.io.*;
import java.nio.charset.CharacterCodingException;
+import java.util.LinkedList;
import java.util.zip.ZipException;
-/*
- * ReadBridge class creates appropriate accessor and resolver.
- * It will then create the correct output conversion
- * class (e.g. Text or GPDBWritable) and get records from accessor,
- * let resolver deserialize them and reserialize them using the
- * output conversion class.
- *
- * The class handles BadRecordException and other exception type
- * and marks the record as invalid for GPDB.
+/**
+ * ReadBridge class creates appropriate accessor and resolver. It will then
+ * create the correct output conversion class (e.g. Text or GPDBWritable)
and
+ * get records from accessor, let resolver deserialize them and
reserialize them
+ * using the output conversion class.
+ * The class handles BadRecordException and other exception type and marks
the
+ * record as invalid for HAWQ.
*/
public class ReadBridge implements Bridge {
ReadAccessor fileAccessor = null;
ReadResolver fieldsResolver = null;
BridgeOutputBuilder outputBuilder = null;
+LinkedList outputQueue = null;
-private Log Log;
+static private Log Log = LogFactory.getLog(ReadBridge.class);
-/*
- * C'tor - set the implementation of the bridge
+/**
+ * C'tor - set the implementation of the bridge.
+ *
+ * @param protData input containing accessor and resolver names
+ * @throws Exception if accessor or resolver can't be instantiated
*/
public ReadBridge(ProtocolData protData) throws Exception {
outputBuilder = new BridgeOutputBuilder(protData);
-Log = LogFactory.getLog(ReadBridge.class);
+outputQueue = new LinkedList();
fileAccessor = getFileAccessor(protData);
fieldsResolver = getFieldsResolver(protData);
}
-/*
- * Accesses the underlying HDFS file
+/**
+ * Accesses the underlying HDFS file.
*/
@Override
public boolean beginIteration() throws Exception {
return fileAccessor.openForRead();
}
-/*
- * Fetch next object from file and turn it into a record that the GPDB
backend can process
+/**
+ * Fetches next object from file and turn it into a record that the
HAWQ
+ * backend can process.
*/
@Override
public Writable getNext() throws Exception {
-Writable output;
+Writable output = null;
--- End diff --
java by default initializes objects to null
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-102. Change error message when P...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/100#discussion_r44692378
--- Diff: src/backend/access/external/libchurl.c ---
@@ -815,42 +815,49 @@ void check_response_code(churl_context* context)
}
else if (response_code != 200 && response_code != 100)
{
- if (!handle_special_error(response_code))
- {
- StringInfoData err;
- char*http_error_msg;
- char*addr;
+ StringInfoData err;
+ char*http_error_msg;
+ char*addr;
- initStringInfo();
+ initStringInfo();
- /* prepare response text if any */
- if (context->download_buffer->ptr)
- {
-
context->download_buffer->ptr[context->download_buffer->top] = '\0';
- response_text = context->download_buffer->ptr +
context->download_buffer->bot;
- }
+ /* prepare response text if any */
+ if (context->download_buffer->ptr)
+ {
+
context->download_buffer->ptr[context->download_buffer->top] = '\0';
+ response_text = context->download_buffer->ptr +
context->download_buffer->bot;
+ }
- /* add remote http error code */
- appendStringInfo(, "remote component error (%ld)",
response_code);
+ /* add remote http error code */
+ appendStringInfo(, "remote component error (%ld)",
response_code);
- addr = get_dest_address(context->curl_handle);
- if (strlen(addr) != 0)
- appendStringInfo(, " from %s", addr);
- pfree(addr);
+ addr = get_dest_address(context->curl_handle);
+ if (strlen(addr) != 0)
+ {
+ appendStringInfo(, " from %s", addr);
--- End diff --
Sounds good
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-102. Change error message when P...
Github user shivzone commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/100#discussion_r44687600
--- Diff: src/backend/access/external/libchurl.c ---
@@ -815,42 +815,49 @@ void check_response_code(churl_context* context)
}
else if (response_code != 200 && response_code != 100)
{
- if (!handle_special_error(response_code))
- {
- StringInfoData err;
- char*http_error_msg;
- char*addr;
+ StringInfoData err;
+ char*http_error_msg;
+ char*addr;
- initStringInfo();
+ initStringInfo();
- /* prepare response text if any */
- if (context->download_buffer->ptr)
- {
-
context->download_buffer->ptr[context->download_buffer->top] = '\0';
- response_text = context->download_buffer->ptr +
context->download_buffer->bot;
- }
+ /* prepare response text if any */
+ if (context->download_buffer->ptr)
+ {
+
context->download_buffer->ptr[context->download_buffer->top] = '\0';
+ response_text = context->download_buffer->ptr +
context->download_buffer->bot;
+ }
- /* add remote http error code */
- appendStringInfo(, "remote component error (%ld)",
response_code);
+ /* add remote http error code */
+ appendStringInfo(, "remote component error (%ld)",
response_code);
- addr = get_dest_address(context->curl_handle);
- if (strlen(addr) != 0)
- appendStringInfo(, " from %s", addr);
- pfree(addr);
+ addr = get_dest_address(context->curl_handle);
+ if (strlen(addr) != 0)
+ {
+ appendStringInfo(, " from %s", addr);
--- End diff --
hawq codebase seems to follow convention of not using braces on single
statement action on conditional blocks
if(case)
code
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request: HAWQ-102. Change error message when P...
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/100#issuecomment-156178390 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-126. Switch PXF to use 3 digit s...
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq/pull/80 HAWQ-126. Switch PXF to use 3 digit semnatic versioning You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-hawq HAWQ-126 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/80.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #80 commit f0625f1bf889844465463c18da3059ddb5511ebd Author: Shivram Mani <shivram.m...@gmail.com> Date: 2015-11-05T23:29:05Z HAWQ-126. Switch PXF to use 3 digit semnatic versioning --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-45. PXF Package Namespace change
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/62#issuecomment-153121094 @hornn Will do this post namespace change. The goal of the first change is to make sure we are feature parity including automation tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-45. PXF Package Namespace change
Github user shivzone commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/62#discussion_r43522666 --- Diff: pxf/pxf-hdfs/src/test/java/org/apache/hawq/pxf/plugins/hdfs/ChunkReaderTest.java --- @@ -1,18 +1,15 @@ -package com.pivotal.pxf.plugins.hdfs; +package org.apache.hawq.pxf.plugins.hdfs; import org.apache.hadoop.io.Writable; import org.apache.hadoop.hdfs.DFSInputStream; -import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.mockito.Mockito; import org.mockito.stubbing.*; import org.mockito.invocation.*; -import org.powermock.api.mockito.PowerMockito; import org.powermock.core.classloader.annotations.PrepareForTest; -import org.powermock.core.classloader.annotations.SuppressStaticInitializationFor; --- End diff -- Those were unused imports. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-45. PXF Package Namespace change
GitHub user shivzone opened a pull request: https://github.com/apache/incubator-hawq/pull/62 HAWQ-45. PXF Package Namespace change This change will go hand in hand with other dependent changes listed below 1. Update to the internal automation tests 2. Update to internal cluster tools (singlecluster) 3. Doc updates You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/incubator-hawq HAWQ-45 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/62.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #62 commit 1c7ab9eb83858629089e52176e1a231f91ed923f Author: Shivram Mani <shivram.m...@gmail.com> Date: 2015-10-28T22:12:41Z HAWQ-45. PXF Package Namespace change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-45. PXF Package Namespace change
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/62#issuecomment-152312514 @hornn I completely agree. I think Goden created a task for it. Besides, all of our documents will need to be updated along with this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: HAWQ-53. Fix to handling nulls in com...
Github user shivzone closed the pull request at: https://github.com/apache/incubator-hawq/pull/33 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request: Hawq 5
Github user shivzone commented on the pull request: https://github.com/apache/incubator-hawq/pull/8#issuecomment-145150745 Nice catch with the closeForRead in ReadBridge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
