[jira] [Commented] (HAWQ-256) Integrate Security with Apache Ranger
[ https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438560#comment-15438560 ] Don Bosco Durai commented on HAWQ-256: -- I agree. We can disable grant/revoke on the HAWQ side it Ranger is enabled. I feel, those using Ranger will prefer to manage the policies from one place. Also, even if we support GRANT/REVOKE from the HAWQ SQL command, we shouldn't expect that the behavior will be same when Ranger is enabled. E.g. Currently, when you give delegated admin privilege to any user (e.g. user 1) for the resource, then user1 can give any access to other users (e.g. user2) for that resource regardless what permission user1 has. This addresses use case where you don't want Admin to read/write, but be able to manage permissions for others. > Integrate Security with Apache Ranger > - > > Key: HAWQ-256 > URL: https://issues.apache.org/jira/browse/HAWQ-256 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF, Security >Reporter: Michael Andre Pearce (IG) >Assignee: Lili Ma > Fix For: backlog > > Attachments: HAWQRangerSupportDesign.pdf > > > Integrate security with Apache Ranger for a unified Hadoop security solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-968) Incorrect free in url_close
[ https://issues.apache.org/jira/browse/HAWQ-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-968: --- Assignee: hongwu (was: Lei Chang) > Incorrect free in url_close > --- > > Key: HAWQ-968 > URL: https://issues.apache.org/jira/browse/HAWQ-968 > Project: Apache HAWQ > Issue Type: Bug > Components: External Tables >Affects Versions: backlog >Reporter: hongwu >Assignee: hongwu > Fix For: backlog > > > There is potential double free risk in url_close of url.c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-638) gpload bug using pip installed PyGreSQL
[ https://issues.apache.org/jira/browse/HAWQ-638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-638: --- Assignee: hongwu (was: Lei Chang) > gpload bug using pip installed PyGreSQL > --- > > Key: HAWQ-638 > URL: https://issues.apache.org/jira/browse/HAWQ-638 > Project: Apache HAWQ > Issue Type: Bug > Components: Command Line Tools >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > > Since greenplum's gpload is based on private patch upon PyGreSQL for internal > usage, and gpload.py is copied from greenplum incompletely, it will generate > error while using gpload tools. > Details: > self.db.notices() depends on the implementation of pg.DB.notices, which was > implemented internal in greenplum, it is wrong to use this attribute in > gpload tool of hawq. > Reference: > https://github.com/apache/incubator-hawq/blob/master/tools/bin/gpload.py#L704 > https://github.com/greenplum-db/gpdb/blob/master/gpMgmt/bin/pythonSrc/PyGreSQL-4.0/pgmodule.c#L2929 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-647) configure compilation flag bug
[ https://issues.apache.org/jira/browse/HAWQ-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-647: --- Assignee: hongwu (was: Lei Chang) > configure compilation flag bug > -- > > Key: HAWQ-647 > URL: https://issues.apache.org/jira/browse/HAWQ-647 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > > It is dangerous to use -O3 compilation flag in debug mode with -g together... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-707) Remove google test dependency from libhdfs3 and libyarn folder
[ https://issues.apache.org/jira/browse/HAWQ-707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-707: --- Assignee: hongwu (was: Lei Chang) > Remove google test dependency from libhdfs3 and libyarn folder > -- > > Key: HAWQ-707 > URL: https://issues.apache.org/jira/browse/HAWQ-707 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-734) refactor feature test build
[ https://issues.apache.org/jira/browse/HAWQ-734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-734: --- Assignee: hongwu (was: Jiali Yao) > refactor feature test build > --- > > Key: HAWQ-734 > URL: https://issues.apache.org/jira/browse/HAWQ-734 > Project: Apache HAWQ > Issue Type: Improvement > Components: Tests >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > > Refactor feature test build for general makefile which could avoid conflict > pushing pull requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-754) Open googletest code inside hawq for test usage
[ https://issues.apache.org/jira/browse/HAWQ-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-754: --- Assignee: hongwu (was: Lei Chang) > Open googletest code inside hawq for test usage > --- > > Key: HAWQ-754 > URL: https://issues.apache.org/jira/browse/HAWQ-754 > Project: Apache HAWQ > Issue Type: Test > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-721) New feature test skeleton
[ https://issues.apache.org/jira/browse/HAWQ-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-721: --- Assignee: hongwu (was: Lei Chang) > New feature test skeleton > - > > Key: HAWQ-721 > URL: https://issues.apache.org/jira/browse/HAWQ-721 > Project: Apache HAWQ > Issue Type: New Feature > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-735) Import thrift-0.9.1
[ https://issues.apache.org/jira/browse/HAWQ-735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-735: --- Assignee: hongwu (was: Lei Chang) > Import thrift-0.9.1 > --- > > Key: HAWQ-735 > URL: https://issues.apache.org/jira/browse/HAWQ-735 > Project: Apache HAWQ > Issue Type: Improvement > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > > Since apt-get do not have `libthrift-dev` source package, we need to import > thrift inside hawq and supply a flag: --with-thrift. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-772) Hotfix for HAWQ-760
[ https://issues.apache.org/jira/browse/HAWQ-772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-772: --- Assignee: hongwu (was: Lei Chang) > Hotfix for HAWQ-760 > --- > > Key: HAWQ-772 > URL: https://issues.apache.org/jira/browse/HAWQ-772 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > > HAWQ-760 code is based on old code base, which conflicts with HAWQ-768, the > new merged commit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-757) Hotfix feature-test building bug.
[ https://issues.apache.org/jira/browse/HAWQ-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-757: --- Assignee: hongwu (was: Lei Chang) > Hotfix feature-test building bug. > - > > Key: HAWQ-757 > URL: https://issues.apache.org/jira/browse/HAWQ-757 > Project: Apache HAWQ > Issue Type: Test > Components: Build >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.0.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1023) Incorrect usage of java.lang.String.replaceAll
[ https://issues.apache.org/jira/browse/HAWQ-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1023: - Fix Version/s: backlog > Incorrect usage of java.lang.String.replaceAll > -- > > Key: HAWQ-1023 > URL: https://issues.apache.org/jira/browse/HAWQ-1023 > Project: Apache HAWQ > Issue Type: Bug >Reporter: hongwu >Assignee: Lei Chang > Fix For: backlog > > > Incorrect usage for java.lang.String.replaceAll generate useless calls: > https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPath.java#L51 > https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPoint.java#L45 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1023) Incorrect usage of java.lang.String.replaceAll
[ https://issues.apache.org/jira/browse/HAWQ-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1023: - Priority: Minor (was: Major) > Incorrect usage of java.lang.String.replaceAll > -- > > Key: HAWQ-1023 > URL: https://issues.apache.org/jira/browse/HAWQ-1023 > Project: Apache HAWQ > Issue Type: Bug >Reporter: hongwu >Assignee: Lei Chang >Priority: Minor > Fix For: backlog > > > Incorrect usage for java.lang.String.replaceAll generate useless calls: > https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPath.java#L51 > https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPoint.java#L45 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1023) Incorrect usage of java.lang.String.replaceAll
[ https://issues.apache.org/jira/browse/HAWQ-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-1023: Assignee: hongwu (was: Lei Chang) > Incorrect usage of java.lang.String.replaceAll > -- > > Key: HAWQ-1023 > URL: https://issues.apache.org/jira/browse/HAWQ-1023 > Project: Apache HAWQ > Issue Type: Bug >Reporter: hongwu >Assignee: hongwu >Priority: Minor > Fix For: backlog > > > Incorrect usage for java.lang.String.replaceAll generate useless calls: > https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPath.java#L51 > https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPoint.java#L45 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #837: HAWQ-779 support pxf filter pushdwon at th...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/837#discussion_r76370526
--- Diff:
pxf/pxf-api/src/test/java/org/apache/hawq/pxf/api/FilterParserTest.java ---
@@ -215,6 +215,10 @@ public void parseColumnOnLeft() throws Exception {
filter = "a1c2o7";
op = Operation.HDOP_AND;
runParseOneOperation("this filter was build from HDOP_AND",
filter, op);
+
+filter = "a1c2o8";
+op = Operation.HDOP_LIKE;
+runParseOneOperation("this filter was build from HDOP_LIKE",
filter, op);
--- End diff --
build -> built
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/845#discussion_r76369938
--- Diff: src/backend/cdb/cdbdatalocality.c ---
@@ -3791,12 +3791,27 @@ run_allocation_algorithm(SplitAllocResult *result,
List *virtual_segments, Query
targetPolicy = GpPolicyFetch(CurrentMemoryContext, myrelid);
bool isRelationHash = is_relation_hash(targetPolicy);
+ int fileCountInRelation = list_length(rel_data->files);
+ bool FileCountBucketNumMismatch = false;
+ if (targetPolicy->bucketnum > 0) {
+ FileCountBucketNumMismatch = fileCountInRelation %
--- End diff --
why not `fileCountBucketNumMismatch = fileCountInRelation %
targetPolicy->bucketnum != 0`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/845#discussion_r76369662
--- Diff: src/backend/utils/misc/guc.c ---
@@ -3176,6 +3177,17 @@ static struct config_bool ConfigureNamesBool[] =
},
{
+ {"allow_file_count_bucket_num_mismatch", PGC_POSTMASTER,
CLIENT_CONN_LOCALE,
+ gettext_noop("allow hash table to be treated as random when
file count and"
--- End diff --
allow file count and bucket number mismatch by viewing hash distributed
table as randomly distributed table
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (HAWQ-256) Integrate Security with Apache Ranger
[ https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438530#comment-15438530 ] Lili Ma commented on HAWQ-256: -- Agree on the Grant/Revoke should be disabled is ranger is enabled. In this way, the with grant option will not be considered. Another thing is the owner management. Besides normal ACL, HAWQ has a definition of owner. The owner of the object can do any operation. And for the owner part, "Grant parent role to member role" and "reassign" are two SQL commands for owner control. I think we should move owner control to Ranger, to enable a fully Ranger-centralized access control. Your thoughts? [~vVineet][~bosco] > Integrate Security with Apache Ranger > - > > Key: HAWQ-256 > URL: https://issues.apache.org/jira/browse/HAWQ-256 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF, Security >Reporter: Michael Andre Pearce (IG) >Assignee: Lili Ma > Fix For: backlog > > Attachments: HAWQRangerSupportDesign.pdf > > > Integrate security with Apache Ranger for a unified Hadoop security solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-256) Integrate Security with Apache Ranger
[ https://issues.apache.org/jira/browse/HAWQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438531#comment-15438531 ] Lili Ma commented on HAWQ-256: -- [~bosco] Do you have any feedback on the API definition from Hortonworks side? Thanks > Integrate Security with Apache Ranger > - > > Key: HAWQ-256 > URL: https://issues.apache.org/jira/browse/HAWQ-256 > Project: Apache HAWQ > Issue Type: New Feature > Components: PXF, Security >Reporter: Michael Andre Pearce (IG) >Assignee: Lili Ma > Fix For: backlog > > Attachments: HAWQRangerSupportDesign.pdf > > > Integrate security with Apache Ranger for a unified Hadoop security solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/845#discussion_r76369259
--- Diff: src/backend/cdb/cdbdatalocality.c ---
@@ -3791,12 +3791,27 @@ run_allocation_algorithm(SplitAllocResult *result,
List *virtual_segments, Query
targetPolicy = GpPolicyFetch(CurrentMemoryContext, myrelid);
bool isRelationHash = is_relation_hash(targetPolicy);
+ int fileCountInRelation = list_length(rel_data->files);
+ bool FileCountBucketNumMismatch = false;
+ if (targetPolicy->bucketnum > 0) {
+ FileCountBucketNumMismatch = fileCountInRelation %
+ targetPolicy->bucketnum == 0 ? false : true;
+ }
+ if (FileCountBucketNumMismatch &&
!allow_file_count_bucket_num_mismatch) {
+ elog(ERROR, "file count %d in catalog is not in proportion to
the bucket "
+ "number %d of hash table with oid=%u, some data may be
lost, if you "
--- End diff --
number %d of distribution policy for table with oid = %u, some data may be
missing or invalid.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/845#discussion_r76369524 --- Diff: src/backend/utils/misc/guc.c --- @@ -450,6 +450,7 @@ char *Debug_dtm_action_protocol_str; /* Enable check for compatibility of encoding and locale in createdb */ bool gp_encoding_check_locale_compatibility; +bool allow_file_count_bucket_num_mismatch; --- End diff -- indent --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/845#discussion_r76369429
--- Diff: src/backend/cdb/cdbdatalocality.c ---
@@ -3791,12 +3791,27 @@ run_allocation_algorithm(SplitAllocResult *result,
List *virtual_segments, Query
targetPolicy = GpPolicyFetch(CurrentMemoryContext, myrelid);
bool isRelationHash = is_relation_hash(targetPolicy);
+ int fileCountInRelation = list_length(rel_data->files);
+ bool FileCountBucketNumMismatch = false;
+ if (targetPolicy->bucketnum > 0) {
+ FileCountBucketNumMismatch = fileCountInRelation %
+ targetPolicy->bucketnum == 0 ? false : true;
+ }
+ if (FileCountBucketNumMismatch &&
!allow_file_count_bucket_num_mismatch) {
+ elog(ERROR, "file count %d in catalog is not in proportion to
the bucket "
+ "number %d of hash table with oid=%u, some data may be
lost, if you "
+ "still want to continue the query by considering the
table as random, set GUC "
--- End diff --
You can force running the query by viewing the table as randomly
distributed table. set GUC allow_file_count_bucket_num_mismatch on to enable
this feature.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/845#discussion_r76368733 --- Diff: src/backend/cdb/cdbdatalocality.c --- @@ -3791,12 +3791,27 @@ run_allocation_algorithm(SplitAllocResult *result, List *virtual_segments, Query targetPolicy = GpPolicyFetch(CurrentMemoryContext, myrelid); bool isRelationHash = is_relation_hash(targetPolicy); + int fileCountInRelation = list_length(rel_data->files); + bool FileCountBucketNumMismatch = false; --- End diff -- why is this var capital leading? `FileCountBucketNumMismatch` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #837: HAWQ-779 support pxf filter pushdwon at th...
Github user jiadexin commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/837#discussion_r76368428 --- Diff: src/backend/optimizer/plan/createplan.c --- @@ -1146,7 +1146,7 @@ static char** create_pxf_plan(char **segdb_file_map, RelOptInfo *rel, int total_ Relation relation = RelationIdGetRelation(planner_rt_fetch(scan_relid, ctx->root)->relid); segdb_work_map = map_hddata_2gp_segments(uri_str, total_segs, segs_participating, - relation, NULL); + relation, ctx->root->parse->jointree->quals); --- End diff -- ok. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #845: HAWQ-999. Treat hash table as random when ...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/845#discussion_r76368378
--- Diff: src/backend/cdb/cdbdatalocality.c ---
@@ -3822,8 +3837,13 @@ run_allocation_algorithm(SplitAllocResult *result,
List *virtual_segments, Query
uint64_t before_run_allocate_hash_or_random =
gettime_microsec();
/*allocate hash relation*/
if (isRelationHash) {
- if (context->keep_hash &&
assignment_context.virtual_segment_num
- == targetPolicy->bucketnum) {
+ /*
+ * if file count of the table is not equal to or multiple of
+ * bucket number, we should process it as random table.
+ */
+ if (context->keep_hash
+ && assignment_context.virtual_segment_num==
targetPolicy->bucketnum
--- End diff --
space before ==
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #861: HAWQ-1001. Implement HAWQ user ACL check t...
Github user hsyuan commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/861#discussion_r76368047
--- Diff: src/backend/utils/misc/guc.c ---
@@ -4323,6 +4324,16 @@ static struct config_bool ConfigureNamesBool[] =
},
{
+{"enable_ranger", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
+ gettext_noop("support to using ranger to manage hawq privilege."),
--- End diff --
Enable Apache Ranger for HAWQ privilege management.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (HAWQ-984) hawq config is too slow.
[ https://issues.apache.org/jira/browse/HAWQ-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438486#comment-15438486 ] Paul Guo commented on HAWQ-984: --- 1) gpscp It still uses popen2.Popen4(). This function introduces millions of useless close() sys call finally in above test command. Using subprocess.Popen() without close_fds as an alternative easily resolves this. 2) gppylib/commands/base.py def __init__(self,name,pool,timeout=5): The worker thread will block at most 5 seconds in each loop (Queue.get()) to fetch potential commands even we have known that there will be no more commands to run for some threads. This really does not make sense since some idle threads will block for 5 seconds also before exiting. Setting timeout to zero will make python code spin. I tested a small timeout value e.g. 0.1s and it works fine. It seems that 0.1 is a good timeout candidate. > hawq config is too slow. > > > Key: HAWQ-984 > URL: https://issues.apache.org/jira/browse/HAWQ-984 > Project: Apache HAWQ > Issue Type: Bug > Components: Command Line Tools >Reporter: Paul Guo >Assignee: Lei Chang > Fix For: 2.0.1.0-incubating > > > I tried to set a simple guc value via "hawq config" on my centos > virtual system, but it spends >6 seconds. I know "hawq config" just > simple scp-update xml files to various nodes. This should have > been very fast if network situation is fine (My test hawq system is all-in-one > test environment so network latency is not an issue.) > I expect this is done on my test system (just one seg node) in < 1 second. > But actually the time is: > $ time hawq config -c lc_messages -v en_US.UTF-8 > GUC lc_messages already exist in hawq-site.xml > Update it with value: en_US.UTF-8 > GUC : lc_messages > Value : en_US.UTF-8 > real 0m6.363s > user 0m0.943s > sys 0m1.258s > I quickly monitored the processes when running "hawq config", > it seems that the scp command finishes very early and adding > print debug code in python file gpscp shows that the code in it > finishes also very early. > I'm not sure if anyone sees that long like my case (my system > is probably quite mangled after dev and test). I roughly traced > the python stack. I suspect there is something wrong in python > or we are not using related module well or we should replace modules. > I just checked "hawq config" but it is possible other CLI suffers from this > also. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-984) hawq config is too slow.
[ https://issues.apache.org/jira/browse/HAWQ-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Guo closed HAWQ-984. - Resolution: Fixed Assignee: Paul Guo (was: Lei Chang) Fix Version/s: (was: backlog) 2.0.1.0-incubating > hawq config is too slow. > > > Key: HAWQ-984 > URL: https://issues.apache.org/jira/browse/HAWQ-984 > Project: Apache HAWQ > Issue Type: Bug > Components: Command Line Tools >Reporter: Paul Guo >Assignee: Paul Guo > Fix For: 2.0.1.0-incubating > > > I tried to set a simple guc value via "hawq config" on my centos > virtual system, but it spends >6 seconds. I know "hawq config" just > simple scp-update xml files to various nodes. This should have > been very fast if network situation is fine (My test hawq system is all-in-one > test environment so network latency is not an issue.) > I expect this is done on my test system (just one seg node) in < 1 second. > But actually the time is: > $ time hawq config -c lc_messages -v en_US.UTF-8 > GUC lc_messages already exist in hawq-site.xml > Update it with value: en_US.UTF-8 > GUC : lc_messages > Value : en_US.UTF-8 > real 0m6.363s > user 0m0.943s > sys 0m1.258s > I quickly monitored the processes when running "hawq config", > it seems that the scp command finishes very early and adding > print debug code in python file gpscp shows that the code in it > finishes also very early. > I'm not sure if anyone sees that long like my case (my system > is probably quite mangled after dev and test). I roughly traced > the python stack. I suspect there is something wrong in python > or we are not using related module well or we should replace modules. > I just checked "hawq config" but it is possible other CLI suffers from this > also. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #863: HAWQ-1021. Need to log for some local_ssh functio...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/863 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #856: HAWQ-984. hawq config is too slow.
Github user paul-guo- closed the pull request at: https://github.com/apache/incubator-hawq/pull/856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #862: HAWQ-900. Add dependency in PL/R rpm build...
Github user paul-guo- closed the pull request at: https://github.com/apache/incubator-hawq/pull/862 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #856: HAWQ-984. hawq config is too slow.
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/856 Merged. Closing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #862: HAWQ-900. Add dependency in PL/R rpm build spec f...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/862 Merged. Closing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (HAWQ-1011) Check whether the table to be registered is existed
[ https://issues.apache.org/jira/browse/HAWQ-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu resolved HAWQ-1011. -- Resolution: Implemented > Check whether the table to be registered is existed > --- > > Key: HAWQ-1011 > URL: https://issues.apache.org/jira/browse/HAWQ-1011 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > > Check whether the table to be registered is existed or not. If it is, print a > error message and then exit. This is a feature for hawq register to make sure > the right name from user input. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #859: HAWQ-1011. Check whether the table to be r...
Github user xunzhang closed the pull request at: https://github.com/apache/incubator-hawq/pull/859 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #859: HAWQ-1011. Check whether the table to be register...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/859 Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (HAWQ-838) include an alternative for paramiko python module
[ https://issues.apache.org/jira/browse/HAWQ-838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radar Lei resolved HAWQ-838. Resolution: Fixed > include an alternative for paramiko python module > - > > Key: HAWQ-838 > URL: https://issues.apache.org/jira/browse/HAWQ-838 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Reporter: Radar Lei >Assignee: Radar Lei >Priority: Minor > Fix For: 2.0.1.0-incubating > > > Since paramiko's license is conflict with Apach HAWQ's license, so we can't > deliver it with HAWQ package. So users need to install it before installing > hawq. > To save user's time to install paramiko manually, we can find a good > replacement which don't have license conflict. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-1008) Remove legacy options for hawq command line tools
[ https://issues.apache.org/jira/browse/HAWQ-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radar Lei resolved HAWQ-1008. - Resolution: Fixed > Remove legacy options for hawq command line tools > - > > Key: HAWQ-1008 > URL: https://issues.apache.org/jira/browse/HAWQ-1008 > Project: Apache HAWQ > Issue Type: Task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: Radar Lei >Assignee: Radar Lei >Priority: Minor > Fix For: 2.0.1.0-incubating > > > There are some unused/legacy command line tool options, we should remove them. > See: > hawq state: > -d MASTER_DIR, --datadir=MASTER_DIR > HAWQ Master data directory. > --hawqhome=HAWQ_HOME HAWQ Master data directory. > hawq config: > -d HAWQ_HOME HAWQ home directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-1009) Remove requirement of environment value 'MASTER_DATA_DIRECTORY'
[ https://issues.apache.org/jira/browse/HAWQ-1009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radar Lei resolved HAWQ-1009. - Resolution: Fixed > Remove requirement of environment value 'MASTER_DATA_DIRECTORY' > --- > > Key: HAWQ-1009 > URL: https://issues.apache.org/jira/browse/HAWQ-1009 > Project: Apache HAWQ > Issue Type: Task >Reporter: Radar Lei >Assignee: Radar Lei > Fix For: 2.0.1.0-incubating > > > In old HAWQ command line tools, some tools will require > 'MASTER_DATA_DIRECTORY' environment to be set. > Since now we define 'MASTER_DATA_DIRECTORY' in hawq-site.xml, we do not need > to set it anymore. So we'd better remove these requirements to avoid user > confusion. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #852: HAWQ-1008. Remove legacy options for hawq ...
Github user radarwave closed the pull request at: https://github.com/apache/incubator-hawq/pull/852 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #859: HAWQ-1011. Check whether the table to be register...
Github user ictmalili commented on the issue: https://github.com/apache/incubator-hawq/pull/859 LGTM. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #859: HAWQ-1011. Check whether the table to be register...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/859 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #859: HAWQ-1011. Check whether the table to be register...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/859 also cc @radarwave --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #864: HAWQ-1009 Remove requirement of environmen...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/864 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #859: HAWQ-1011. Check whether the table to be r...
Github user ictmalili commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/859#discussion_r76351833
--- Diff: tools/bin/hawqregister ---
@@ -131,6 +130,18 @@ def option_parser_yml(yml_file):
def create_table(dburl, tablename, schema_info, fmt, distrbution_policy,
file_locations):
try:
+query = "select count(*) from pg_class where relname = '%s'" %
tablename.split('.')[-1]
--- End diff --
Maybe we need to consider the upper/lower cases of tablename. All the
table name stored in HAWQ are converted to lower case. So I think we need to
convert tablename to lower case and then do comparison.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Assigned] (HAWQ-1025) Check the consistency of AO/Parquet_FileLocations.Files.size attribute in extracted yaml file and the actual file size in HDFS.
[ https://issues.apache.org/jira/browse/HAWQ-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-1025: Assignee: hongwu (was: Lei Chang) > Check the consistency of AO/Parquet_FileLocations.Files.size attribute in > extracted yaml file and the actual file size in HDFS. > --- > > Key: HAWQ-1025 > URL: https://issues.apache.org/jira/browse/HAWQ-1025 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1025) Check the consistency of AO/Parquet_FileLocations.Files.size attribute in extracted yaml file and the actual file size in HDFS.
[ https://issues.apache.org/jira/browse/HAWQ-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1025: - Affects Version/s: 2.0.1.0-incubating > Check the consistency of AO/Parquet_FileLocations.Files.size attribute in > extracted yaml file and the actual file size in HDFS. > --- > > Key: HAWQ-1025 > URL: https://issues.apache.org/jira/browse/HAWQ-1025 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1025) Check the consistency of AO/Parquet_FileLocations.Files.size attribute in extracted yaml file and the actual file size in HDFS.
hongwu created HAWQ-1025: Summary: Check the consistency of AO/Parquet_FileLocations.Files.size attribute in extracted yaml file and the actual file size in HDFS. Key: HAWQ-1025 URL: https://issues.apache.org/jira/browse/HAWQ-1025 Project: Apache HAWQ Issue Type: Sub-task Components: Command Line Tools Reporter: hongwu Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1024) Rollback if hawq register failed in process
[ https://issues.apache.org/jira/browse/HAWQ-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-1024: Assignee: hongwu (was: Lei Chang) > Rollback if hawq register failed in process > --- > > Key: HAWQ-1024 > URL: https://issues.apache.org/jira/browse/HAWQ-1024 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1024) Rollback if hawq register failed in process
[ https://issues.apache.org/jira/browse/HAWQ-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438238#comment-15438238 ] hongwu commented on HAWQ-1024: -- In the second use of hawq register, if hawq register fail in process, we should drop the created table. > Rollback if hawq register failed in process > --- > > Key: HAWQ-1024 > URL: https://issues.apache.org/jira/browse/HAWQ-1024 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1024) Rollback if hawq register failed in process
[ https://issues.apache.org/jira/browse/HAWQ-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1024: - Affects Version/s: 2.0.1.0-incubating > Rollback if hawq register failed in process > --- > > Key: HAWQ-1024 > URL: https://issues.apache.org/jira/browse/HAWQ-1024 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1023) Incorrect usage of java.lang.String.replaceAll
hongwu created HAWQ-1023: Summary: Incorrect usage of java.lang.String.replaceAll Key: HAWQ-1023 URL: https://issues.apache.org/jira/browse/HAWQ-1023 Project: Apache HAWQ Issue Type: Bug Reporter: hongwu Assignee: Lei Chang Incorrect usage for java.lang.String.replaceAll generate useless calls: https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPath.java#L51 https://github.com/apache/incubator-hawq/blob/master/contrib/hawq-hadoop/hawq-mapreduce-common/src/main/java/com/pivotal/hawq/mapreduce/datatype/HAWQPoint.java#L45 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1024) Rollback if hawq register failed in process
hongwu created HAWQ-1024: Summary: Rollback if hawq register failed in process Key: HAWQ-1024 URL: https://issues.apache.org/jira/browse/HAWQ-1024 Project: Apache HAWQ Issue Type: Sub-task Components: Command Line Tools Reporter: hongwu Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #866: HAWQ-1013. Add Copyright to hawq-ambari-pl...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/866 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #866: HAWQ-1013. Add Copyright to hawq-ambari-pl...
GitHub user mithmatt opened a pull request: https://github.com/apache/incubator-hawq/pull/866 HAWQ-1013. Add Copyright to hawq-ambari-plugin You can merge this pull request into a Git repository by running: $ git pull https://github.com/mithmatt/incubator-hawq add-copyright-field Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/866.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #866 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt closed the pull request at: https://github.com/apache/incubator-hawq/pull/857 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/857#discussion_r76317991 --- Diff: contrib/hawq-ambari-plugin/build.properties --- @@ -0,0 +1,8 @@ +hawq.release.version=2.0.1 +hawq.common.services.version=2.0.0 +pxf.release.version=3.0.1 +pxf.common.services.version=3.0.0 +hawq.repo.prefix=hawq +hawq.addons.repo.prefix=hawq-add-ons +repository.version=2.0.1.0 +default.stack=HDP-2.4 --- End diff -- Nothing to do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76317914
--- Diff: contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py
---
@@ -0,0 +1,498 @@
+#!/usr/bin/env python
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import base64
+import getpass
+import json
+import os
+import shutil
+import socket
+import urllib2
+import xml.etree.ElementTree as ET
+from optparse import OptionParser
+
+DEFAULT_STACK = '${default.stack}'
+SUPPORTED_OS_LIST = ['redhat6']
+HAWQ_LIB_STAGING_DIR = '${hawq.lib.staging.dir}'
+REPO_VERSION = '${repository.version}'
+HAWQ_REPO = '${hawq.repo.prefix}'
+HAWQ_ADD_ONS_REPO = '${hawq.addons.repo.prefix}'
+
+REPO_INFO = {
+ HAWQ_REPO: {
+'repoid': '-'.join([HAWQ_REPO, REPO_VERSION]),
+'input_param': '--hawqrepo',
+'optional': False
+ },
+ HAWQ_ADD_ONS_REPO: {
+'repoid': '-'.join([HAWQ_ADD_ONS_REPO, REPO_VERSION]),
+'input_param': '--addonsrepo',
+'optional': True
+ }
+}
+
+
+class APIClient:
+ """
+ Class which interacts with Ambari Server API
+ """
+
+ # Base API URL points to localhost. This script is to be executed on the
Ambari Server
+ BASE_API_URL = 'http://localhost:8080/api/v1'
+
+ def __init__(self, user, password):
+self.user = user
+self.password = password
+self.encoded_credentials = base64.encodestring(self.user + ':' +
self.password).replace('\n', '')
+
+ def __request(self, method, url_path, headers=None, data=None):
+"""
+Creates API requests and packages response into the following format:
(response code, response body in json object)
+"""
+headers = headers if headers is not None else {}
+headers['Authorization'] = 'Basic {0}'.format(self.encoded_credentials)
+
+req = urllib2.Request(self.BASE_API_URL + url_path, data, headers)
+req.get_method = lambda: method
+response = urllib2.urlopen(req)
+response_str = response.read()
+
+return response.getcode(), json.loads(response_str) if response_str
else None
+
+ def verify_api_reachable(self):
+"""
+Returns true if Ambari Server is reachable through API
+"""
+try:
+ status_code, _ = self.__request('GET', '/stacks')
+except Exception as e:
+ if type(e) == urllib2.HTTPError and e.code == 403:
+raise Exception('Invalid username and/or password.')
+ elif type(e) == urllib2.URLError:
+raise Exception('Ambari-server is not running. Please start
ambari-server.')
+ else:
+raise Exception('Unable to connect to Ambari Server.\n' + str(e))
+
+ def get_cluster_name(self):
+"""
+Returns the name of the installed cluster
+"""
+_, response_json = self.__request('GET', '/clusters')
+return None if len(response_json['items']) == 0 else
response_json['items'][0]['Clusters']['cluster_name']
+
+ def get_stack_info(self, cluster_name):
+"""
+Returns stack information (stack name, stack version, repository
version) of stack installed on cluster
+"""
+_, response_json = self.__request('GET',
+
'/clusters/{0}/stack_versions?ClusterStackVersions/state.matches(CURRENT)'.format(
+ cluster_name))
+if 'items' not in response_json or len(response_json['items']) == 0:
+ raise Exception('No Stack found to be installed on the cluster
{0}'.format(cluster_name))
+stack_versions = response_json['items'][0]['ClusterStackVersions']
+return stack_versions['stack'], stack_versions['version'],
stack_versions['repository_version']
+
+ def get_existing_repository_info(self, stack_name, stack_version,
repository_version):
+"""
+Returns existing repo information for a given stack
+"""
+
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76317948
--- Diff: contrib/hawq-ambari-plugin/pom.xml ---
@@ -0,0 +1,125 @@
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd";>
+
+ 4.0.0
+ org.apache.hawq
+ hawq-ambari-plugin
+ 2.0.1
+ hawq-ambari-plugin
+ http://maven.apache.org
+
+
+1.0.0
+UTF-8
+${basedir}/target/resources-temp
+/var/lib/hawq
+${hawq.lib.dir}/staging
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-resources-plugin
+3.0.1
+
+
+
+none
+
+ copy-resources
+
+
+
+
+
+
+${basedir}/build.properties
+
+ ${resources.temp}
+
+
+ ${basedir}/src/main/resources/
+ true
+
+
+
+
+
+
+org.codehaus.mojo
+rpm-maven-plugin
+2.1.4
+
+
+
+none
+
+ rpm
+
+
+
+
+
+ org.apache.hawq
+
+HAWQ plugin contains Ambari's stack definition for HAWQ.
+When installed, Ambari will be able to support HAWQ as a
service.
+
+ x86_64
+ ${build_number}%{?dist}
+
+ambari-server >= 2.2
+python >= 2.6
+
+
+
+ ${hawq.lib.staging.dir}
+
+
+ ${resources.temp}/services
+
+
+
+
+ ${hawq.lib.dir}
+ 755
+
+
+ ${resources.temp}/utils
+
+
+
+
+
+
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/857#discussion_r76317925 --- Diff: contrib/hawq-ambari-plugin/README.md --- @@ -0,0 +1,97 @@ +# HAWQ Ambari Plugin + +hawq-ambari-plugin helps users install HAWQ and PXF using Ambari. + +To ensure that Ambari recognizes HAWQ and PXF as services that can be installed for a specific stack, the following manual steps are required: + * Add HAWQ and PXF metainfo.xml files (containing metadata about the service) under the stack to be installed. + * Add repositories where HAWQ and PXF rpms reside, so that Ambari can use it during installation. This requires updating repoinfo.xml under the stack HAWQ and PXF is to be added. + * If a stack is already installed using Ambari, add repositories to the existing stack using the Ambari REST API. + + +## Source Code +Source code directory structure of the hawq-ambari-plugin is as follows: +``` +hawq-ambari-plugin ++-- README.md ++-- build.properties ++-- pom.xml ++-- src ++-- main ++-- resources ++-- services +¦ +-- HAWQ +¦ ¦ +-- metainfo.xml +¦ +-- PXF +¦ +-- metainfo.xml ++-- utils ++-- add-hawq.py +``` + +### build.properties +[build.properties](build.properties) contains properties required for building the plugin. + +### metainfo.xml +[metainfo.xml](src/main/resources/services/HAWQ/metainfo.xml) contains the metadata about the service. The metainfo.xml specifies that the service definition is to be inherited from Ambari common-services. HAWQ and PXF common-services code can be found under [Apache Ambari repository](https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/common-services/). + +### add-hawq.py +[add-hawq.py](src/main/resources/utils/add-hawq.py) deploys HAWQ and PXF metainfo.xml files under the stack and adds the repositories to Ambari. + + +## Building the plugin +***Build Environment***: centos6 is the typical operating system used for building. + +Properties specified in the [build.properties](build.properties) file: + +| Property | Description | Value | +| --- | --- | --- | +| hawq.release.version | Release version of HAWQ | 2.0.1 | +| hawq.common.services.version | HAWQ common services code in Ambari to be inherited | 2.0.0 | +| pxf.release.version | Release version of PXF | 3.0.1 | +| pxf.common.services.version | PXF common services code in Ambari to be inherited | 3.0.0 | +| hawq.repo.prefix | Repository name for HAWQ core repository | hawq | +| hawq.addons.repo.prefix | Repository name for HAWQ Add Ons repository | hawq-add-ons | +| repository.version | Repository Version to be used in repository information | 2.0.1.0 | +| default.stack | Default stack under which, metainfo.xml and repositories have to be added | HDP-2.4 | + +To build the rpm for hawq-ambari-plugin, change the [build.properties](build.properties) file with the required parameters and run ```mvn install``` command under hawq-ambari-plugin directory: +``` +$ pwd +incubator-hawq/contrib/hawq-ambari-plugin +$ mvn install --- End diff -- Done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #839: HAWQ-983. Fix bug that minirepro generates wrong ...
Github user oarap commented on the issue: https://github.com/apache/incubator-hawq/pull/839 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #838: HAWQ-934. Populate canSetTag of PlannedStmt from ...
Github user oarap commented on the issue: https://github.com/apache/incubator-hawq/pull/838 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #857: HAWQ-1013. Move HAWQ Ambari plugin to Apache HAWQ
Github user vVineet commented on the issue: https://github.com/apache/incubator-hawq/pull/857 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/857#discussion_r76301165 --- Diff: contrib/hawq-ambari-plugin/build.properties --- @@ -0,0 +1,8 @@ +hawq.release.version=2.0.1 +hawq.common.services.version=2.0.0 +pxf.release.version=3.0.1 +pxf.common.services.version=3.0.0 +hawq.repo.prefix=hawq +hawq.addons.repo.prefix=hawq-add-ons +repository.version=2.0.1.0 +default.stack=HDP-2.4 --- End diff -- Keeping build.properties for ease of use with CI builds --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76301092
--- Diff: contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py
---
@@ -0,0 +1,498 @@
+#!/usr/bin/env python
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import base64
+import getpass
+import json
+import os
+import shutil
+import socket
+import urllib2
+import xml.etree.ElementTree as ET
+from optparse import OptionParser
+
+DEFAULT_STACK = '${default.stack}'
+SUPPORTED_OS_LIST = ['redhat6']
+HAWQ_LIB_STAGING_DIR = '${hawq.lib.staging.dir}'
+REPO_VERSION = '${repository.version}'
+HAWQ_REPO = '${hawq.repo.prefix}'
+HAWQ_ADD_ONS_REPO = '${hawq.addons.repo.prefix}'
+
+REPO_INFO = {
+ HAWQ_REPO: {
+'repoid': '-'.join([HAWQ_REPO, REPO_VERSION]),
+'input_param': '--hawqrepo',
+'optional': False
+ },
+ HAWQ_ADD_ONS_REPO: {
+'repoid': '-'.join([HAWQ_ADD_ONS_REPO, REPO_VERSION]),
+'input_param': '--addonsrepo',
+'optional': True
+ }
+}
+
+
+class APIClient:
+ """
+ Class which interacts with Ambari Server API
+ """
+
+ # Base API URL points to localhost. This script is to be executed on the
Ambari Server
+ BASE_API_URL = 'http://localhost:8080/api/v1'
+
+ def __init__(self, user, password):
+self.user = user
+self.password = password
+self.encoded_credentials = base64.encodestring(self.user + ':' +
self.password).replace('\n', '')
+
+ def __request(self, method, url_path, headers=None, data=None):
+"""
+Creates API requests and packages response into the following format:
(response code, response body in json object)
+"""
+headers = headers if headers is not None else {}
+headers['Authorization'] = 'Basic {0}'.format(self.encoded_credentials)
+
+req = urllib2.Request(self.BASE_API_URL + url_path, data, headers)
+req.get_method = lambda: method
+response = urllib2.urlopen(req)
+response_str = response.read()
+
+return response.getcode(), json.loads(response_str) if response_str
else None
+
+ def verify_api_reachable(self):
+"""
+Returns true if Ambari Server is reachable through API
+"""
+try:
+ status_code, _ = self.__request('GET', '/stacks')
+except Exception as e:
+ if type(e) == urllib2.HTTPError and e.code == 403:
+raise Exception('Invalid username and/or password.')
+ elif type(e) == urllib2.URLError:
+raise Exception('Ambari-server is not running. Please start
ambari-server.')
+ else:
+raise Exception('Unable to connect to Ambari Server.\n' + str(e))
+
+ def get_cluster_name(self):
+"""
+Returns the name of the installed cluster
+"""
+_, response_json = self.__request('GET', '/clusters')
+return None if len(response_json['items']) == 0 else
response_json['items'][0]['Clusters']['cluster_name']
+
+ def get_stack_info(self, cluster_name):
+"""
+Returns stack information (stack name, stack version, repository
version) of stack installed on cluster
+"""
+_, response_json = self.__request('GET',
+
'/clusters/{0}/stack_versions?ClusterStackVersions/state.matches(CURRENT)'.format(
+ cluster_name))
+if 'items' not in response_json or len(response_json['items']) == 0:
+ raise Exception('No Stack found to be installed on the cluster
{0}'.format(cluster_name))
+stack_versions = response_json['items'][0]['ClusterStackVersions']
+return stack_versions['stack'], stack_versions['version'],
stack_versions['repository_version']
+
+ def get_existing_repository_info(self, stack_name, stack_version,
repository_version):
+"""
+Returns existing repo information for a given stack
+"""
+
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76289077
--- Diff: contrib/hawq-ambari-plugin/pom.xml ---
@@ -0,0 +1,125 @@
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd";>
+
+ 4.0.0
+ org.apache.hawq
+ hawq-ambari-plugin
+ 2.0.1
+ hawq-ambari-plugin
+ http://maven.apache.org
+
+
+1.0.0
+UTF-8
+${basedir}/target/resources-temp
+/var/lib/hawq
+${hawq.lib.dir}/staging
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-resources-plugin
+3.0.1
+
+
+
+none
+
+ copy-resources
+
+
+
+
+
+
+${basedir}/build.properties
+
+ ${resources.temp}
+
+
+ ${basedir}/src/main/resources/
+ true
+
+
+
+
+
+
+org.codehaus.mojo
+rpm-maven-plugin
+2.1.4
+
+
+
+none
+
+ rpm
+
+
+
+
+
+ org.apache.hawq
+
+HAWQ plugin contains Ambari's stack definition for HAWQ.
+When installed, Ambari will be able to support HAWQ as a
service.
+
+ x86_64
+ ${build_number}%{?dist}
+
+ambari-server >= 2.2
+python >= 2.6
+
+
+
+ ${hawq.lib.staging.dir}
+
+
+ ${resources.temp}/services
+
+
+
+
+ ${hawq.lib.dir}
+ 755
+
+
+ ${resources.temp}/utils
+
+
+
+
+
+
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user mithmatt commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76288743
--- Diff: contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py
---
@@ -0,0 +1,498 @@
+#!/usr/bin/env python
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import base64
+import getpass
+import json
+import os
+import shutil
+import socket
+import urllib2
+import xml.etree.ElementTree as ET
+from optparse import OptionParser
+
+DEFAULT_STACK = '${default.stack}'
+SUPPORTED_OS_LIST = ['redhat6']
+HAWQ_LIB_STAGING_DIR = '${hawq.lib.staging.dir}'
+REPO_VERSION = '${repository.version}'
+HAWQ_REPO = '${hawq.repo.prefix}'
+HAWQ_ADD_ONS_REPO = '${hawq.addons.repo.prefix}'
+
+REPO_INFO = {
+ HAWQ_REPO: {
+'repoid': '-'.join([HAWQ_REPO, REPO_VERSION]),
+'input_param': '--hawqrepo',
+'optional': False
+ },
+ HAWQ_ADD_ONS_REPO: {
+'repoid': '-'.join([HAWQ_ADD_ONS_REPO, REPO_VERSION]),
+'input_param': '--addonsrepo',
+'optional': True
+ }
+}
+
+
+class APIClient:
+ """
+ Class which interacts with Ambari Server API
+ """
+
+ # Base API URL points to localhost. This script is to be executed on the
Ambari Server
+ BASE_API_URL = 'http://localhost:8080/api/v1'
+
+ def __init__(self, user, password):
+self.user = user
+self.password = password
+self.encoded_credentials = base64.encodestring(self.user + ':' +
self.password).replace('\n', '')
+
+ def __request(self, method, url_path, headers=None, data=None):
+"""
+Creates API requests and packages response into the following format:
(response code, response body in json object)
+"""
+headers = headers if headers is not None else {}
+headers['Authorization'] = 'Basic {0}'.format(self.encoded_credentials)
+
+req = urllib2.Request(self.BASE_API_URL + url_path, data, headers)
+req.get_method = lambda: method
+response = urllib2.urlopen(req)
+response_str = response.read()
+
+return response.getcode(), json.loads(response_str) if response_str
else None
+
+ def verify_api_reachable(self):
+"""
+Returns true if Ambari Server is reachable through API
+"""
+try:
+ status_code, _ = self.__request('GET', '/stacks')
+except Exception as e:
+ if type(e) == urllib2.HTTPError and e.code == 403:
+raise Exception('Invalid username and/or password.')
+ elif type(e) == urllib2.URLError:
+raise Exception('Ambari-server is not running. Please start
ambari-server.')
+ else:
+raise Exception('Unable to connect to Ambari Server.\n' + str(e))
+
+ def get_cluster_name(self):
+"""
+Returns the name of the installed cluster
+"""
+_, response_json = self.__request('GET', '/clusters')
+return None if len(response_json['items']) == 0 else
response_json['items'][0]['Clusters']['cluster_name']
+
+ def get_stack_info(self, cluster_name):
+"""
+Returns stack information (stack name, stack version, repository
version) of stack installed on cluster
+"""
+_, response_json = self.__request('GET',
+
'/clusters/{0}/stack_versions?ClusterStackVersions/state.matches(CURRENT)'.format(
+ cluster_name))
+if 'items' not in response_json or len(response_json['items']) == 0:
+ raise Exception('No Stack found to be installed on the cluster
{0}'.format(cluster_name))
+stack_versions = response_json['items'][0]['ClusterStackVersions']
+return stack_versions['stack'], stack_versions['version'],
stack_versions['repository_version']
+
+ def get_existing_repository_info(self, stack_name, stack_version,
repository_version):
+"""
+Returns existing repo information for a given stack
+"""
+
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user ljainpivotalio commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76278294
--- Diff: contrib/hawq-ambari-plugin/src/main/resources/utils/add-hawq.py
---
@@ -0,0 +1,498 @@
+#!/usr/bin/env python
+
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+"""
+
+import base64
+import getpass
+import json
+import os
+import shutil
+import socket
+import urllib2
+import xml.etree.ElementTree as ET
+from optparse import OptionParser
+
+DEFAULT_STACK = '${default.stack}'
+SUPPORTED_OS_LIST = ['redhat6']
+HAWQ_LIB_STAGING_DIR = '${hawq.lib.staging.dir}'
+REPO_VERSION = '${repository.version}'
+HAWQ_REPO = '${hawq.repo.prefix}'
+HAWQ_ADD_ONS_REPO = '${hawq.addons.repo.prefix}'
+
+REPO_INFO = {
+ HAWQ_REPO: {
+'repoid': '-'.join([HAWQ_REPO, REPO_VERSION]),
+'input_param': '--hawqrepo',
+'optional': False
+ },
+ HAWQ_ADD_ONS_REPO: {
+'repoid': '-'.join([HAWQ_ADD_ONS_REPO, REPO_VERSION]),
+'input_param': '--addonsrepo',
+'optional': True
+ }
+}
+
+
+class APIClient:
+ """
+ Class which interacts with Ambari Server API
+ """
+
+ # Base API URL points to localhost. This script is to be executed on the
Ambari Server
+ BASE_API_URL = 'http://localhost:8080/api/v1'
+
+ def __init__(self, user, password):
+self.user = user
+self.password = password
+self.encoded_credentials = base64.encodestring(self.user + ':' +
self.password).replace('\n', '')
+
+ def __request(self, method, url_path, headers=None, data=None):
+"""
+Creates API requests and packages response into the following format:
(response code, response body in json object)
+"""
+headers = headers if headers is not None else {}
+headers['Authorization'] = 'Basic {0}'.format(self.encoded_credentials)
+
+req = urllib2.Request(self.BASE_API_URL + url_path, data, headers)
+req.get_method = lambda: method
+response = urllib2.urlopen(req)
+response_str = response.read()
+
+return response.getcode(), json.loads(response_str) if response_str
else None
+
+ def verify_api_reachable(self):
+"""
+Returns true if Ambari Server is reachable through API
+"""
+try:
+ status_code, _ = self.__request('GET', '/stacks')
+except Exception as e:
+ if type(e) == urllib2.HTTPError and e.code == 403:
+raise Exception('Invalid username and/or password.')
+ elif type(e) == urllib2.URLError:
+raise Exception('Ambari-server is not running. Please start
ambari-server.')
+ else:
+raise Exception('Unable to connect to Ambari Server.\n' + str(e))
+
+ def get_cluster_name(self):
+"""
+Returns the name of the installed cluster
+"""
+_, response_json = self.__request('GET', '/clusters')
+return None if len(response_json['items']) == 0 else
response_json['items'][0]['Clusters']['cluster_name']
+
+ def get_stack_info(self, cluster_name):
+"""
+Returns stack information (stack name, stack version, repository
version) of stack installed on cluster
+"""
+_, response_json = self.__request('GET',
+
'/clusters/{0}/stack_versions?ClusterStackVersions/state.matches(CURRENT)'.format(
+ cluster_name))
+if 'items' not in response_json or len(response_json['items']) == 0:
+ raise Exception('No Stack found to be installed on the cluster
{0}'.format(cluster_name))
+stack_versions = response_json['items'][0]['ClusterStackVersions']
+return stack_versions['stack'], stack_versions['version'],
stack_versions['repository_version']
+
+ def get_existing_repository_info(self, stack_name, stack_version,
repository_version):
+"""
+Returns existing repo information for a given stack
+"""
[GitHub] incubator-hawq pull request #857: HAWQ-1013. Move HAWQ Ambari plugin to Apac...
Github user ljainpivotalio commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/857#discussion_r76278102
--- Diff: contrib/hawq-ambari-plugin/pom.xml ---
@@ -0,0 +1,125 @@
+
+
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd";>
+
+ 4.0.0
+ org.apache.hawq
+ hawq-ambari-plugin
+ 2.0.1
+ hawq-ambari-plugin
+ http://maven.apache.org
+
+
+1.0.0
+UTF-8
+${basedir}/target/resources-temp
+/var/lib/hawq
+${hawq.lib.dir}/staging
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-resources-plugin
+3.0.1
+
+
+
+none
+
+ copy-resources
+
+
+
+
+
+
+${basedir}/build.properties
+
+ ${resources.temp}
+
+
+ ${basedir}/src/main/resources/
+ true
+
+
+
+
+
+
+org.codehaus.mojo
+rpm-maven-plugin
+2.1.4
+
+
+
+none
+
+ rpm
+
+
+
+
+
+ org.apache.hawq
+
+HAWQ plugin contains Ambari's stack definition for HAWQ.
+When installed, Ambari will be able to support HAWQ as a
service.
+
+ x86_64
+ ${build_number}%{?dist}
+
+ambari-server >= 2.2
+python >= 2.6
+
+
+
+ ${hawq.lib.staging.dir}
+
+
+ ${resources.temp}/services
+
+
+
+
+ ${hawq.lib.dir}
+ 755
+
+
+ ${resources.temp}/utils
+
+
+
+
+
+
[jira] [Resolved] (HAWQ-1022) Fix Command::getCommandOutput bug: using reference to temporary variable in stack.
[ https://issues.apache.org/jira/browse/HAWQ-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu resolved HAWQ-1022. -- Resolution: Fixed > Fix Command::getCommandOutput bug: using reference to temporary variable in > stack. > -- > > Key: HAWQ-1022 > URL: https://issues.apache.org/jira/browse/HAWQ-1022 > Project: Apache HAWQ > Issue Type: Bug >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq pull request #865: HAWQ-1022. Bugfix for getCommandOutput: us...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-hawq/pull/865 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #865: HAWQ-1022. Bugfix for getCommandOutput: using ref...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/865 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (HAWQ-1022) Fix Command::getCommandOutput bug: using reference to temporary variable in stack.
[ https://issues.apache.org/jira/browse/HAWQ-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1022: - Fix Version/s: 2.0.1.0-incubating > Fix Command::getCommandOutput bug: using reference to temporary variable in > stack. > -- > > Key: HAWQ-1022 > URL: https://issues.apache.org/jira/browse/HAWQ-1022 > Project: Apache HAWQ > Issue Type: Bug >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1022) Fix Command::getCommandOutput bug: using reference to temporary variable in stack.
[ https://issues.apache.org/jira/browse/HAWQ-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu updated HAWQ-1022: - Affects Version/s: 2.0.1.0-incubating > Fix Command::getCommandOutput bug: using reference to temporary variable in > stack. > -- > > Key: HAWQ-1022 > URL: https://issues.apache.org/jira/browse/HAWQ-1022 > Project: Apache HAWQ > Issue Type: Bug >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1011) Check whether the table to be registered is existed
[ https://issues.apache.org/jira/browse/HAWQ-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436683#comment-15436683 ] hongwu commented on HAWQ-1011: -- [~kdunn926], good advice! I think in our first implementation(v2.0.1.0-incubating), for simplicity, the default behaviour is fine. --force or --repair will be implemented as future options for hawq register. I will create another JIRA for that improvement in the future. Thanks again for the advice. > Check whether the table to be registered is existed > --- > > Key: HAWQ-1011 > URL: https://issues.apache.org/jira/browse/HAWQ-1011 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Command Line Tools >Affects Versions: 2.0.1.0-incubating >Reporter: hongwu >Assignee: hongwu > Fix For: 2.0.1.0-incubating > > > Check whether the table to be registered is existed or not. If it is, print a > error message and then exit. This is a feature for hawq register to make sure > the right name from user input. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #865: HAWQ-1022. Bugfix for getCommandOutput: using ref...
Github user wcl14 commented on the issue: https://github.com/apache/incubator-hawq/pull/865 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #865: HAWQ-1022. Bugfix for getCommandOutput: using ref...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/865 also cc @wcl14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #865: HAWQ-1022. Bugfix for getCommandOutput: us...
GitHub user xunzhang opened a pull request: https://github.com/apache/incubator-hawq/pull/865 HAWQ-1022. Bugfix for getCommandOutput: using reference to temporary variable in stack address. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xunzhang/incubator-hawq HAWQ-1022 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/865.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #865 commit 35e7aa43ab9bd37fd194c6eb81821dc37c19319e Author: xunzhang Date: 2016-08-25T10:24:52Z HAWQ-1022. Bugfix for getCommandOutput: using reference to temporary variable in stack address. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #865: HAWQ-1022. Bugfix for getCommandOutput: using ref...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/865 cc @huor @paul-guo- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (HAWQ-1022) Fix Command::getCommandOutput bug: using reference to temporary variable in stack.
[ https://issues.apache.org/jira/browse/HAWQ-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-1022: Assignee: hongwu (was: Lei Chang) > Fix Command::getCommandOutput bug: using reference to temporary variable in > stack. > -- > > Key: HAWQ-1022 > URL: https://issues.apache.org/jira/browse/HAWQ-1022 > Project: Apache HAWQ > Issue Type: Bug >Reporter: hongwu >Assignee: hongwu > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1021) Need to log for some local_ssh function calls.
[ https://issues.apache.org/jira/browse/HAWQ-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongwu reassigned HAWQ-1021: Assignee: hongwu (was: Lei Chang) > Need to log for some local_ssh function calls. > -- > > Key: HAWQ-1021 > URL: https://issues.apache.org/jira/browse/HAWQ-1021 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Paul Guo >Assignee: hongwu > > HAWQ management tools calls a lot of local_ssh() function to run some > external commands. The function is defined in hawqpylib/hawqlib.py. Many > callers does not set logger so we do not know any details about the command > running process. This is annoying when users/developers fail to run some > related commands and want to know the root cause quickly. > Besides, there is two definitions of local_ssh(). Although they are not in > the same namespace, it is kind of annoying. We need to rename either one or > even both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1022) Fix Command::getCommandOutput bug: using reference to temporary variable in stack.
hongwu created HAWQ-1022: Summary: Fix Command::getCommandOutput bug: using reference to temporary variable in stack. Key: HAWQ-1022 URL: https://issues.apache.org/jira/browse/HAWQ-1022 Project: Apache HAWQ Issue Type: Bug Reporter: hongwu Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-969) Add getting configuration from HDFS and YARN
[ https://issues.apache.org/jira/browse/HAWQ-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang closed HAWQ-969. -- > Add getting configuration from HDFS and YARN > > > Key: HAWQ-969 > URL: https://issues.apache.org/jira/browse/HAWQ-969 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Tests >Reporter: Chunling Wang >Assignee: Chunling Wang > Fix For: 2.0.1.0-incubating > > > Add getting configuration from HDFS and YARN and writing xml file in > xml_parser.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (HAWQ-1020) Fix bugs to let feature tests TestCommonLib.TestHdfsConfig and TestCommonLib.TestYanConfig run in concourse
[ https://issues.apache.org/jira/browse/HAWQ-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang closed HAWQ-1020. --- > Fix bugs to let feature tests TestCommonLib.TestHdfsConfig and > TestCommonLib.TestYanConfig run in concourse > --- > > Key: HAWQ-1020 > URL: https://issues.apache.org/jira/browse/HAWQ-1020 > Project: Apache HAWQ > Issue Type: Bug > Components: Tests >Affects Versions: 2.0.1.0-incubating >Reporter: Chunling Wang >Assignee: Chunling Wang > Fix For: 2.0.1.0-incubating > > > Fix bugs to let feature tests TestCommonLib.TestHdfsConfig and > TestCommonLib.TestYanConfig run in concourse -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HAWQ-969) Add getting configuration from HDFS and YARN
[ https://issues.apache.org/jira/browse/HAWQ-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang resolved HAWQ-969. Resolution: Fixed > Add getting configuration from HDFS and YARN > > > Key: HAWQ-969 > URL: https://issues.apache.org/jira/browse/HAWQ-969 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Tests >Reporter: Chunling Wang >Assignee: Chunling Wang > Fix For: 2.0.1.0-incubating > > > Add getting configuration from HDFS and YARN and writing xml file in > xml_parser.cpp. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #864: HAWQ-1009 Remove requirement of environment value...
Github user xunzhang commented on the issue: https://github.com/apache/incubator-hawq/pull/864 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (HAWQ-1020) Fix bugs to let feature tests TestCommonLib.TestHdfsConfig and TestCommonLib.TestYanConfig run in concourse
[ https://issues.apache.org/jira/browse/HAWQ-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunling Wang resolved HAWQ-1020. - Resolution: Fixed Fix Version/s: 2.0.1.0-incubating > Fix bugs to let feature tests TestCommonLib.TestHdfsConfig and > TestCommonLib.TestYanConfig run in concourse > --- > > Key: HAWQ-1020 > URL: https://issues.apache.org/jira/browse/HAWQ-1020 > Project: Apache HAWQ > Issue Type: Bug > Components: Tests >Affects Versions: 2.0.1.0-incubating >Reporter: Chunling Wang >Assignee: Chunling Wang > Fix For: 2.0.1.0-incubating > > > Fix bugs to let feature tests TestCommonLib.TestHdfsConfig and > TestCommonLib.TestYanConfig run in concourse -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #863: HAWQ-1021. Need to log for some local_ssh functio...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/863 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #863: HAWQ-1021. Need to log for some local_ssh functio...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/863 @xunzhang @radarwave --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #807: HAWQ-897. Add feature test for create tabl...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/807#discussion_r76211338
--- Diff: src/test/feature/catalog/test_create_table.cpp ---
@@ -22,26 +22,26 @@ class TestCreateTable : public ::testing::Test {
TEST_F(TestCreateTable, TestCreateTable1) {
hawq::test::SQLUtility util;
// prepare
- util.execute("DROP TABLE IF EXISTS aggtest");
--- End diff --
I think jinyi's intent is to test "create/drop table".
Actually the constructor and destructor of class hawq::test::SQLUtility are
exactly doing what you've suggested.
See src/test/feature/lib/sql_util.cpp for details.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #864: HAWQ-1009 Remove requirement of environment value...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/864 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #861: HAWQ-1001. Implement HAWQ user ACL check t...
Github user paul-guo- commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/861#discussion_r76209892 --- Diff: src/backend/utils/misc/guc.c --- @@ -349,6 +349,7 @@ bool Debug_datumstream_write_use_small_initial_buffers = false; bool gp_temporary_files_filespace_repair = false; bool filesystem_support_truncate = true; bool gp_allow_non_uniform_partitioning_ddl = true; +bool enable_ranger = false; --- End diff -- ident --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #861: HAWQ-1001. Implement HAWQ user ACL check t...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/861#discussion_r76209911
--- Diff: src/backend/utils/misc/guc.c ---
@@ -4323,6 +4324,16 @@ static struct config_bool ConfigureNamesBool[] =
},
{
+{"enable_ranger", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
+ gettext_noop("support to using ranger to manage hawq privilege."),
--- End diff --
s/using/use/
s/ranger/Apache Ranger/ ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #864: HAWQ-1009 Remove requirement of environmen...
GitHub user radarwave opened a pull request: https://github.com/apache/incubator-hawq/pull/864 HAWQ-1009 Remove requirement of environment value MASTER_DATA_DIRECTORY You can merge this pull request into a Git repository by running: $ git pull https://github.com/radarwave/incubator-hawq mdd Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/864.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #864 commit 1d37e9d3f69ae1cff596ed348ba885cfaca971a8 Author: rlei Date: 2016-08-25T09:13:36Z HAWQ-1009 Remove requirement of environment value MASTER_DATA_DIRECTORY --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #861: HAWQ-1001. Implement HAWQ user ACL check through ...
Github user paul-guo- commented on the issue: https://github.com/apache/incubator-hawq/pull/861 Whether need a test case for this code change? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #863: HAWQ-1021. Need to log for some local_ssh ...
GitHub user paul-guo- opened a pull request: https://github.com/apache/incubator-hawq/pull/863 HAWQ-1021. Need to log for some local_ssh function calls. You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-guo-/incubator-hawq python2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/863.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #863 commit 4cf7458785b7c10a2681ecbce300821f60e2bdb9 Author: Paul Guo Date: 2016-08-25T09:05:38Z HAWQ-1021. Need to log for some local_ssh function calls. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (HAWQ-1021) Need to log for some local_ssh function calls.
Paul Guo created HAWQ-1021: -- Summary: Need to log for some local_ssh function calls. Key: HAWQ-1021 URL: https://issues.apache.org/jira/browse/HAWQ-1021 Project: Apache HAWQ Issue Type: Bug Reporter: Paul Guo Assignee: Lei Chang HAWQ management tools calls a lot of local_ssh() function to run some external commands. The function is defined in hawqpylib/hawqlib.py. Many callers does not set logger so we do not know any details about the command running process. This is annoying when users/developers fail to run some related commands and want to know the root cause quickly. Besides, there is two definitions of local_ssh(). Although they are not in the same namespace, it is kind of annoying. We need to rename either one or even both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #862: HAWQ-900. Add dependency in PL/R rpm build spec f...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/862 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #862: HAWQ-900. Add dependency in PL/R rpm build spec f...
Github user radarwave commented on the issue: https://github.com/apache/incubator-hawq/pull/862 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #862: HAWQ-900. Add dependency in PL/R rpm build...
GitHub user paul-guo- opened a pull request: https://github.com/apache/incubator-hawq/pull/862 HAWQ-900. Add dependency in PL/R rpm build spec file plr.spec To build, r-core-devel is enough (no need of r-devel). You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-guo-/incubator-hawq plr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/862.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #862 commit ffe8e486e55adc7a35f93d28e342e1f4bdbeaa99 Author: Paul Guo Date: 2016-07-06T10:09:42Z HAWQ-900. Add dependency in PL/R rpm build spec file plr.spec --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
