[GitHub] incubator-hawq pull request #1237: HAWQ-1459. Tweak the feature test related...
Github user paul-guo- closed the pull request at: https://github.com/apache/incubator-hawq/pull/1237 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1246: HAWQ-1475. Add LICENSE, NOTICE, and DISCLAIMER f...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/1246 @rvs, @edespino, @paul-guo- @radarwave, please review and let us know your feedback. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1246: HAWQ-1475. Add LICENSE, NOTICE, and DISCL...
GitHub user huor opened a pull request: https://github.com/apache/incubator-hawq/pull/1246 HAWQ-1475. Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release This commit includes: 1. The LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ c/c++ components for the binary release. 2. The build process to add the LICENSE, NOTICE, and DISCLAIMER files in Apache HAWQ binary package. You can merge this pull request into a Git repository by running: $ git pull https://github.com/huor/incubator-hawq license Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1246.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1246 commit e62a5934f7f9927efe86833fe49ec293a7487f13 Author: Ruilong Huo Date: 2017-05-27T07:41:35Z HAWQ-1475. Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1245: HAWQ-1476. Augment enable-ranger-plugin.s...
GitHub user zhangh43 opened a pull request: https://github.com/apache/incubator-hawq/pull/1245 HAWQ-1476. Augment enable-ranger-plugin.sh to support kerberos. Now ranger can lookup hawq resource in kerberized environment. So we also need to change enable-ranger-plugin.sh to support automatically fill the authentication type and hawq kerberos service name fields You can merge this pull request into a Git repository by running: $ git pull https://github.com/zhangh43/incubator-hawq hawq1476 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1245.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1245 commit fd79c3e3322e2d9cf8a58ce755a1d8a0 Author: hubertzhang Date: 2017-05-27T09:40:51Z HAWQ-1476. Augment enable-ranger-plugin.sh to support kerberos. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (HAWQ-1476) Augment enable-ranger-plugin.sh to support kerberos.
[ https://issues.apache.org/jira/browse/HAWQ-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hubert Zhang reassigned HAWQ-1476: -- Assignee: Hubert Zhang (was: Ed Espino) > Augment enable-ranger-plugin.sh to support kerberos. > > > Key: HAWQ-1476 > URL: https://issues.apache.org/jira/browse/HAWQ-1476 > Project: Apache HAWQ > Issue Type: Sub-task > Components: Security >Reporter: Hubert Zhang >Assignee: Hubert Zhang > Fix For: backlog > > > Now ranger can lookup hawq resource in kerberized environment. So we also > need to change enable-ranger-plugin.sh to automatically fill the > authentication type and hawq kerberos service name fields -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HAWQ-1476) Augment enable-ranger-plugin.sh to support kerberos.
Hubert Zhang created HAWQ-1476: -- Summary: Augment enable-ranger-plugin.sh to support kerberos. Key: HAWQ-1476 URL: https://issues.apache.org/jira/browse/HAWQ-1476 Project: Apache HAWQ Issue Type: Sub-task Reporter: Hubert Zhang Assignee: Ed Espino Now ranger can lookup hawq resource in kerberized environment. So we also need to change enable-ranger-plugin.sh to automatically fill the authentication type and hawq kerberos service name fields -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (HAWQ-1475) Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release
[ https://issues.apache.org/jira/browse/HAWQ-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo reassigned HAWQ-1475: - Assignee: Ruilong Huo (was: Ed Espino) > Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release > > > Key: HAWQ-1475 > URL: https://issues.apache.org/jira/browse/HAWQ-1475 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Ruilong Huo >Assignee: Ruilong Huo > Fix For: 2.2.0.0-incubating > > > Per review from IPMC vote for Apache HAWQ 2.2.0.0-incubating binary release, > we need to add LICENSE, NOTICE, and DISCLAIMER files for the binary release. > Refer to > https://lists.apache.org/thread.html/13a66f0f5762166f37354ec2cdc7e82f531b74650be16239f5f738ce@%3Cdev.hawq.apache.org%3E > for detail. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HAWQ-1475) Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release
[ https://issues.apache.org/jira/browse/HAWQ-1475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruilong Huo updated HAWQ-1475: -- Description: Per review from IPMC vote for Apache HAWQ 2.2.0.0-incubating binary release, we need to add LICENSE, NOTICE, and DISCLAIMER files for the binary release. Refer to https://lists.apache.org/thread.html/13a66f0f5762166f37354ec2cdc7e82f531b74650be16239f5f738ce@%3Cdev.hawq.apache.org%3E for detail. was:Per review from IPMC vote for Apache HAWQ 2.2.0.0-incubating binary release, we need to add LICENSE, NOTICE, and DISCLAIMER files for the binary release. Refer to https://lists.apache.org/thread.html/13a66f0f5762166f37354ec2cdc7e82f531b74650be16239f5f738ce@%3Cdev.hawq.apache.org%3E for detail. > Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release > > > Key: HAWQ-1475 > URL: https://issues.apache.org/jira/browse/HAWQ-1475 > Project: Apache HAWQ > Issue Type: Bug > Components: Build >Reporter: Ruilong Huo >Assignee: Ed Espino > Fix For: 2.2.0.0-incubating > > > Per review from IPMC vote for Apache HAWQ 2.2.0.0-incubating binary release, > we need to add LICENSE, NOTICE, and DISCLAIMER files for the binary release. > Refer to > https://lists.apache.org/thread.html/13a66f0f5762166f37354ec2cdc7e82f531b74650be16239f5f738ce@%3Cdev.hawq.apache.org%3E > for detail. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1243: HAWQ-1458. Fix share input scan bug for w...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1243#discussion_r118815751
--- Diff: src/backend/utils/misc/guc.c ---
@@ -6685,6 +6687,15 @@ static struct config_int ConfigureNamesInt[] =
&metadata_cache_max_hdfs_file_num,
524288, 32768, 8388608, NULL, NULL
},
+ {
+ {"share_input_scan_wait_lockfile_timeout", PGC_USERSET,
DEVELOPER_OPTIONS,
+ gettext_noop("timeout for waiting lock file which
writer creates."),
+ NULL
+ },
--- End diff --
So this is ms or second?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1243: HAWQ-1458. Fix share input scan bug for w...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1243#discussion_r118815742
--- Diff: src/backend/executor/nodeShareInputScan.c ---
@@ -793,15 +877,78 @@ shareinput_reader_waitready(int share_id,
PlanGenerator planGen)
}
else if(n==0)
{
- elog(DEBUG1, "SISC READER (shareid=%d, slice=%d): Wait
ready time out once",
- share_id, currentSliceId);
+ file_exists = access(writer_lock_file, F_OK);
+ if(file_exists != 0)
+ {
+ elog(DEBUG3, "Wait lock file for writer time
out interval is %d", timeout_interval);
+ if(timeout_interval >=
share_input_scan_wait_lockfile_timeout || flag == true) //If lock file never
exists or disappeared, reader will no longer waiting for writer
+ {
+ elog(LOG, "SISC READER (shareid=%d,
slice=%d): Wait ready time out and break",
+ share_id, currentSliceId);
+ pfree(writer_lock_file);
+ break;
+ }
+ timeout_interval += tval.tv_sec;
--- End diff --
Why you ignore tv_usec?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Created] (HAWQ-1475) Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release
Ruilong Huo created HAWQ-1475: - Summary: Add LICENSE, NOTICE, and DISCLAIMER files for Apache HAWQ binary release Key: HAWQ-1475 URL: https://issues.apache.org/jira/browse/HAWQ-1475 Project: Apache HAWQ Issue Type: Bug Components: Build Reporter: Ruilong Huo Assignee: Ed Espino Fix For: 2.2.0.0-incubating Per review from IPMC vote for Apache HAWQ 2.2.0.0-incubating binary release, we need to add LICENSE, NOTICE, and DISCLAIMER files for the binary release. Refer to https://lists.apache.org/thread.html/13a66f0f5762166f37354ec2cdc7e82f531b74650be16239f5f738ce@%3Cdev.hawq.apache.org%3E for detail. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] incubator-hawq pull request #1243: HAWQ-1458. Fix share input scan bug for w...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1243#discussion_r118815361
--- Diff: src/backend/executor/nodeShareInputScan.c ---
@@ -793,15 +877,78 @@ shareinput_reader_waitready(int share_id,
PlanGenerator planGen)
}
else if(n==0)
{
- elog(DEBUG1, "SISC READER (shareid=%d, slice=%d): Wait
ready time out once",
- share_id, currentSliceId);
+ file_exists = access(writer_lock_file, F_OK);
+ if(file_exists != 0)
+ {
+ elog(DEBUG3, "Wait lock file for writer time
out interval is %d", timeout_interval);
+ if(timeout_interval >=
share_input_scan_wait_lockfile_timeout || flag == true) //If lock file never
exists or disappeared, reader will no longer waiting for writer
+ {
+ elog(LOG, "SISC READER (shareid=%d,
slice=%d): Wait ready time out and break",
+ share_id, currentSliceId);
+ pfree(writer_lock_file);
+ break;
+ }
+ timeout_interval += tval.tv_sec;
+ }
+ else
+ {
+ elog(LOG, "writer lock file of
shareinput_reader_waitready() is %s", writer_lock_file);
+ flag = true;
+ lock_fd = open(writer_lock_file, O_RDONLY);
+ if(lock_fd < 0)
+ {
+ elog(DEBUG3, "Open writer's lock file
%s failed!, error number is %d", writer_lock_file, errno);
+ continue;
+ }
+ lock = flock(lock_fd, LOCK_EX | LOCK_NB);
+ if(lock == -1)
+ {
+ /*
+* Reader try to lock the lock file
which writer created until locked the lock file successfully
+ * which means that writer process quit. If reader
lock the lock file failed, it means that writer
+* process is healthy.
+*/
+ elog(DEBUG3, "Lock writer's lock file
%s failed!, error number is %d", writer_lock_file, errno);
+ }
+ else if(lock == 0)
+ {
+ /*
+* There is one situation to consider
about.
+* Writer need a time interval to lock
the lock file after the lock file has been created.
+* So, if reader lock the lock file
ahead of writer, we should unlock it.
+* If reader lock the lock file after
writer, it means that writer process has abort.
+* We should break the loop to make
sure reader no longer wait for writer.
+*/
+ if(is_lock_firsttime == true)
+ {
+ lock = flock(lock_fd, LOCK_UN);
+ is_lock_firsttime = false;
+ elog(DEBUG3, "Lock writer's
lock file %s first time successfully in SISC! Unlock it.", writer_lock_file);
+ continue;
+ }
+ else
+ {
+ elog(LOG, "Lock writer's lock
file %s successfully in SISC!", writer_lock_file);
+ /* Retry to close the fd in
case there is interruption from signal */
+ while ((close(lock_fd) < 0) &&
(errno == EINTR))
--- End diff --
This is a legal condition. Should not elog(ERROR).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1243: HAWQ-1458. Fix share input scan bug for w...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1243#discussion_r118815310
--- Diff: src/backend/executor/nodeShareInputScan.c ---
@@ -666,6 +717,29 @@ static int retry_write(int fd, char *buf, int wsize)
return 0;
}
+
+
+/*
+ * generate_lock_file_name
+ *
+ * Called by reader or writer to make the unique lock file name.
+ */
+void generate_lock_file_name(char* p, int size, int share_id, const char*
name)
+{
+ if (strncmp(name , "writer", strlen("writer")) == 0)
+ {
+ sisc_lockname(p, size, share_id, "ready");
+ strncat(p, name, sizeof(p) - strlen(p) - 1);
+ }
+ else
+ {
+ sisc_lockname(p, size, share_id, "done");
+ strncat(p, name, sizeof(p) - strlen(p) - 1);
--- End diff --
same as above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1243: HAWQ-1458. Fix share input scan bug for w...
Github user paul-guo- commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1243#discussion_r118815308
--- Diff: src/backend/executor/nodeShareInputScan.c ---
@@ -666,6 +717,29 @@ static int retry_write(int fd, char *buf, int wsize)
return 0;
}
+
+
+/*
+ * generate_lock_file_name
+ *
+ * Called by reader or writer to make the unique lock file name.
+ */
+void generate_lock_file_name(char* p, int size, int share_id, const char*
name)
+{
+ if (strncmp(name , "writer", strlen("writer")) == 0)
+ {
+ sisc_lockname(p, size, share_id, "ready");
+ strncat(p, name, sizeof(p) - strlen(p) - 1);
--- End diff --
bug here. sizeof(p) = 8 (for x64), and strlen(name)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
