[GitHub] incubator-hawq pull request #1234: HAWQ-1436. Implement ranger pulgin servic...
Github user linwen closed the pull request at: https://github.com/apache/incubator-hawq/pull/1234 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1234: HAWQ-1436. Implement ranger pulgin servic...
Github user stanlyxiang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1234#discussion_r115410444
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -383,58 +383,100 @@ static size_t write_callback(char *contents, size_t
size, size_t nitems,
static int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
int ret = -1;
+ int retry = 2;
CURLcode res;
Assert(request != NULL);
/*
-* Re-initializes all options previously set on a specified CURL handle
-* to the default values. This puts back the handle to the same state as
-* it was in when it was just created with curl_easy_init.It does not
-* change the following information kept in the handle: live
connections,
-* the Session ID cache, the DNS cache, the cookies and shares.
+* If master is talking with standby RPS, for every predefined interval
+* (controlled by a GUC hawq_rps_check_local_interval) it will check if
local RPS works now.
*/
- curl_easy_reset(curl_handle->curl_handle);
- /* timeout: hard-coded temporarily and maybe should be a guc in future
*/
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
-
- /* specify URL to get */
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://";);
- appendStringInfo(&tname, "%s", master_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/");
- appendStringInfo(&tname, "%s", "rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
- pfree(tname.data);
-
- struct curl_slist *headers = NULL;
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION,
write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void
*)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
- if(request_id == INT_MAX)
+ if (curl_handle->talkingWithStandby)
{
- request_id = 0;
- }
- request_id++;
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(ERROR, "ranger plugin service from http://%s:%d/rps is
unavailable : %s.\n",
- master_addr_host, rps_addr_port,
curl_easy_strerror(res));
+ uint64_t current_time = gettime_microsec();
+ if ((current_time - curl_handle->lastCheckTimestamp) >
100LL * rps_check_local_interval)
+ {
+ curl_handle->talkingWithStandby = false;
+ curl_handle->lastCheckTimestamp = 0;
+ elog(RANGER_LOG,
+ "master has been talking to standby RPS
for a predefined period, try switching to master RPS");
+ }
}
- else
+
+ /*
+* try to connect standby's RPS if fail in connecting master's RPS
+*/
+ while(retry > 0 && ret != 0)
{
- ret = 0;
- elog(RANGER_LOG, "retrieved %d bytes data from ranger restful
response.",
- curl_handle->response.response_size);
+ /*
+* Re-initializes all options previously set on a specified
CURL handle
+* to the default values. This puts back the handle to the same
state as
+* it was in when it was just created with curl_easy_init.It
does not
+* change the following information kept in the handle: live
connections,
+* the Session ID cache, the DNS cache, the cookies and shares.
+*/
+ curl_easy_reset(curl_handle->curl_handle);
+ /* timeout: hard-coded temporarily and maybe should be a guc in
future */
+ curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT,
30L);
+
+ /* specify URL to get */
+ StringInfoData tname;
+ initStringInfo(&tname);
+ appendStringInfo(&tname, "http://";);
+ appendStringInfo(&tname, "%s",
curl_handle->talkingWithStandby?standby_addr_host:master_addr_host);
+ appendStringInfo(&tname, ":");
+ appendStringInfo(&tname, "%d", rps_addr_port);
+ appendStringInfo(&tname, "/");
+ appendStringInfo(&tname, "%s", "rps");
--- End diff --
These 2 lines can squash to 1 line . "appendStringInfo(&tname, "/rps");"
---
If your project is
[GitHub] incubator-hawq pull request #1234: HAWQ-1436. Implement ranger pulgin servic...
Github user stanlyxiang commented on a diff in the pull request:
https://github.com/apache/incubator-hawq/pull/1234#discussion_r115410368
--- Diff: src/backend/libpq/rangerrest.c ---
@@ -383,58 +383,100 @@ static size_t write_callback(char *contents, size_t
size, size_t nitems,
static int call_ranger_rest(CURL_HANDLE curl_handle, const char* request)
{
int ret = -1;
+ int retry = 2;
CURLcode res;
Assert(request != NULL);
/*
-* Re-initializes all options previously set on a specified CURL handle
-* to the default values. This puts back the handle to the same state as
-* it was in when it was just created with curl_easy_init.It does not
-* change the following information kept in the handle: live
connections,
-* the Session ID cache, the DNS cache, the cookies and shares.
+* If master is talking with standby RPS, for every predefined interval
+* (controlled by a GUC hawq_rps_check_local_interval) it will check if
local RPS works now.
*/
- curl_easy_reset(curl_handle->curl_handle);
- /* timeout: hard-coded temporarily and maybe should be a guc in future
*/
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_TIMEOUT, 30L);
-
- /* specify URL to get */
- StringInfoData tname;
- initStringInfo(&tname);
- appendStringInfo(&tname, "http://";);
- appendStringInfo(&tname, "%s", master_addr_host);
- appendStringInfo(&tname, ":");
- appendStringInfo(&tname, "%d", rps_addr_port);
- appendStringInfo(&tname, "/");
- appendStringInfo(&tname, "%s", "rps");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_URL, tname.data);
- pfree(tname.data);
-
- struct curl_slist *headers = NULL;
- headers = curl_slist_append(headers, "Content-Type:application/json");
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_HTTPHEADER, headers);
-
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_POSTFIELDS,request);
- /* send all data to this function */
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEFUNCTION,
write_callback);
- curl_easy_setopt(curl_handle->curl_handle, CURLOPT_WRITEDATA, (void
*)curl_handle);
-
- res = curl_easy_perform(curl_handle->curl_handle);
- if(request_id == INT_MAX)
+ if (curl_handle->talkingWithStandby)
{
- request_id = 0;
- }
- request_id++;
- /* check for errors */
- if(res != CURLE_OK)
- {
- elog(ERROR, "ranger plugin service from http://%s:%d/rps is
unavailable : %s.\n",
- master_addr_host, rps_addr_port,
curl_easy_strerror(res));
+ uint64_t current_time = gettime_microsec();
+ if ((current_time - curl_handle->lastCheckTimestamp) >
100LL * rps_check_local_interval)
+ {
+ curl_handle->talkingWithStandby = false;
+ curl_handle->lastCheckTimestamp = 0;
+ elog(RANGER_LOG,
+ "master has been talking to standby RPS
for a predefined period, try switching to master RPS");
--- End diff --
we could print the rps_check_local_interval time instead of "predefined
period".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq pull request #1234: HAWQ-1436. Implement ranger pulgin servic...
GitHub user linwen opened a pull request: https://github.com/apache/incubator-hawq/pull/1234 HAWQ-1436. Implement ranger pulgin service High Availability. 1. master will the connect to standby RPS for policy search if RPS on master failed; 2. if master has been talking to standby RPS for a period(controlled by GUC hawq_rps_check_local_interval, 5 minutes by default), it will try to connect local RPS again. Please review, thanks! You can merge this pull request into a Git repository by running: $ git pull https://github.com/linwen/incubator-hawq hawq_1436 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1234.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1234 commit baf088d7187de4c757c7d44e7b174615c84303b1 Author: Wen Lin Date: 2017-05-09T03:36:13Z HAWQ-1436. Implement ranger pulgin service High Availability. 1. master will the connect to standby RPS for policy search if RPS on master failed; 2. if master has been talking to standby RPS for a period(controlled by GUC hawq_rps_check_local_interval, 5 minutes by default), it will try to connect local RPS again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
