[GitHub] incubator-hawq pull request #1030: HAWQ-1171. Add UDF check before register
GitHub user linwen opened a pull request: https://github.com/apache/incubator-hawq/pull/1030 HAWQ-1171. Add UDF check before register Please review, thanks! You can merge this pull request into a Git repository by running: $ git pull https://github.com/linwen/incubator-hawq HAWQ-1171 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1030.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1030 commit 25ba6169379fb7ae337a0a44640d93b3c345fe34 Author: Wen Lin Date: 2016-11-29T07:49:52Z HAWQ-1171. Add UDF check before register --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1029: HAWQ-1174. double type core counter of container...
Github user huor commented on the issue: https://github.com/apache/incubator-hawq/pull/1029 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1029: HAWQ-1174. double type core counter of container...
Github user zhangh43 commented on the issue: https://github.com/apache/incubator-hawq/pull/1029 got it, +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1029: HAWQ-1174. double type core counter of container...
Github user jiny2 commented on the issue:
https://github.com/apache/incubator-hawq/pull/1029
@zhangh43 YARN mode, 2 segments, each segment has 75gb mem, 15core
resource, and the default queue has 25% capacity, 80% maximum capacity. Then
allocated and return 12 x 256mb vsegs, then wait more than 5 min. RM will try
to return yarn containers but due to precision problem, there are always some
yarn containers cannot be returned. because 0.99... is less than 1. As
to EPSILON, I adopted previous definition introduce by Wen. I think it should
work appropriately.
(gdb) p *containerset
$6 = {Allocated = {MemoryMB = 5120, Core = 1, Ratio = 0}, Available =
{MemoryMB = 5120,
Core = 0.99956, Ratio = 0}, Containers = 0x7ff07a93c870}
(gdb) quit
A debugging session is active.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hawq issue #1029: HAWQ-1174. double type core counter of container...
Github user zhangh43 commented on the issue: https://github.com/apache/incubator-hawq/pull/1029 +1, Yi could you please tell us the failure case before? I think the failure case can help us to determine the EPSILON --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1029: HAWQ-1174. double type core counter of container...
Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/1029 @huor please review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1029: HAWQ-1174. double type core counter of co...
Github user jiny2 commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1029#discussion_r89943290 --- Diff: src/backend/resourcemanager/include/resourcepool.h --- @@ -715,6 +715,7 @@ SimpStringPtr build_segment_status_description(SegStat segstat); #define EPSILON 1e-7 #define IS_DOUBLE_ZERO(d) (fabs(d) < EPSILON) #define IS_DOUBLE_EQ(x, y) ((fabs((x) - (y))) <= (EPSILON)) +#define IS_DOUBLE_EQGREATER(x, y) (((x) + EPSILON) >= (y)) --- End diff -- Fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1029: HAWQ-1174. double type core counter of co...
Github user linwen commented on a diff in the pull request: https://github.com/apache/incubator-hawq/pull/1029#discussion_r89940729 --- Diff: src/backend/resourcemanager/include/resourcepool.h --- @@ -715,6 +715,7 @@ SimpStringPtr build_segment_status_description(SegStat segstat); #define EPSILON 1e-7 #define IS_DOUBLE_ZERO(d) (fabs(d) < EPSILON) #define IS_DOUBLE_EQ(x, y) ((fabs((x) - (y))) <= (EPSILON)) +#define IS_DOUBLE_EQGREATER(x, y) (((x) + EPSILON) >= (y)) --- End diff -- or IS_DOUBLE_GE (greater than or equal to) some examples of abbreviation for comparison operator: http://folk.uio.no/hpl/scripting/doc/f77/tutorial/logical.html https://cn.perlmaven.com/comparing-scalars-in-perl http://www.computerperformance.co.uk/powershell/powershell_comparison_operators.htm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #1029: HAWQ-1174. double type core counter of container...
Github user jiny2 commented on the issue: https://github.com/apache/incubator-hawq/pull/1029 @linwen @ictmalili please help to review and comment, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1029: HAWQ-1174. double type core counter of co...
GitHub user jiny2 opened a pull request: https://github.com/apache/incubator-hawq/pull/1029 HAWQ-1174. double type core counter of container set has precision issue This fix is to avoid double precision problem when counting available core resource for a container set. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jiny2/incubator-hawq HAWQ-1174 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq/pull/1029.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1029 commit 2fb2c15641ab56c6edf5bc5c0bd59c6234e920d2 Author: Yi Date: 2016-11-28T06:04:26Z HAWQ-1174. double type core counter of container set has precision issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Github user kdunn926 commented on the issue: https://github.com/apache/incubator-hawq/pull/700 @liming01, agreed, performance will not be optimal without indexes. I believe there is value in this PR regardless, until [HAWQ-303](https://issues.apache.org/jira/browse/HAWQ-303) is implemented. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq pull request #1026: HAWQ-1171. Support upgrade for hawq regis...
Github user zhangh43 closed the pull request at: https://github.com/apache/incubator-hawq/pull/1026 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Github user liming01 commented on the issue: https://github.com/apache/incubator-hawq/pull/700 @kdunn926, if spatial index can not support, then the query performance will be very slow. What we can offer will be far away from user's expectation. Need to think a workaround. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (HAWQ-1179) Call Bridge api with profile value read from Fragmenter call
[ https://issues.apache.org/jira/browse/HAWQ-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko reassigned HAWQ-1179: - Assignee: Oleksandr Diachenko (was: Lei Chang) > Call Bridge api with profile value read from Fragmenter call > > > Key: HAWQ-1179 > URL: https://issues.apache.org/jira/browse/HAWQ-1179 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Oleksandr Diachenko > Fix For: backlog > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1179) Call Bridge api with profile value read from Fragmenter call
Oleksandr Diachenko created HAWQ-1179: - Summary: Call Bridge api with profile value read from Fragmenter call Key: HAWQ-1179 URL: https://issues.apache.org/jira/browse/HAWQ-1179 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Oleksandr Diachenko Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1178) Enhance Fragmenter api to return profile name
[ https://issues.apache.org/jira/browse/HAWQ-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko reassigned HAWQ-1178: - Assignee: Oleksandr Diachenko (was: Lei Chang) > Enhance Fragmenter api to return profile name > - > > Key: HAWQ-1178 > URL: https://issues.apache.org/jira/browse/HAWQ-1178 > Project: Apache HAWQ > Issue Type: Sub-task > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Oleksandr Diachenko > Fix For: backlog > > > Add "profile" field into PXFFragments object. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1178) Enhance Fragmenter api to return profile name
Oleksandr Diachenko created HAWQ-1178: - Summary: Enhance Fragmenter api to return profile name Key: HAWQ-1178 URL: https://issues.apache.org/jira/browse/HAWQ-1178 Project: Apache HAWQ Issue Type: Sub-task Components: PXF Reporter: Oleksandr Diachenko Assignee: Lei Chang Add "profile" field into PXFFragments object. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HAWQ-1177) Use profile based on file format in HCatalog integration
[ https://issues.apache.org/jira/browse/HAWQ-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko reassigned HAWQ-1177: - Assignee: Oleksandr Diachenko (was: Lei Chang) > Use profile based on file format in HCatalog integration > > > Key: HAWQ-1177 > URL: https://issues.apache.org/jira/browse/HAWQ-1177 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Oleksandr Diachenko > Fix For: backlog > > > When user queries Hive table using HCatalog integration feature, HAWQ uses > "Hive" profile, which might be not optimal. Idea is to choose profile for > each fragment, based of underlying file format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1177) Use profile based on file format in HCatalog integration
[ https://issues.apache.org/jira/browse/HAWQ-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko updated HAWQ-1177: -- Description: When user queries Hive table using HCatalog integration feature, HAWQ uses "Hive" profile, which might be not optimal. Idea is to choose profile for each fragment, based of underlying file format. > Use profile based on file format in HCatalog integration > > > Key: HAWQ-1177 > URL: https://issues.apache.org/jira/browse/HAWQ-1177 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Lei Chang > Fix For: backlog > > > When user queries Hive table using HCatalog integration feature, HAWQ uses > "Hive" profile, which might be not optimal. Idea is to choose profile for > each fragment, based of underlying file format. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1177) Use profile based on file format in HCatalog integration
Oleksandr Diachenko created HAWQ-1177: - Summary: Use profile based on file format in HCatalog integration Key: HAWQ-1177 URL: https://issues.apache.org/jira/browse/HAWQ-1177 Project: Apache HAWQ Issue Type: Improvement Components: PXF Reporter: Oleksandr Diachenko Assignee: Lei Chang -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HAWQ-1177) Use profile based on file format in HCatalog integration
[ https://issues.apache.org/jira/browse/HAWQ-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksandr Diachenko updated HAWQ-1177: -- Fix Version/s: backlog > Use profile based on file format in HCatalog integration > > > Key: HAWQ-1177 > URL: https://issues.apache.org/jira/browse/HAWQ-1177 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF >Reporter: Oleksandr Diachenko >Assignee: Lei Chang > Fix For: backlog > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1175) Correct hawq scp syntax description for -J option
[ https://issues.apache.org/jira/browse/HAWQ-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703370#comment-15703370 ] ASF GitHub Bot commented on HAWQ-1175: -- GitHub user janebeckman opened a pull request: https://github.com/apache/incubator-hawq-docs/pull/66 Feature/sqlsyntaxcaps Syntax description correction for hawq scp: https://issues.apache.org/jira/browse/HAWQ-1175 Consistent capitalization of PSQL commands. Parameters consistently use command font. Grammar and clarity edits. You can merge this pull request into a Git repository by running: $ git pull https://github.com/janebeckman/incubator-hawq-docs feature/SQLsyntaxcaps Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq-docs/pull/66.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #66 commit cc9bfcb7432380a2d0bfb4d7e99f18daae6fe657 Author: Jane Beckman Date: 2016-11-23T23:14:52Z Syntax standardization, grammar and usage corrections commit f83f351d6af1a394c7808085d10f3919840a029a Author: Jane Beckman Date: 2016-11-28T21:37:34Z Syntax and verbiage cleanup > Correct hawq scp syntax description for -J option > - > > Key: HAWQ-1175 > URL: https://issues.apache.org/jira/browse/HAWQ-1175 > Project: Apache HAWQ > Issue Type: Bug > Components: Documentation >Reporter: Jane Beckman >Assignee: David Yozie > > The -J description uses "copy to" and "copy from" destination strings, while > the syntax references ` and ` strings. The > description should match the syntax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1176) docs - pxf install via command line is missing some steps
[ https://issues.apache.org/jira/browse/HAWQ-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703295#comment-15703295 ] ASF GitHub Bot commented on HAWQ-1176: -- Github user dyozie commented on a diff in the pull request: https://github.com/apache/incubator-hawq-docs/pull/65#discussion_r89896512 --- Diff: pxf/InstallPXFPlugins.html.md.erb --- @@ -27,7 +27,7 @@ If you are using Ambari to install and manage your HAWQ cluster, you do *not* ne Each PXF service plug-in resides in its own RPM. You may have built these RPMs in the Apache HAWQ open source project repository (see [PXF Build Instructions](https://github.com/apache/incubator-hawq/blob/master/pxf/README.md)), or these RPMs may have been included in a commercial product download package. -PXF software must be installed on *each* node in your cluster. +Perform the following steps to install PXF software on **_each_** node in your cluster. --- End diff -- I think for clarity this should be reversed: Perform the following steps on **_each_** node in your cluster to install PXF. > docs - pxf install via command line is missing some steps > - > > Key: HAWQ-1176 > URL: https://issues.apache.org/jira/browse/HAWQ-1176 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Lisa Owen >Assignee: Lei Chang > Fix For: 2.0.1.0-incubating > > > add additional steps in "Installing PXF Plug-ins" doc section to initialize > and start the PXF service after pxf RPM install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1176) docs - pxf install via command line is missing some steps
[ https://issues.apache.org/jira/browse/HAWQ-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15703102#comment-15703102 ] ASF GitHub Bot commented on HAWQ-1176: -- GitHub user lisakowen opened a pull request: https://github.com/apache/incubator-hawq-docs/pull/65 HAWQ-1176 - include init and start steps in pxf install section updated the pxf plugin install doc to include init and start steps. You can merge this pull request into a Git repository by running: $ git pull https://github.com/lisakowen/incubator-hawq-docs feature/pxfinstall-service Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-hawq-docs/pull/65.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #65 commit 571b8fa3bf25573e85f2b67e7ef50f07a37b3365 Author: Lisa Owen Date: 2016-11-28T20:38:52Z include init and start steps in pxf install section > docs - pxf install via command line is missing some steps > - > > Key: HAWQ-1176 > URL: https://issues.apache.org/jira/browse/HAWQ-1176 > Project: Apache HAWQ > Issue Type: Bug >Reporter: Lisa Owen >Assignee: Lei Chang > Fix For: 2.0.1.0-incubating > > > add additional steps in "Installing PXF Plug-ins" doc section to initialize > and start the PXF service after pxf RPM install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1176) docs - pxf install via command line is missing some steps
Lisa Owen created HAWQ-1176: --- Summary: docs - pxf install via command line is missing some steps Key: HAWQ-1176 URL: https://issues.apache.org/jira/browse/HAWQ-1176 Project: Apache HAWQ Issue Type: Bug Reporter: Lisa Owen Assignee: Lei Chang Fix For: 2.0.1.0-incubating add additional steps in "Installing PXF Plug-ins" doc section to initialize and start the PXF service after pxf RPM install. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HAWQ-1175) Correct hawq scp syntax description for -J option
Jane Beckman created HAWQ-1175: -- Summary: Correct hawq scp syntax description for -J option Key: HAWQ-1175 URL: https://issues.apache.org/jira/browse/HAWQ-1175 Project: Apache HAWQ Issue Type: Bug Components: Documentation Reporter: Jane Beckman Assignee: David Yozie The -J description uses "copy to" and "copy from" destination strings, while the syntax references ` and ` strings. The description should match the syntax. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HAWQ-1171) Support upgrade for hawq register.
[ https://issues.apache.org/jira/browse/HAWQ-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702745#comment-15702745 ] Jane Beckman commented on HAWQ-1171: Hawq upgrade is a fairly generic command. Would this script possibly be providing multiple update functions (more than just updating hawq register functionality)? > Support upgrade for hawq register. > -- > > Key: HAWQ-1171 > URL: https://issues.apache.org/jira/browse/HAWQ-1171 > Project: Apache HAWQ > Issue Type: New Feature > Components: Core >Reporter: Hubert Zhang >Assignee: Hubert Zhang > > For Hawq register feature, we need to add some build-in functions to support > some catalog changes. This could be done by a hawqupgrade script. > User interface: > Hawq upgrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-hawq issue #700: HAWQ-799. Initial commmit for HAWQ Postgis suppor...
Github user kdunn926 commented on the issue: https://github.com/apache/incubator-hawq/pull/700 @liming01 - you are correct, because HAWQ doesn't support indexes, this work has the same limitation. All the spatial operation tests completed successfully, with the exception of spatial indexes and UPDATEs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
