[jira] [Closed] (HAWQ-960) Remove the file BUILD_INSTRUCTIONS.md and move the content in it to README.md

[Amy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amy closed HAWQ-960.

Resolution: Fixed
  Assignee: Amy  (was: Ed Espino)

> Remove the file BUILD_INSTRUCTIONS.md and move the content in it to README.md
> -
>
> Key: HAWQ-960
> URL: https://issues.apache.org/jira/browse/HAWQ-960
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build, Documentation
>Reporter: Goden Yao
>Assignee: Amy
>Priority: Minor
> Fix For: 2.0.0.0-incubating
>
>
> From [~jmclean] IPMC Release Candidate Review Feedback:
> {quote}
> I expected it in BUILD_INSTRUCTIONS.md but it just contains a link to the 
> wiki page, but anywhere in the release is fine.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq pull request #892: HAWQ-1007. Add the pgcrypto code into hawq

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/892


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #892: HAWQ-1007. Add the pgcrypto code into hawq

[wengyanqing]

Github user wengyanqing commented on the issue:

https://github.com/apache/incubator-hawq/pull/892
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #892: HAWQ-1007. Add the pgcrypto code into hawq

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/892
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #874: Remove Thrift 0.9.3 from source tree - HAW...

[edespino]

Github user edespino closed the pull request at:

https://github.com/apache/incubator-hawq/pull/874


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #892: HAWQ-1007. Add the pgcrypto code into hawq

[paul-guo-]

GitHub user paul-guo- opened a pull request:

https://github.com/apache/incubator-hawq/pull/892

HAWQ-1007. Add the pgcrypto code into hawq

Avoid maven license check for the pgcrypto code since it comes from 
upstream postgres.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/paul-guo-/incubator-hawq crypto2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/892.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #892


commit dbbdbe48879442fcecdea8153cd833fe76578479
Author: Paul Guo 
Date:   2016-09-07T04:28:02Z

HAWQ-1007. Add the pgcrypto code into hawq

Avoid maven license check for pgcrypto since it comes from upstream 
postgres.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Reopened] (HAWQ-1010) Add the crypto notice in README

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo reopened HAWQ-1010:


> Add the crypto notice in README
> ---
>
> Key: HAWQ-1010
> URL: https://issues.apache.org/jira/browse/HAWQ-1010
> Project: Apache HAWQ
>  Issue Type: Sub-task
>Reporter: Paul Guo
>Assignee: Roman Shaposhnik
> Fix For: 2.0.0.0-incubating
>
>
> According to http://www.apache.org/dev/crypto.html
> A crypto notice is needed in README. We need to add it after we pass the 
> Apache cryptography process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-1010) Add the crypto notice in README

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo closed HAWQ-1010.
--
   Resolution: Fixed
Fix Version/s: (was: 2.0.1.0-incubating)
   2.0.0.0-incubating

> Add the crypto notice in README
> ---
>
> Key: HAWQ-1010
> URL: https://issues.apache.org/jira/browse/HAWQ-1010
> Project: Apache HAWQ
>  Issue Type: Sub-task
>Reporter: Paul Guo
>Assignee: Roman Shaposhnik
> Fix For: 2.0.0.0-incubating
>
>
> According to http://www.apache.org/dev/crypto.html
> A crypto notice is needed in README. We need to add it after we pass the 
> Apache cryptography process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (HAWQ-1007) Add the pgcrypto code into hawq

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo closed HAWQ-1007.
--
Resolution: Fixed
  Assignee: Paul Guo  (was: Lei Chang)

> Add the pgcrypto code into hawq
> ---
>
> Key: HAWQ-1007
> URL: https://issues.apache.org/jira/browse/HAWQ-1007
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Reporter: Paul Guo
>Assignee: Paul Guo
> Fix For: 2.0.0.0-incubating
>
>
> We are using pgcrypto with a hacking solution by dynamically git-cloning 
> postgresql and patching the code. This is inefficient for development.
> We are doing like this just because of the Apache crypto process.
> http://www.apache.org/dev/crypto.html
> Recently the community decides to go through the apache crypto process.
> Now it is a good chance for us to add the code into hawq.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #890: HAWQ-958, HAWQ-976, HAWQ-957. LICENSE & NOTICE up...

[huor]

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/890
  
Huge effort for this and good to have them in +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #891: HAWQ-960. Remove the file BUILD_INSTRUCTIONS.md a...

[zhangh43]

Github user zhangh43 commented on the issue:

https://github.com/apache/incubator-hawq/pull/891
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #891: HAWQ-960. Remove the file BUILD_INSTRUCTIONS.md a...

[wengyanqing]

Github user wengyanqing commented on the issue:

https://github.com/apache/incubator-hawq/pull/891
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #891: HAWQ-960. Remove the file BUILD_INSTRUCTIO...

[amyrazz44]

GitHub user amyrazz44 opened a pull request:

https://github.com/apache/incubator-hawq/pull/891

HAWQ-960. Remove the file BUILD_INSTRUCTIONS.md and move the content

…in it to README.md.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amyrazz44/incubator-hawq HAWQ-960aa

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/891.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #891


commit ebc3e52d393e678484d43aed5a0157a1fd1feace
Author: amyrazz44 
Date:   2016-09-07T03:01:33Z

HAWQ-960. Remove the file BUILD_INSTRUCTIONS.md and move the content in it 
to README.md.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #889: HAWQ-1007. Add the pgcrypto code into hawq

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/889


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #825: HAWQ-961. Dispatch session user id (not cu...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-hawq/pull/825


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #890: HAWQ-958, HAWQ-976, HAWQ-957. LICENSE & NOTICE up...

[linwen]

Github user linwen commented on the issue:

https://github.com/apache/incubator-hawq/pull/890
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-960) Remove the file BUILD_INSTRUCTIONS.md and move the content in it to README.md

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo updated HAWQ-960:
--
Summary: Remove the file BUILD_INSTRUCTIONS.md and move the content in it 
to README.md  (was: BUILD_INSTRUCTIONS.md should contain more than just a link)

> Remove the file BUILD_INSTRUCTIONS.md and move the content in it to README.md
> -
>
> Key: HAWQ-960
> URL: https://issues.apache.org/jira/browse/HAWQ-960
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build, Documentation
>Reporter: Goden Yao
>Assignee: Ed Espino
>Priority: Minor
> Fix For: 2.0.0.0-incubating
>
>
> From [~jmclean] IPMC Release Candidate Review Feedback:
> {quote}
> I expected it in BUILD_INSTRUCTIONS.md but it just contains a link to the 
> wiki page, but anywhere in the release is fine.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-952) Clean up COPYRIGHT file and review NOTICE File

[Ed Espino (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469327#comment-15469327
 ] 

Ed Espino commented on HAWQ-952:


We are in PR: https://github.com/apache/incubator-hawq/pull/890

> Clean up COPYRIGHT file and review NOTICE File
> --
>
> Key: HAWQ-952
> URL: https://issues.apache.org/jira/browse/HAWQ-952
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Goden Yao
>Assignee: Ed Espino
> Fix For: 2.0.0.0-incubating
>
>
> Per mentor's suggestion, we should not have a separate COPYRIGHT file 
> And we need to review which copyright should be in NOTICE file in what way.
> (need more clarification and discussion with mentors)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-957) NOTICE file clean up

[Ed Espino (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469326#comment-15469326
 ] 

Ed Espino commented on HAWQ-957:


We are in PR: https://github.com/apache/incubator-hawq/pull/890

> NOTICE file clean up
> 
>
> Key: HAWQ-957
> URL: https://issues.apache.org/jira/browse/HAWQ-957
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Reporter: Goden Yao
>Assignee: Ed Espino
> Fix For: 2.0.0.0-incubating
>
>
> From [~jmclean] IPMC review feedback:
> {quote}
> NOTICE incorrecly contains a long list of copyright statements. I would 
> expect to see one or perhaps two here i.e. the original authors who donated 
> the software and who copyright statements were removed from the original 
> files.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-958) LICENSE file missing checklist

[Ed Espino (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469324#comment-15469324
 ] 

Ed Espino commented on HAWQ-958:


We are in PR: https://github.com/apache/incubator-hawq/pull/890

> LICENSE file missing checklist
> --
>
> Key: HAWQ-958
> URL: https://issues.apache.org/jira/browse/HAWQ-958
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Documentation
>Affects Versions: 2.0.0.0-incubating
>Reporter: Goden Yao
>Assignee: Ed Espino
>Priority: Blocker
> Fix For: 2.0.0.0-incubating
>
>
> From [~jmclean] IPMC release VOTE feedback
> {quote}
> - Please use the short form of the license linking to a license files in 
> LICENSE
> - BSD licensed code [3] 3. ./tools/bin/pythonSrc/unittest2-0.5.1/setup.py  
> [GY] [~rlei] can you check this one as you're the last committer through 
> HAWQ-837
> - BSD license code [7] 7. ./depends/thirdparty/thrift/compiler/cpp/src/md5.? 
> [GY] [~xunzhang] , this one was you through HAWQ-735
> - license for this file [9] 9. ./src/backend/port/dynloader/ultrix4.h   [GY] 
> original import, this one seems to be postgres license  plus a couple others.
> - license for this file [10] Are we OK this was taken form GNU C? 10. 
> ./src/port/inet_aton.c   [GY] original import
> - MIT license PSI [11] 11. ./tools/bin/pythonSrc/PSI-0.3b2_gp/ [GY] [~rlei] 
> this is from HAWQ-837 a month ago, MIT license
> - BSD licensed code [12] 12. ./src/port/snprintf.c
> - BSD licensed code [13] 13 ./src/port/crypt.c  Is this regard as 
> cryptography code? [14] 14. http://www.apache.org/dev/crypto.html
> - BSD licensed code [15][16] 15. ./src/port/memcmp.c , 16. 
> ./src/backend/utils/mb/wstrcmp.c
> - license for this file [17] 17. ./src/port/rand.c
> - license of these files [18][19] 18. ./src/backend/utils/adt/inet_net_ntop.c
> 19. ./src/backend/utils/adt/inet_net_pton.c
> - license of this file [20] 20 ./src/port/strlcpy.c
> - regex license [21] 21. ./src/backend/regex/COPYRIGHT
> - How are these files licensed? [22] + others copyright AEG Automation GmbH 
> 22. ./src/backend/port/qnx4/shm.c
> - How is this file licensed? [23] 23. ./src/backend/port/beos/shm.c
> - BSD licensed libpq [24]. 24. ./src/backend/libpq/sha2.?
> Is this considered crypto code and may need an export license?
> - pgdump [25] 25. ./src/bin/pg_dump/
> - license for this file [26] 26. ./src/port/gettimeofday.c
> - license for this file [27] Look like an ASF header may of been incorrectly 
> added to this. 27. 
> ./depends/thirdparty/thrift/lib/cpp/src/thrift/windows/SocketPair.cpp
> - This BSD licensed file [36] 36. ./src/bin/pg_controldata/pg_controldata.c
> - license for these files [37][38] and others in [39]
> 37. ./depends/thirdparty/thrift/aclocal/ax_cxx_compile_stdcxx_11.m4
> 38. ./depends/thirdparty/thrift/aclocal/ax_boost_base.m4
> 39. ./depends/thirdparty/thrift/aclocal
> - This BSD licensed file [40]
> 40. ./depends/thirdparty/thrift/build/cmake/FindGLIB.cmake
> - This BSD licensed file [41]
> 41. ./tools/bin/pythonSrc/unittest2-0.5.1/setup.py
> - BSD licensed pychecker [42]
> 42. ./tools/bin/pythonSrc/pychecker-0.8.18/
> - licenses for all of these files [43]
> 43. ./src/interfaces/libpq/po/*.po
> - BSD license pg800 [44]
> 44. ./tools/bin/ext/pg8000/*
> - how is this file licensed? [45]
> 45. ./src/backend/utils/mb/Unicode/UCS_to_GB18030.pl
> - license for this file [47] 47 
> ./tools/bin/pythonSrc/lockfile-0.9.1/lockfile/pidlockfile.py
> - Python license for this file [48]. Is this an Apache comparable license? 48 
> ./tools/bin/pythonSrc/pychecker-0.8.18/pychecker2/symbols.py
> - How are these files licensed? [49] Note multiple copyright owners and 
> missing headers.
> 49.  ./src/backend/utils/mb/Unicode/*
> - BSD licensed fig leaf. [50] Note that files incorrectly has had ASF headers 
> applied.
> 50. ./tools/bin/ext/figleaf/*
> - This BSD licensed file [51]
> 51. ./depends/thirdparty/thrift/lib/py/compat/win32/stdint.h
> - This public domain style sheet [52]
> 52. ./tools/bin/pythonSrc/PyGreSQL-4.0/docs/default.css
> - This file [53]
> 53. ./src/test/locale/test-ctype.c
> - License for unit test2 [54]
> 54 ./tools/bin/pythonSrc/unittest2-0.5.1/unittest2/
> - MIT licensed lock file [55]
> 55. ./tools/bin/pythonSrc/lockfile-0.9.1/LICENSE
> - JSON code here [56]
> 56. ./src/include/catalog/JSON
> - License for this file [57]
> 57. ./src/pl/plperl/ppport.h
> Looks like GPL/LPGL licensed code may be included [4][5][6] in the release.
> 4. ./depends/thirdparty/thrift/debian/copyright (end of file)
> 5. ./depends/thirdparty/thrift/doc/licenses/lgpl-2.1.txt
> 6. ./tools/bin/gppylib/operations/test/test_package.py
> This file [8] and others(?) may incorrectly have an ASF headers on it. Also 
> why does this file have an ASF header with copyright line? [46]
> 8. 

[GitHub] incubator-hawq pull request #890: HAWQ-958, HAWQ-976, HAWQ-957. LICENSE & NO...

[edespino]

GitHub user edespino opened a pull request:

https://github.com/apache/incubator-hawq/pull/890

HAWQ-958, HAWQ-976, HAWQ-957. LICENSE & NOTICE updates

* Update Stream license
* Update PyGreSQL license
* Update CMake license
* Remove ASF license header from CMake files
* Update pom.xml to exclude all libyarn and libhdfs3 CMake files
* Using new license format seen in spark.
* HAWQ-976. dynloader license (and corresponding source change)
* HAWQ-976. glob license (and corresponding source change)
* Add unittest2, pychecker, pg_controldata, pg8000, lockfile LICENSE files
* Add figleaf LICENSE file
* Remove incorrect ASF headers for figleaf
* update bzip2 license
* update googletest license
* update pexpect license
* update pitlockfile license
* update pljava license
* update oraface license
* Add licenses for:
LICENSE-json.txt
LICENSE-plperl.txt
LICENSE-postgresql.txt
LICENSE-test-ctype.txt
* Fix incorrect GPL reference - 
tools/bin/gppylib/operations/test/test_package.py
* HAWQ-957. Update NOTICE
* Remove ASF license header for plperl
* Add PSI,regents-of-the-university-of-california, 
university-of-california-at-berkeley LICENSE file
* Add port, wstrcmp BSD license
* Add isc, libpq-sha2, port-rand, port-gettimeofday, regex license
* Add pg_dump license

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wengyanqing/incubator-hawq master-HAWQ-958

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/890.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #890


commit 5fb7db359be4adf454b968d63c828ad7b1faf55b
Author: Ed Espino 
Date:   2016-09-05T09:00:48Z

HAWQ-958, HAWQ-976, HAWQ-957. LICENSE & NOTICE updates

* Update Stream license
* Update PyGreSQL license
* Update CMake license
* Remove ASF license header from CMake files
* Update pom.xml to exclude all libyarn and libhdfs3 CMake files
* Using new license format seen in spark.
* HAWQ-976. dynloader license (and corresponding source change)
* HAWQ-976. glob license (and corresponding source change)
* Add unittest2, pychecker, pg_controldata, pg8000, lockfile LICENSE files
* Add figleaf LICENSE file
* Remove incorrect ASF headers for figleaf
* update bzip2 license
* update googletest license
* update pexpect license
* update pitlockfile license
* update pljava license
* update oraface license
* Add licenses for:
LICENSE-json.txt
LICENSE-plperl.txt
LICENSE-postgresql.txt
LICENSE-test-ctype.txt
* Fix incorrect GPL reference - 
tools/bin/gppylib/operations/test/test_package.py
* HAWQ-957. Update NOTICE
* Remove ASF license header for plperl
* Add PSI,regents-of-the-university-of-california, 
university-of-california-at-berkeley LICENSE file
* Add port, wstrcmp BSD license
* Add isc, libpq-sha2, port-rand, port-gettimeofday, regex license
* Add pg_dump license




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Closed] (HAWQ-1010) Add the crypto notice in README

[Paul Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Guo closed HAWQ-1010.
--
   Resolution: Fixed
 Assignee: Roman Shaposhnik  (was: Lei Chang)
Fix Version/s: (was: 2.0.0.0-incubating)
   2.0.1.0-incubating

> Add the crypto notice in README
> ---
>
> Key: HAWQ-1010
> URL: https://issues.apache.org/jira/browse/HAWQ-1010
> Project: Apache HAWQ
>  Issue Type: Sub-task
>Reporter: Paul Guo
>Assignee: Roman Shaposhnik
> Fix For: 2.0.1.0-incubating
>
>
> According to http://www.apache.org/dev/crypto.html
> A crypto notice is needed in README. We need to add it after we pass the 
> Apache cryptography process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1010) Add the crypto notice in README

[Paul Guo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469311#comment-15469311
 ] 

Paul Guo commented on HAWQ-1010:


Roman checked in the change, so I'm closing the JIRA.

commit 327863331746513d454518aa3543bc6f3389bf62
Author: Roman Shaposhnik 
Date:   Tue Sep 6 11:15:38 2016 -0700

Including Crypto NOTICE in the README.md


> Add the crypto notice in README
> ---
>
> Key: HAWQ-1010
> URL: https://issues.apache.org/jira/browse/HAWQ-1010
> Project: Apache HAWQ
>  Issue Type: Sub-task
>Reporter: Paul Guo
>Assignee: Lei Chang
> Fix For: 2.0.0.0-incubating
>
>
> According to http://www.apache.org/dev/crypto.html
> A crypto notice is needed in README. We need to add it after we pass the 
> Apache cryptography process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-hawq issue #889: HAWQ-1007. Add the pgcrypto code into hawq

[huor]

Github user huor commented on the issue:

https://github.com/apache/incubator-hawq/pull/889
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #889: HAWQ-1007. Add the pgcrypto code into hawq

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/889
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #889: HAWQ-1007. Add the pgcrypto code into hawq

[paul-guo-]

GitHub user paul-guo- opened a pull request:

https://github.com/apache/incubator-hawq/pull/889

HAWQ-1007. Add the pgcrypto code into hawq



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/paul-guo-/incubator-hawq crypto2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/889.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #889






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq issue #877: HAWQ-1016. Add feature test for rowtypes with new...

[zhangh43]

Github user zhangh43 commented on the issue:

https://github.com/apache/incubator-hawq/pull/877
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #861: HAWQ-1001. Implement HAWQ user ACL check t...

[zhangh43]

Github user zhangh43 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/861#discussion_r77746338
  
--- Diff: src/backend/utils/misc/guc.c ---
@@ -4323,6 +4324,16 @@ static struct config_bool ConfigureNamesBool[] =
},
 
{
+{"enable_ranger", PGC_POSTMASTER, CONN_AUTH_SETTINGS,
+ gettext_noop("support to using ranger to manage hawq privilege."),
--- End diff --

fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #861: HAWQ-1001. Implement HAWQ user ACL check t...

[zhangh43]

Github user zhangh43 commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/861#discussion_r77746304
  
--- Diff: src/backend/catalog/aclchk.c ---
@@ -225,10 +225,18 @@ restrict_and_check_grant(bool is_grant, AclMode 
avail_goptions, bool all_privs,
 */
if (avail_goptions == ACL_NO_RIGHTS)
{
-   if (pg_aclmask(objkind, objectId, grantorId,
-  whole_mask | 
ACL_GRANT_OPTION_FOR(whole_mask),
-  ACLMASK_ANY) == ACL_NO_RIGHTS)
-   aclcheck_error(ACLCHECK_NO_PRIV, objkind, objname);
+ if (enable_ranger) {
--- End diff --

fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[shivzone]

Github user shivzone commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77741459
  
--- Diff: 
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/EnumHiveToHawqType.java
 ---
@@ -110,4 +122,68 @@ public static EnumHiveToHawqType 
getHiveToHawqType(String hiveType) {
 + hiveType + " to HAWQ's type");
 }
 
+
+/**
+ * 
+ * @param dataType Hawq data type
+ * @return compatible Hive type to given Hawq type, if there are more 
than one compatible types, it returns one with bigger size
+ * @throws UnsupportedTypeException if there is no corresponding Hive 
type for given Hawq type
+ */
+public static EnumHiveToHawqType getCompatibleHawqToHiveType(DataType 
dataType) {
+
+SortedSet types = new 
TreeSet(
+new Comparator() {
+public int compare(EnumHiveToHawqType a,
+EnumHiveToHawqType b) {
+return Byte.compare(a.getSize(), b.getSize());
+}
+});
+
+for (EnumHiveToHawqType t : values()) {
+if (t.getHawqType().getDataType().equals(dataType)) {
+types.add(t);
+}
+}
+
+if (types.size() == 0)
+throw new UnsupportedTypeException("Unable to find compatible 
Hive type for given HAWQ's type: " + dataType);
+
+return types.last();
+}
+
+/**
+ * 
+ * @param hiveType full Hive data type, i.e. varchar(10) etc
+ * @return array of type modifiers
+ * @throws UnsupportedTypeException if there is no such Hive type 
supported
+ */
+public static Integer[] extractModifiers(String hiveType) {
+Integer[] result = null;
+for (EnumHiveToHawqType t : values()) {
+String hiveTypeName = hiveType;
+String splitExpression = t.getSplitExpression();
+if (splitExpression != null) {
+String[] tokens = hiveType.split(splitExpression);
+hiveTypeName = tokens[0];
+result = new Integer[tokens.length - 1];
+for (int i = 0; i < tokens.length - 1; i++)
+result[i] = Integer.parseInt(tokens[i+1]);
+}
+if (t.getTypeName().toLowerCase()
+.equals(hiveTypeName.toLowerCase())) {
+return result;
+}
+}
+throw new UnsupportedTypeException("Unable to map Hive's type: "
++ hiveType + " to HAWQ's type");
+}
+
+/**
--- End diff --

Would be useful if the first line in this comment is added when we do the 
field declaration in the class (private byte size)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77741421
  
--- Diff: 
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/io/GPDBWritable.java 
---
@@ -832,40 +832,10 @@ private boolean isTextForm(int type) {
  * @return type name
  */
 public static String getTypeName(int oid) {
-switch (DataType.get(oid)) {
-case BOOLEAN:
-return "BOOLEAN";
-case BYTEA:
-return "BYTEA";
-case CHAR:
-return "CHAR";
-case BIGINT:
-return "BIGINT";
-case SMALLINT:
-return "SMALLINT";
-case INTEGER:
-return "INTEGER";
-case TEXT:
-return "TEXT";
-case REAL:
-return "REAL";
-case FLOAT8:
-return "FLOAT8";
-case BPCHAR:
-return "BPCHAR";
-case VARCHAR:
-return "VARCHAR";
-case DATE:
-return "DATE";
-case TIME:
-return "TIME";
-case TIMESTAMP:
-return "TIMESTAMP";
-case NUMERIC:
-return "NUMERIC";
-default:
-return "TEXT";
-}
+DataType type = DataType.get(oid);
--- End diff --

DataType.get works in such way that it either returns any known data type 
or UNSUPPORTED_TYPE if oid is invalid. And for UNSUPPORTED_TYPE we still return 
TEXT.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[shivzone]

Github user shivzone commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77741194
  
--- Diff: 
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/io/GPDBWritable.java 
---
@@ -832,40 +832,10 @@ private boolean isTextForm(int type) {
  * @return type name
  */
 public static String getTypeName(int oid) {
-switch (DataType.get(oid)) {
-case BOOLEAN:
-return "BOOLEAN";
-case BYTEA:
-return "BYTEA";
-case CHAR:
-return "CHAR";
-case BIGINT:
-return "BIGINT";
-case SMALLINT:
-return "SMALLINT";
-case INTEGER:
-return "INTEGER";
-case TEXT:
-return "TEXT";
-case REAL:
-return "REAL";
-case FLOAT8:
-return "FLOAT8";
-case BPCHAR:
-return "BPCHAR";
-case VARCHAR:
-return "VARCHAR";
-case DATE:
-return "DATE";
-case TIME:
-return "TIME";
-case TIMESTAMP:
-return "TIMESTAMP";
-case NUMERIC:
-return "NUMERIC";
-default:
-return "TEXT";
-}
+DataType type = DataType.get(oid);
--- End diff --

Earlier we handled the default case as "TEXT". With this change we will get 
"UNSUPPORTED_TYPE"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77727612
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java 
---
@@ -43,37 +45,40 @@ public void serialize(EnumHawqType value, JsonGenerator 
generator,
  */
 @JsonSerialize(using = EnumHawqTypeSerializer.class)
 public enum EnumHawqType {
-Int2Type("int2"),
-Int4Type("int4"),
-Int8Type("int8"),
-Float4Type("float4"),
-Float8Type("float8"),
-TextType("text"),
-VarcharType("varchar", (byte) 1, true),
-ByteaType("bytea"),
-DateType("date"),
-TimestampType("timestamp"),
-BoolType("bool"),
-NumericType("numeric", (byte) 2, true),
-BpcharType("bpchar", (byte) 1, true);
+Int2Type("int2", DataType.SMALLINT),
+Int4Type("int4", DataType.INTEGER),
+Int8Type("int8", DataType.BIGINT),
+Float4Type("float4", DataType.REAL),
+Float8Type("float8", DataType.FLOAT8),
+TextType("text", DataType.TEXT),
+VarcharType("varchar", DataType.VARCHAR, (byte) 1, false),
+ByteaType("bytea", DataType.BYTEA),
+DateType("date", DataType.DATE),
+TimestampType("timestamp", DataType.TIMESTAMP),
+BoolType("bool", DataType.BOOLEAN),
+NumericType("numeric", DataType.NUMERIC, (byte) 2, false),
+BpcharType("bpchar", DataType.BPCHAR, (byte) 1, false),
+//modifier is mandatory for this type because by default it's 1
+CharType("char", DataType.CHAR, (byte) 1, true);
--- End diff --

Deleted


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77727566
  
--- Diff: 
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java
 ---
@@ -256,4 +257,68 @@ private static boolean verifyIntegerModifiers(String[] 
modifiers) {
 throw new RuntimeException("Failed connecting to Hive 
MetaStore service: " + cause.getMessage(), cause);
 }
 }
+
+
+/**
+ * Converts HAWQ type to hive type. The supported mappings are:
+ * {@code BOOLEAN -> boolean}
+ * {@code SMALLINT -> smallint (tinyint is converted to 
smallint)}
+ * {@code BIGINT -> bigint}
+ * {@code TIMESTAMP -> timestamp}
+ * {@code NUMERIC -> decimal}
+ * {@code BYTEA -> binary}
+ * {@code INTERGER -> int}
+ * {@code TEXT -> string}
+ * {@code REAL -> float}
+ * {@code FLOAT8 -> double}
+ * 
+ * All other types (both in HAWQ and in HIVE) are not supported.
+ *
+ * @param type HAWQ data type
+ * @param name field name
+ * @return Hive type
+ * @throws UnsupportedTypeException if type is not supported
+ */
+public static String toCompatibleHiveType(DataType type) {
+
+EnumHiveToHawqType hiveToHawqType = 
EnumHiveToHawqType.getCompatibleHawqToHiveType(type);
+return hiveToHawqType.getTypeName();
+}
+
+
+
+public static void validateTypeCompatible(DataType hawqDataType, 
Integer[] hawqTypeMods, String hiveType, String hawqColumnName) {
+
+EnumHiveToHawqType hiveToHawqType = 
EnumHiveToHawqType.getHiveToHawqType(hiveType);
+EnumHawqType expectedHawqType = hiveToHawqType.getHawqType();
+
+if (!expectedHawqType.getDataType().equals(hawqDataType)) {
+throw new UnsupportedTypeException("Invalid definition for 
column " + hawqColumnName
++  ": expected HAWQ type " + 
expectedHawqType.getDataType() +
+", actual HAWQ type " + hawqDataType);
+}
+
+if ((hawqTypeMods == null || hawqTypeMods.length == 0) && 
expectedHawqType.isMandatoryModifiers())
+throw new UnsupportedTypeException("Invalid definition for 
column " + hawqColumnName +  ": modifiers are mandatory for type " + 
expectedHawqType.getTypeName());
+
--- End diff --

Deleted this check after discussions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77727073
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java 
---
@@ -43,37 +45,40 @@ public void serialize(EnumHawqType value, JsonGenerator 
generator,
  */
 @JsonSerialize(using = EnumHawqTypeSerializer.class)
 public enum EnumHawqType {
-Int2Type("int2"),
-Int4Type("int4"),
-Int8Type("int8"),
-Float4Type("float4"),
-Float8Type("float8"),
-TextType("text"),
-VarcharType("varchar", (byte) 1, true),
-ByteaType("bytea"),
-DateType("date"),
-TimestampType("timestamp"),
-BoolType("bool"),
-NumericType("numeric", (byte) 2, true),
-BpcharType("bpchar", (byte) 1, true);
+Int2Type("int2", DataType.SMALLINT),
+Int4Type("int4", DataType.INTEGER),
+Int8Type("int8", DataType.BIGINT),
+Float4Type("float4", DataType.REAL),
+Float8Type("float8", DataType.FLOAT8),
+TextType("text", DataType.TEXT),
+VarcharType("varchar", DataType.VARCHAR, (byte) 1, false),
+ByteaType("bytea", DataType.BYTEA),
+DateType("date", DataType.DATE),
+TimestampType("timestamp", DataType.TIMESTAMP),
+BoolType("bool", DataType.BOOLEAN),
+NumericType("numeric", DataType.NUMERIC, (byte) 2, false),
+BpcharType("bpchar", DataType.BPCHAR, (byte) 1, false),
--- End diff --

deleted mandatory attribute after discussions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77726599
  
--- Diff: 
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/HiveUtilities.java
 ---
@@ -256,4 +257,68 @@ private static boolean verifyIntegerModifiers(String[] 
modifiers) {
 throw new RuntimeException("Failed connecting to Hive 
MetaStore service: " + cause.getMessage(), cause);
 }
 }
+
+
+/**
+ * Converts HAWQ type to hive type. The supported mappings are:
+ * {@code BOOLEAN -> boolean}
+ * {@code SMALLINT -> smallint (tinyint is converted to 
smallint)}
+ * {@code BIGINT -> bigint}
+ * {@code TIMESTAMP -> timestamp}
+ * {@code NUMERIC -> decimal}
+ * {@code BYTEA -> binary}
+ * {@code INTERGER -> int}
+ * {@code TEXT -> string}
+ * {@code REAL -> float}
+ * {@code FLOAT8 -> double}
+ * 
--- End diff --

Thanks, updated JavaDoc to reference to EnumHiveToHawqType where those 
mappings are defined so we don't have to keep docs in two places.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77724313
  
--- Diff: 
pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/ColumnDescriptor.java
 ---
@@ -26,10 +26,11 @@
  */
 public class ColumnDescriptor {
 
-   int gpdbColumnTypeCode;
-String gpdbColumnName;
-String gpdbColumnTypeName;
-int gpdbColumnIndex;
+   int dbColumnTypeCode;
--- End diff --

Thanks, fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77723697
  
--- Diff: 
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/EnumHiveToHawqType.java
 ---
@@ -110,4 +122,68 @@ public static EnumHiveToHawqType 
getHiveToHawqType(String hiveType) {
 + hiveType + " to HAWQ's type");
 }
 
+
+/**
+ * 
+ * @param dataType Hawq data type
+ * @return compatible Hive type to given Hawq type, if there are more 
than one compatible types, it returns one with bigger size
+ * @throws UnsupportedTypeException if there is no corresponding Hive 
type for given Hawq type
+ */
+public static EnumHiveToHawqType getCompatibleHawqToHiveType(DataType 
dataType) {
+
+SortedSet types = new 
TreeSet(
+new Comparator() {
+public int compare(EnumHiveToHawqType a,
+EnumHiveToHawqType b) {
+return Byte.compare(a.getSize(), b.getSize());
+}
+});
--- End diff --

It's actually being populated based on dataType argument.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq pull request #873: HAWQ-992. PXF Hive data type check in Frag...

[sansanichfb]

Github user sansanichfb commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/873#discussion_r77723398
  
--- Diff: 
pxf/pxf-hive/src/main/java/org/apache/hawq/pxf/plugins/hive/utilities/EnumHiveToHawqType.java
 ---
@@ -29,8 +35,8 @@
  */
 public enum EnumHiveToHawqType {
 
-TinyintType("tinyint", EnumHawqType.Int2Type),
-SmallintType("smallint", EnumHawqType.Int2Type),
+TinyintType("tinyint", EnumHawqType.Int2Type, (byte) 1),
+SmallintType("smallint", EnumHawqType.Int2Type, (byte) 2),
--- End diff --

Sure, we already have explanation in EnumHiveToHawqType#getSize()


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (HAWQ-1036) Support user impersonation in PXF for external tables

[Goden Yao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Goden Yao updated HAWQ-1036:

Description: 
Currently HAWQ executes all queries as the user running the HAWQ process or the 
user running the PXF process, not as the user who issued the query via 
ODBC/JDBC/... This restricts the options available for integrating with 
existing security defined in HDFS, Hive, etc.

Impersonation provides an alternative Ranger integration (as discussed in 
HAWQ-256 ) for consistent security across HAWQ, HDFS, Hive...

Implementation High Level steps:
1) HAWQ needs to integrate with existing authentication components for the user 
who invokes the query
2) HAWQ needs to pass down the user id to PXF after authorization is passed 
3) PXF needs to do "run as ..." the user id to execute APIs to access Hive/HDFS 

  was:
Currently HAWQ executes all queries as the user running the HAWQ process or the 
user running the PXF process, not as the user who issued the query via 
ODBC/JDBC/... This restricts the options available for integrating with 
existing security defined in HDFS, Hive, etc.

Impersonation provides an alternative Ranger integration (as discussed in 
HAWQ-256 ) for consistent security across HAWQ, HDFS, Hive...


> Support user impersonation in PXF for external tables
> -
>
> Key: HAWQ-1036
> URL: https://issues.apache.org/jira/browse/HAWQ-1036
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Alastair "Bell" Turner
>Assignee: Goden Yao
>Priority: Critical
> Fix For: backlog
>
> Attachments: HAWQ_Impersonation_rationale.txt
>
>
> Currently HAWQ executes all queries as the user running the HAWQ process or 
> the user running the PXF process, not as the user who issued the query via 
> ODBC/JDBC/... This restricts the options available for integrating with 
> existing security defined in HDFS, Hive, etc.
> Impersonation provides an alternative Ranger integration (as discussed in 
> HAWQ-256 ) for consistent security across HAWQ, HDFS, Hive...
> Implementation High Level steps:
> 1) HAWQ needs to integrate with existing authentication components for the 
> user who invokes the query
> 2) HAWQ needs to pass down the user id to PXF after authorization is passed 
> 3) PXF needs to do "run as ..." the user id to execute APIs to access 
> Hive/HDFS 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HAWQ-1036) Support user impersonation in PXF for external tables

[Goden Yao (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468123#comment-15468123
 ] 

Goden Yao edited comment on HAWQ-1036 at 9/6/16 6:54 PM:
-

this request has nothing to do with database objects privilege management or 
ranger integration.
Alastair‘s attached text file well elaborated on the implementation for this 
JIRA.

I'll make sure I describe that in the JIRA description as well.


was (Author: godenyao):
this request has nothing to do with database objects privilege management or 
ranger integration.
I'll post a detailed discussion with Alastair before.

> Support user impersonation in PXF for external tables
> -
>
> Key: HAWQ-1036
> URL: https://issues.apache.org/jira/browse/HAWQ-1036
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Alastair "Bell" Turner
>Assignee: Goden Yao
>Priority: Critical
> Fix For: backlog
>
> Attachments: HAWQ_Impersonation_rationale.txt
>
>
> Currently HAWQ executes all queries as the user running the HAWQ process or 
> the user running the PXF process, not as the user who issued the query via 
> ODBC/JDBC/... This restricts the options available for integrating with 
> existing security defined in HDFS, Hive, etc.
> Impersonation provides an alternative Ranger integration (as discussed in 
> HAWQ-256 ) for consistent security across HAWQ, HDFS, Hive...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1036) Support user impersonation in PXF for external tables

[Goden Yao (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468138#comment-15468138
 ] 

Goden Yao commented on HAWQ-1036:
-

a) - yes
b) not sure if that's a statement or question - user impersonation should be 
only exercised if/when HAWQ chose to integrate with HADOOP user identification. 
so there will be 1> default - hawq manages users as dbms separately so no 
behavior changes 2> integrated with hadoop , so all db users should be in 
kerberos or LDAP, through ranger or other centralized user authentication 
system.
c) no matter which mode Hive chooses, HDFS layer you still have hdfs users 
(OS/Hadoop users) specific ACL.

impersonation is not our side to do authentication, we just need to trust hive 
APIs and pass down the real user ID who's invoking the query and run as that 
user to make sure no illegal access during the process.

> Support user impersonation in PXF for external tables
> -
>
> Key: HAWQ-1036
> URL: https://issues.apache.org/jira/browse/HAWQ-1036
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Alastair "Bell" Turner
>Assignee: Goden Yao
>Priority: Critical
> Fix For: backlog
>
> Attachments: HAWQ_Impersonation_rationale.txt
>
>
> Currently HAWQ executes all queries as the user running the HAWQ process or 
> the user running the PXF process, not as the user who issued the query via 
> ODBC/JDBC/... This restricts the options available for integrating with 
> existing security defined in HDFS, Hive, etc.
> Impersonation provides an alternative Ranger integration (as discussed in 
> HAWQ-256 ) for consistent security across HAWQ, HDFS, Hive...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-1036) Support user impersonation in PXF for external tables

[Goden Yao (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468123#comment-15468123
 ] 

Goden Yao commented on HAWQ-1036:
-

this request has nothing to do with database objects privilege management or 
ranger integration.
I'll post a detailed discussion with Alastair before.

> Support user impersonation in PXF for external tables
> -
>
> Key: HAWQ-1036
> URL: https://issues.apache.org/jira/browse/HAWQ-1036
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF, Security
>Reporter: Alastair "Bell" Turner
>Assignee: Goden Yao
>Priority: Critical
> Fix For: backlog
>
> Attachments: HAWQ_Impersonation_rationale.txt
>
>
> Currently HAWQ executes all queries as the user running the HAWQ process or 
> the user running the PXF process, not as the user who issued the query via 
> ODBC/JDBC/... This restricts the options available for integrating with 
> existing security defined in HDFS, Hive, etc.
> Impersonation provides an alternative Ranger integration (as discussed in 
> HAWQ-256 ) for consistent security across HAWQ, HDFS, Hive...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HAWQ-959) Clean up Binary files based on RAT reports

[Ed Espino (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino resolved HAWQ-959.

Resolution: Fixed

Thrift has been removed.  We have added a comment to this jira to reflect the 
large set of binary files and how they are used in the source tree (example: 
test files, language files, test language files, etc).  Anyone performing a 
review of the source base can reference this jira.

> Clean up Binary files based on RAT reports
> --
>
> Key: HAWQ-959
> URL: https://issues.apache.org/jira/browse/HAWQ-959
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build
>Affects Versions: 2.0.0.0-incubating
>Reporter: Goden Yao
>Assignee: Ed Espino
> Fix For: 2.0.0.0-incubating
>
>
> From [~jmclean] and [~alangates] IPMC release candidate review
> An unexpected binary in the source code base:
> depends/thirdparty/thrift/lib/erl/rebar
> There might be others given rat reports 770+ binary files
> We need to clean them up or make exceptions for them with solid reasons.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HAWQ-959) Clean up Binary files based on RAT reports

[Ed Espino (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466902#comment-15466902
 ] 

Ed Espino commented on HAWQ-959:


{code}
The following list of files are reported as binary files by the Apache
Release Audit Tool (RAT).

These are acceptable and are categoriezed as follows:

HAWQ Test binary files:


./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179546.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179551.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179556.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179561.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179566.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179571.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179576.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179581.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179586.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179591.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179596.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179601.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179606.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179626.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179631.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179636.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179641.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179646.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179651.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179656.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179661.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179666.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179671.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179676.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179681.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179686.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179691.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179696.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179701.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179706.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179711.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179721.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179731.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179766.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179771.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179776.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179786.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179801.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179806.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179811.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179816.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179821.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179826.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179831.1

./contrib/hawq-hadoop/hawq-mapreduce-tool/test-data/hawq-data/gpseg0/16385/179545/179836.1

[jira] [Resolved] (HAWQ-993) Symbolic link (contrib/pgcrypto) pointing to non-existent location

[Ed Espino (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ed Espino resolved HAWQ-993.

Resolution: Won't Fix

After discussing with [~Paul Guo], he has informed me this is not an issue.  I 
am resolving this as "Won't Fix".

> Symbolic link (contrib/pgcrypto) pointing to non-existent location
> --
>
> Key: HAWQ-993
> URL: https://issues.apache.org/jira/browse/HAWQ-993
> Project: Apache HAWQ
>  Issue Type: Bug
>  Components: Build
>Affects Versions: 2.0.0.0-incubating
>Reporter: Ed Espino
>Assignee: Ed Espino
>Priority: Minor
> Fix For: 2.0.0.0-incubating
>
>
> The following symbolic link points to a non-existent location.  This 
> generates a potential error for the "rat" utility (see below):
> contrib/pgcrypto@ -> ../depends/thirdparty/postgres/contrib/pgcrypto
> {code}
> java -Xms1024m -Xmx1024m -jar ~/Downloads/apache-rat-0.11/apache-rat-0.11.jar 
> .
> Exception in thread "main" org.apache.rat.api.RatException: Cannot read header
> at 
> org.apache.rat.report.claim.util.ClaimReporterMultiplexer.report(ClaimReporterMultiplexer.java:44)
> at org.apache.rat.walker.DirectoryWalker.report(DirectoryWalker.java:147)
> at 
> org.apache.rat.walker.DirectoryWalker.processNonDirectories(DirectoryWalker.java:131)
> at org.apache.rat.walker.DirectoryWalker.process(DirectoryWalker.java:98)
> at 
> org.apache.rat.walker.DirectoryWalker.processDirectory(DirectoryWalker.java:71)
> at 
> org.apache.rat.walker.DirectoryWalker.processDirectories(DirectoryWalker.java:114)
> at org.apache.rat.walker.DirectoryWalker.process(DirectoryWalker.java:99)
> at org.apache.rat.walker.DirectoryWalker.run(DirectoryWalker.java:83)
> at org.apache.rat.Report.report(Report.java:418)
> at org.apache.rat.Report.report(Report.java:394)
> at org.apache.rat.Report.report(Report.java:366)
> at org.apache.rat.Report.styleReport(Report.java:346)
> at org.apache.rat.Report.main(Report.java:109)
> Caused by: org.apache.rat.document.RatDocumentAnalysisException: Cannot read 
> header
> at 
> org.apache.rat.analysis.DocumentHeaderAnalyser.analyse(DocumentHeaderAnalyser.java:45)
> at 
> org.apache.rat.analysis.DefaultAnalyserFactory$DefaultAnalyser.analyse(DefaultAnalyserFactory.java:60)
> at 
> org.apache.rat.document.impl.util.DocumentAnalyserMultiplexer.analyse(DocumentAnalyserMultiplexer.java:36)
> at 
> org.apache.rat.report.claim.util.ClaimReporterMultiplexer.report(ClaimReporterMultiplexer.java:42)
> ... 12 more
> Caused by: java.io.FileNotFoundException: ./contrib/pgcrypto (No such file or 
> directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:131)
> at java.io.FileReader.(FileReader.java:72)
> at org.apache.rat.document.impl.FileDocument.reader(FileDocument.java:52)
> at 
> org.apache.rat.analysis.DocumentHeaderAnalyser.analyse(DocumentHeaderAnalyser.java:40)
> ... 15 more
> ERROR:  'Pipe broken'
> ERROR:  'com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Pipe 
> broken'
> Exception in thread "Thread-0" org.apache.rat.ReportFailedRuntimeException: 
> javax.xml.transform.TransformerException: 
> com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Pipe broken
> at org.apache.rat.ReportTransformer.run(ReportTransformer.java:59)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.xml.transform.TransformerException: 
> javax.xml.transform.TransformerException: 
> com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Pipe broken
> at 
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:755)
> at 
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:357)
> at org.apache.rat.ReportTransformer.transform(ReportTransformer.java:64)
> at org.apache.rat.ReportTransformer.run(ReportTransformer.java:57)
> ... 1 more
> Caused by: javax.xml.transform.TransformerException: 
> com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Pipe broken
> at 
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.getDOM(TransformerImpl.java:584)
> at 
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:745)
> ... 4 more
> Caused by: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: 
> Pipe broken
> at 
> com.sun.org.apache.xalan.internal.xsltc.dom.XSLTCDTMManager.getDTM(XSLTCDTMManager.java:427)
> at 
> com.sun.org.apache.xalan.internal.xsltc.dom.XSLTCDTMManager.getDTM(XSLTCDTMManager.java:215)
> at 
> com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.getDOM(TransformerImpl.java:562)
> 

[jira] [Commented] (HAWQ-960) BUILD_INSTRUCTIONS.md should contain more than just a link

[Amy (JIRA)]

[ 
https://issues.apache.org/jira/browse/HAWQ-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466787#comment-15466787
 ] 

Amy commented on HAWQ-960:
--

As our opinion, if we move all the context from build wiki page to 
BUILD_INSTUCTIONS.md file,
the BUILD_INSTUCTIONS.md file will become so large and too difficult to read. 
So, our suggestion is that we could move the [build 
link|https://cwiki.apache.org/confluence/display/HAWQ/Build+and+Install] on 
hawq wiki page to README.md , so that BUILD_INSTUCTIONS.md file could be 
removed. By doing all above ,hawq install files will be more clean and neat.

> BUILD_INSTRUCTIONS.md should contain more than just a link
> --
>
> Key: HAWQ-960
> URL: https://issues.apache.org/jira/browse/HAWQ-960
> Project: Apache HAWQ
>  Issue Type: Task
>  Components: Build, Documentation
>Reporter: Goden Yao
>Assignee: Ed Espino
>Priority: Minor
> Fix For: 2.0.0.0-incubating
>
>
> From [~jmclean] IPMC Release Candidate Review Feedback:
> {quote}
> I expected it in BUILD_INSTRUCTIONS.md but it just contains a link to the 
> wiki page, but anywhere in the release is fine.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)