[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. IMPALA-10388: [DOCS] add limitations on mask functions incorporated comments, removed the para as per the feedback listed all the overloads that are introduced stated that Impala does not yet support new Hive UDFs called out how mask functions were introduced through overloads Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Reviewed-on: http://gerrit.cloudera.org:8080/16861 Tested-by: Impala Public Jenkins Reviewed-by: Quanlong Huang --- M docs/topics/impala_authorization.xml 1 file changed, 25 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Quanlong Huang: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 4 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 3 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 06 Jan 2021 23:22:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 3: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/615/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 3 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 06 Jan 2021 19:18:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/615/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 3 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Wed, 06 Jan 2021 19:11:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Hello Quanlong Huang, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16861 to look at the new patch set (#3). Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. IMPALA-10388: [DOCS] add limitations on mask functions incorporated comments, removed the para as per the feedback listed all the overloads that are introduced stated that Impala does not yet support new Hive UDFs called out how mask functions were introduced through overloads Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f --- M docs/topics/impala_authorization.xml 1 file changed, 25 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/16861/3 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 3 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16861/2/docs/topics/impala_authorization.xml File docs/topics/impala_authorization.xml: http://gerrit.cloudera.org:8080/#/c/16861/2/docs/topics/impala_authorization.xml@711 PS2, Line 711: This example shows how to use the overload mask(FLOAT) that is not yet implemented in : Impala. If you want to mask float/double/decimal to NULLs, you can use a more efficient : Nullify policy (MASK_NULL) for the same purpose. Do not use Redact(MASK) on : float/double/decimal types as Hive always returns NULL for them since it is not : redact. This is not an example "shows how to use the overload mask(FLOAT) that is not yet implemented in Impala"... What I want to say is mask(FLOAT), mask(DOUBLE), mask(DECIMAL) are missing in Impala currently. If users use Redact(MASK) column masking policy on float/double/decimal columns, they will hit an error like "No matching function with signature: mask..." in Impala. Then I explain that it doesn't make sense to use Redact(MASK) column masking policy on float/double/decimal columns. The reason is Hive's mask function always returns NULLs on these types. The actually meaning of "Redact" is replacing the content with mask characters, e.g. transforming "ABCDabcd-1234" to "-". So transforming 3.14 to NULL doesn't mean "Redact" actually. "Redact" will transform "3.14" to "n.nn". If users do want to mask float/double/decimal types to NULLs, they should use the Nullify policy. I think you understand this part correctly. :) The missing overloads are not just mask(FLOAT), mask(DOUBLE) and mask(DECIMAL). This is an example explaining why they are missing (because it doesn't make sense to use them), and providing a workaround if users do want to use them. I think if it's hard to explain these, we can remove this paragraph. -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 18 Dec 2020 13:20:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/611/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 17 Dec 2020 18:34:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/611/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 17 Dec 2020 18:27:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Hello Quanlong Huang, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16861 to look at the new patch set (#2). Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. IMPALA-10388: [DOCS] add limitations on mask functions incorporated comments listed all the overloads that are introduced stated that Impala does not yet support new Hive UDFs called out how mask functions were introduced through overloads Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f --- M docs/topics/impala_authorization.xml 1 file changed, 30 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/16861/2 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 2 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/16861/1/docs/topics/impala_authorization.xml File docs/topics/impala_authorization.xml: http://gerrit.cloudera.org:8080/#/c/16861/1/docs/topics/impala_authorization.xml@693 PS1, Line 693: Changes in Hive UDF implemented through "GenericUDF" supports mask functions. Even though : Impala users can call Hive UDFs, Impala does not yet support new Hive UDFs based on the : GenericUDF class. Due to the lack of a corresponding framework for GenericUDF in Impala, : this release introduced some of these mask functions through overloads. The first sentense make me confused. The GenericUDF framework and the mask functions in Hive have been implemented for years. Not sure what "Changes" means. I think this explains the limitation better: The mask functions in Hive are GenericUDFs. However, Impala currently doesn't support Hive GenericUDFs, so you can't use Hive's mask functions in Impala. Impala has builtin mask functions. They are implemented by overloads. Thus, when using mask functions, not all parameter combinations are supported in Impala. I'm not a native English speaker. Please help to correct the grammar mistakes and polish the content. :) http://gerrit.cloudera.org:8080/#/c/16861/1/docs/topics/impala_authorization.xml@697 PS1, Line 697: The following list includes all the overloads that are introduced in this release. This is not a release doc, but a doc that will be used in all releases. I don't think we should mentione "in this release" here. BTW, the mask functions are added in Impala-3.4, not the comming release version (4.0). Can you update this too: The following list includes all the overloads that are implemented. http://gerrit.cloudera.org:8080/#/c/16861/1/docs/topics/impala_authorization.xml@710 PS1, Line 710: The function mask(FLOAT) is not implemented in Impala. Do not use Redact(MASK) on : float/double/decimal types as Hive always returns NULL for them since it is not redact. : You can use a more efficient Nullify policy (MASK_NULL) for the same purpose. This is just an example explaining the overload, mask(FLOAT), is currently missing in Impala. And then provides a work around. I think we can update this paragraph to As an example, the function mask(FLOAT) is currently not implemented in Impala. Do not use Redact(MASK) on float/double/decimal types as Hive always returns NULL for them so it is not redact actually. If you do want to mask float/double/decimal to NULLs, you can use a more efficient Nullify policy (MASK_NULL) for the same purpose. -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 17 Dec 2020 07:32:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/610/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 11 Dec 2020 19:32:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16861 ) Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/610/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 11 Dec 2020 19:25:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10388: [DOCS] add limitations on mask functions
Shajini Thayasingh has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16861 Change subject: IMPALA-10388: [DOCS] add limitations on mask functions .. IMPALA-10388: [DOCS] add limitations on mask functions listed all the overloads that are introduced stated that Impala does not yet support new Hive UDFs called out how mask functions were introduced through overloads Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f --- M docs/topics/impala_authorization.xml 1 file changed, 27 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/16861/1 -- To view, visit http://gerrit.cloudera.org:8080/16861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I37f0bcf4cf586cc5cfd03e4df68443967b6bb88f Gerrit-Change-Number: 16861 Gerrit-PatchSet: 1 Gerrit-Owner: Shajini Thayasingh
