[jira] [Commented] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703725#comment-16703725 ] Brian Towles commented on SENTRY-2458: -- [~spena] Thanks. Fix is in. Was not failing locally at all on that. Cause its really a binary. Not sure why its being picked up as text with the Apache Jenkins. > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch, SENTRY-2458.005.patch, > SENTRY-2458.006.patch, SENTRY-2458.007.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.007.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch, SENTRY-2458.005.patch, > SENTRY-2458.006.patch, SENTRY-2458.007.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703411#comment-16703411 ] Brian Towles commented on SENTRY-2458: -- Yes I can . I can run a full verify (which runs the RAT checks) I think this is an issue with the build on jenkins. I cant look into the build workspace to be able to check the RAT report. If [~spena] or someone can look at the above path on the jenkins build we can see whats going on. I added RAT changes in to take into account the SVGs and other web fonts. > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch, SENTRY-2458.005.patch, > SENTRY-2458.006.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.006.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch, SENTRY-2458.005.patch, > SENTRY-2458.006.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2465) Move SentryStore instatiation from SentryService to a utility and possibly own module
Brian Towles created SENTRY-2465: Summary: Move SentryStore instatiation from SentryService to a utility and possibly own module Key: SENTRY-2465 URL: https://issues.apache.org/jira/browse/SENTRY-2465 Project: Sentry Issue Type: Improvement Components: Service Affects Versions: 2.0.1 Reporter: Brian Towles Fix For: 2.2.0 Getting an instantiated instance of the current SentryStore instance is currently a private function in within SentryService. Since the current SentryStore instance needs to be available to other modules instantiation of SentryStore should be moved to a separate utility and module. It can use the sentry-spi to allow for multiple different SentryStore instances to be created and referenced without defining the full package/class combination in the configuration. It can be developed such a way that the class name definition in thje config can be backwards compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.005.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch, SENTRY-2458.005.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.004.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: (was: SENTRY-2458.004.patch) > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.004.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch, SENTRY-2458.004.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2462) Seperate Metrics into sub module in order to remove circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2462: - Attachment: SENTRY-2462.001.patch Status: Patch Available (was: Open) This is dependent on SENTRY-2458 > Seperate Metrics into sub module in order to remove circular dependencies > - > > Key: SENTRY-2462 > URL: https://issues.apache.org/jira/browse/SENTRY-2462 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2462.001.patch > > > The metrics subsystem currently resides in the sentry-service-server. So any > other module that needs to use it has to depend on sentry-service-server > which is causing circular dependencies when trying to add modules used by > sentry-service-server. > > This will create a sentry-service-metrics that will house and wrap all the > metrics needs for the server. It will need to use SENTRY-2458 as a base to > segment out the web presentation components of the metrics. It will also > allow for the centralization of the roll up and shading of the codehale > metrics package to happen in one sub module rather then split over two. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2462) Seperate Metrics into sub module in order to remove circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2462: - Description: The metrics subsystem currently resides in the sentry-service-server. So any other module that needs to use it has to depend on sentry-service-server which is causing circular dependencies when trying to add modules used by sentry-service-server. This will create a sentry-service-metrics that will house and wrap all the metrics needs for the server. It will need to use SENTRY-2458 as a base to segment out the web presentation components of the metrics. It will also allow for the centralization of the roll up and shading of the codehale metrics package to happen in one sub module rather then split over two. was: The metrics subsystem currently resides in the sentry-service-server. So any other module that needs to use it has to depend on sentry-service-server which is causing circular dependencies when trying to add modules used by sentry-service-server. This will create a sentry-service-metrics that will house and wrap all the metrics needs for the server. It will need to use > Seperate Metrics into sub module in order to remove circular dependencies > - > > Key: SENTRY-2462 > URL: https://issues.apache.org/jira/browse/SENTRY-2462 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > > The metrics subsystem currently resides in the sentry-service-server. So any > other module that needs to use it has to depend on sentry-service-server > which is causing circular dependencies when trying to add modules used by > sentry-service-server. > > This will create a sentry-service-metrics that will house and wrap all the > metrics needs for the server. It will need to use SENTRY-2458 as a base to > segment out the web presentation components of the metrics. It will also > allow for the centralization of the roll up and shading of the codehale > metrics package to happen in one sub module rather then split over two. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2462) Seperate Metrics into sub module in order to remove circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2462: - Description: The metrics subsystem currently resides in the sentry-service-server. So any other module that needs to use it has to depend on sentry-service-server which is causing circular dependencies when trying to add modules used by sentry-service-server. This will create a sentry-service-metrics that will house and wrap all the metrics needs for the server. It will need to use was:The metrics subsystem currently resides in the sentry-service-server. So any other module that needs to use it has to depend on sentry-service-server which is causing circular dependencies when trying to add modules used by sentry-service-server. > Seperate Metrics into sub module in order to remove circular dependencies > - > > Key: SENTRY-2462 > URL: https://issues.apache.org/jira/browse/SENTRY-2462 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > > The metrics subsystem currently resides in the sentry-service-server. So any > other module that needs to use it has to depend on sentry-service-server > which is causing circular dependencies when trying to add modules used by > sentry-service-server. > > This will create a sentry-service-metrics that will house and wrap all the > metrics needs for the server. It will need to use -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2462) Seperate Metrics into sub module in order to remove circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-2462: Assignee: Brian Towles > Seperate Metrics into sub module in order to remove circular dependencies > - > > Key: SENTRY-2462 > URL: https://issues.apache.org/jira/browse/SENTRY-2462 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > > The metrics subsystem currently resides in the sentry-service-server. So any > other module that needs to use it has to depend on sentry-service-server > which is causing circular dependencies when trying to add modules used by > sentry-service-server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2462) Seperate Metrics into sub module in order to remove circular dependencies
Brian Towles created SENTRY-2462: Summary: Seperate Metrics into sub module in order to remove circular dependencies Key: SENTRY-2462 URL: https://issues.apache.org/jira/browse/SENTRY-2462 Project: Sentry Issue Type: Improvement Components: Service Affects Versions: 2.0.1 Reporter: Brian Towles Fix For: 2.2.0 The metrics subsystem currently resides in the sentry-service-server. So any other module that needs to use it has to depend on sentry-service-server which is causing circular dependencies when trying to add modules used by sentry-service-server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Attachment: SENTRY-2444.002.patch > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch, SENTRY-2444.002.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Attachment: (was: SENTRY-2444.002.patch) > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch, SENTRY-2444.002.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Attachment: SENTRY-2444.002.patch > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch, SENTRY-2444.002.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Attachment: (was: SENTRY-2444.002.patch) > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch, SENTRY-2444.002.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Attachment: SENTRY-2444.002.patch > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch, SENTRY-2444.002.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.003.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch, > SENTRY-2458.003.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.002.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch, SENTRY-2458.002.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.001.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: (was: SENTRY-2458.001.patch) > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.001.patch > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: (was: SENTRY-2458.001.patch) > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
[ https://issues.apache.org/jira/browse/SENTRY-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2458: - Attachment: SENTRY-2458.001.patch Status: Patch Available (was: Open) > Separate Web UI and service from service-server to prevent circular > dependencies > > > Key: SENTRY-2458 > URL: https://issues.apache.org/jira/browse/SENTRY-2458 > Project: Sentry > Issue Type: Improvement > Components: Service >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.2.0 > > Attachments: SENTRY-2458.001.patch > > > In order to additional modules to be added to Sentry there needs to be a > separation of some of the features used by service-server into other modules. > This will allow the web server to be pulled out from the base server and > allow for additional modules to be able to add web services and functionality > to the web interface without depending up the whole server module. > It will use the SPI to dynamically include servlets and content from other > modules. > It creates a sentry-service-providers to allow for Server and sub modules SPI > providers to be defined externally and not depended on directly. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2458) Separate Web UI and service from service-server to prevent circular dependencies
Brian Towles created SENTRY-2458: Summary: Separate Web UI and service from service-server to prevent circular dependencies Key: SENTRY-2458 URL: https://issues.apache.org/jira/browse/SENTRY-2458 Project: Sentry Issue Type: Improvement Components: Service Affects Versions: 2.0.1 Reporter: Brian Towles Assignee: Brian Towles Fix For: 2.2.0 In order to additional modules to be added to Sentry there needs to be a separation of some of the features used by service-server into other modules. This will allow the web server to be pulled out from the base server and allow for additional modules to be able to add web services and functionality to the web interface without depending up the whole server module. It will use the SPI to dynamically include servlets and content from other modules. It creates a sentry-service-providers to allow for Server and sub modules SPI providers to be defined externally and not depended on directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Status: Patch Available (was: Open) This is a simple unregister of the function from the signal handler > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
[ https://issues.apache.org/jira/browse/SENTRY-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2444: - Attachment: SENTRY-2444.001.patch > SigUtils signal handler needs a way to unregister functions. > > > Key: SENTRY-2444 > URL: https://issues.apache.org/jira/browse/SENTRY-2444 > Project: Sentry > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: Brian Towles >Priority: Minor > Fix For: 2.2.0 > > Attachments: SENTRY-2444.001.patch > > > The SigUtil is able to register functions but needs a way to un-register > functions that no longer need to be called when the server process receives a > signal. This is necessary for some upcoming work that will unregister signal > handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2444) SigUtils signal handler needs a way to unregister functions.
Brian Towles created SENTRY-2444: Summary: SigUtils signal handler needs a way to unregister functions. Key: SENTRY-2444 URL: https://issues.apache.org/jira/browse/SENTRY-2444 Project: Sentry Issue Type: Improvement Affects Versions: 2.0.1 Reporter: Brian Towles Fix For: 2.2.0 The SigUtil is able to register functions but needs a way to un-register functions that no longer need to be called when the server process receives a signal. This is necessary for some upcoming work that will unregister signal handlers when not the HA master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2422) HMS syntonization is causing multiple entries of the same ID in SENTRY_HMS_NOTIFICATION_ID
Brian Towles created SENTRY-2422:
Summary: HMS syntonization is causing multiple entries of the same
ID in SENTRY_HMS_NOTIFICATION_ID
Key: SENTRY-2422
URL: https://issues.apache.org/jira/browse/SENTRY-2422
Project: Sentry
Issue Type: Improvement
Components: Core
Affects Versions: 2.0.1
Reporter: Brian Towles
When processing updates for the HMS notifications, multiple entries of the same
ID are being put into the SENTRY_HMS_NOTIFICATION_ID table.
This can be seen after doing a non location changing ALTER TABLE in hive for
example.
ALTER TABLE sometable SET TBLPROPERTIES ("someproperty"="somevalue");
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2367: - Attachment: SENTRY-2367.004.patch > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch, > SENTRY-2367.003.patch, SENTRY-2367.004.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2393) sentry-provider-db needs dependency cleanup since move to sentry-service-server
[ https://issues.apache.org/jira/browse/SENTRY-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-2393: Assignee: Brian Towles > sentry-provider-db needs dependency cleanup since move to > sentry-service-server > --- > > Key: SENTRY-2393 > URL: https://issues.apache.org/jira/browse/SENTRY-2393 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > > With the move to sentry-service-server module out of sentry-provider-db a lot > of unused dependencies were left in in sentry-provider-db. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2393) sentry-provider-db needs dependency cleanup since move to sentry-service-server
Brian Towles created SENTRY-2393: Summary: sentry-provider-db needs dependency cleanup since move to sentry-service-server Key: SENTRY-2393 URL: https://issues.apache.org/jira/browse/SENTRY-2393 Project: Sentry Issue Type: Improvement Components: Build Affects Versions: 2.0.1 Reporter: Brian Towles Fix For: 2.1.0 With the move to sentry-service-server module out of sentry-provider-db a lot of unused dependencies were left in in sentry-provider-db. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2367: - Attachment: SENTRY-2367.003.patch > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch, > SENTRY-2367.003.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598947#comment-16598947 ] Brian Towles commented on SENTRY-2367: -- Lombok dependency addition moved to SENTRY-2374 > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch, > SENTRY-2367.003.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2374) Add Lombok for easier development
[ https://issues.apache.org/jira/browse/SENTRY-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2374: - Attachment: SENTRY-2374.001.patch Status: Patch Available (was: Open) > Add Lombok for easier development > - > > Key: SENTRY-2374 > URL: https://issues.apache.org/jira/browse/SENTRY-2374 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Minor > Fix For: 2.1.0 > > Attachments: SENTRY-2374.001.patch > > > Add dependencies for the Lombok development libary -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2374) Add Lombok for easier development
[ https://issues.apache.org/jira/browse/SENTRY-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-2374: Assignee: Brian Towles > Add Lombok for easier development > - > > Key: SENTRY-2374 > URL: https://issues.apache.org/jira/browse/SENTRY-2374 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Minor > Fix For: 2.1.0 > > > Add dependencies for the Lombok development libary -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2140) Metadata Driven Column Masking
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-2140: Assignee: (was: Brian Towles) > Metadata Driven Column Masking > -- > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Priority: Major > Labels: ABAC > Attachments: Sentry ABAC Proposal v1.1.pdf, Sentry ABAC Proposal.pdf > > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "attributes" or other metadata that prevent users/roles from not accessing or > seeing the data. For users/roles that have that attribute, they should be > able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (SENTRY-2140) Metadata Driven Column Masking
[ https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-2140: Assignee: Brian Towles > Metadata Driven Column Masking > -- > > Key: SENTRY-2140 > URL: https://issues.apache.org/jira/browse/SENTRY-2140 > Project: Sentry > Issue Type: New Feature > Components: Core >Reporter: Steve Moist >Assignee: Brian Towles >Priority: Major > Labels: ABAC > Attachments: Sentry ABAC Proposal v1.1.pdf, Sentry ABAC Proposal.pdf > > > As a user, I want to have finer grain control over which users/roles can view > data in Hive. Some information such as Social Security Number is considered > very confidential information. I want to be able to tag columns in Hive with > "attributes" or other metadata that prevent users/roles from not accessing or > seeing the data. For users/roles that have that attribute, they should be > able to see that information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592125#comment-16592125 ] Brian Towles commented on SENTRY-2367: -- I don't believe that that would work. This is a simplified derivative work. The Keycloak version take into account the Keycloak configuration mechanisms and has more Keycloak specific functions as to how the Services get instantiated and when. The concept displayed in the Keycloak SPI has been stripped down significantly and made specificly to be a small clean module for usage by Sentry. There are only two concrete classes in the package and the only code of any significance in the package is ProviderManager and it is 104 lines long. Every thing else is just interfaces. It would not be depended on changes or things that are necessary to be tracked from Keycloak. > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591644#comment-16591644 ] Brian Towles edited comment on SENTRY-2367 at 8/24/18 1:25 PM: --- If there is additional Notices for attribution or modification needed I can put them in. was (Author: btowles): If there is additional Notices for attribution or modification I can put them in. > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591644#comment-16591644 ] Brian Towles commented on SENTRY-2367: -- If there is additional Notices for attribution or modification I can put them in. > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591615#comment-16591615 ] Brian Towles commented on SENTRY-2367: -- This was derived from [Keycloaks SPI implementation|https://github.com/keycloak/keycloak/tree/master/server-spi/src/main/java/org/keycloak/provider] which is licensed under a Apache 2.0 License with author attribution left in place. > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591608#comment-16591608 ] Brian Towles commented on SENTRY-2367: -- Yes this is to allow the easy addition of pluggable components by adding an additional jar and calling it by a simple name from config, in case of attribute providers in MDCM. This is very usable internal as well as a simple way to extend and make pluggable services within the application that might have different initialization and configuration needs but the same execution interface, as in the case of attribute distribution for MDCM. > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591018#comment-16591018 ] Brian Towles commented on SENTRY-2367: -- This is a subtask of the Metadata Driven Column Masking to allow for multiple pluggable attribute providers, broadcast transports and other parts of the MDCM to be implemented. It is done in prep for future commits of the MDCM in the next week or two that will make use of it. In an effort to keep the commits stand alone and easily reviewable this is a fairly independent part that can be looked and and implemented against, even outside of MDCM. > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2367: - Attachment: SENTRY-2367.002.patch > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch, SENTRY-2367.002.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[
https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590517#comment-16590517
]
Brian Towles edited comment on SENTRY-2367 at 8/23/18 4:54 PM:
---
This documentation is in the javadoc on the package info
This module is a way to simplify loading of different "plugin" service
implementations within the sentry. It allows for the easy creation of separate
Services and allows for multiple implementations of that service to be defined
and easily loaded. It is able to get all implementations or a single one
depending on configuration and need.
The Sentry API module takes advantage of the Java Service Loader which was
introduced in Java 7. It uses the Factory pattern in order to get concrete
instances of Services and allow for custom initialization of those concrete
instances.
h3. *Create an Service Instance:*
h3. Create concert classes for the Services Provider and ProviderFactory
interfaces.
In order to create a service instance you need to create instances of the
services Provider and ProviderFactory interfaces. These should contain all of
the necessary functions defined by the interface.
The _ProviderFactory_ instance needs to have the _getId()_ functioned defined
to return a unique name for the Service Provider instance so that it can be
looked up by that name.
The _create()_ function of the _ProviderFactory_ will return a concert instance
of the _Provider_
{{Sample
*src/main/resources/META-INF/services/org.apache.sentry.spi.FooSomeProvider*
file}}
{code:java}
package org.apache.sentry.fake;
public class FooSomeProvider implements SomeProvider {
public void doSomething(){
... does something ...
}
}
{code}
{{Sample
*src/main/resources/META-INF/services/org.apache.sentry.spi.FooSomeProviderFactory*
file}}
{code:java}
package org.apache.sentry.fake;
public class FooSomeProviderFactory implements SomeProviderFactory {
@Override
public String getId() {
return "foo"
}
@Override
public SomeProvider create() {
return new SomeProvider();
}
}
{code}
h3. Create an entry for the _ProviderFactory_ instance in the service
configuration file for the SPI
The service configuration file is placed in the *META-INF/services* directory
of a jar and is named after the _ProviderFactory_ instance of the SPI.
Sample
*src/main/resources/META-INF/services/org.apache.sentry.fake.SomeProviderFactory*
file
{code:java}
org.apache.sentry.fake.FooSomeProviderFactory{code}
h3.
Load the Service instance with the _ProviderManager_
You can then load the service from code with the ProviderManager. Either all
service providers or an individual one.
{code:java}
# Loads instances of all SomeProviderFactory defined
List = ProviderManager.getInstance().load("some-spi");
# Loads only the "foo" instance of the SomeProviderFactory
SomeProviderFactory someProviderFactory =
ProviderManager.getInstance().load("some-spi", "foo");
{code}
h2. How to Implement a New Service:
h3. Create an SPI implantation
You can create Service by implementing a concrete instance of the SPI
interface. This interface will provider information about what interfaces the
SPI will be looking for when loading instances. It requires the Provider and
the ProviderFactory information as well as a unique name for the Service in the
system.
Sample *src/main/java/org/apache/sentry/fake/SomeSpi.java* file
{code:java}
package org.apache.sentry.fake;
public class SomeSpi implements Spi {
@Override
public String getName() {
return "some-spi";
}
@Override
public Class getProviderClass() {
return SomeProvider.class;
}
@Override
public Class getProviderFactoryClass() {
return SomeProviderFactory.class;
}
}
{code}
As well you must put an entry for the SPI concrete class in the
*META-INF/services/org.apache.sentry.spi.Spi* service configuration file
pointing to that instance.
Sample *src/main/resources/META-INF/services/org.apache.sentry.spi.SomeSpi* file
{code:java}
org.apache.sentry.fake.SomeSpi
org.apache.sentry.fake.SomeOtherSpi{code}
h3.
Create a the Provider and Provider Factory interfaces
You need to create the interfaces referenced in the the SPI class. These extend
the Provider and ProviderFactory interfaces and can be customized to have the
function definitions for how you want your service to operate.
Sample *src/main/java/org/apache/sentry/fake/SomeProvider.java* file
{code:java}
package org.apache.sentry.fake;
public interface SomeProvider extends Provider {
void doSomething();
}
{code}
Sample *src/main/java/org/apache/sentry/fake/SomeProviderFactory.java* file
{code:java}
package org.apache.sentry.fake;
public interface SomeProviderFactory extends ProviderFactory {
void init(SomeConfig config);
}
{code}
was (Author: btowles):
This documentation is in the javadoc on the package info
This module is a way to simplify loading of different "plugin"
[jira] [Comment Edited] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[
https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590517#comment-16590517
]
Brian Towles edited comment on SENTRY-2367 at 8/23/18 4:54 PM:
---
This documentation is in the javadoc on the package info
This module is a way to simplify loading of different "plugin" service
implementations within the sentry. It allows for the easy creation of separate
Services and allows for multiple implementations of that service to be defined
and easily loaded. It is able to get all implementations or a single one
depending on configuration and need.
The Sentry API module takes advantage of the Java Service Loader which was
introduced in Java 7. It uses the Factory pattern in order to get concrete
instances of Services and allow for custom initialization of those concrete
instances.
h3. *Create an Service Instance:*
h3. Create concert classes for the Services Provider and ProviderFactory
interfaces.
In order to create a service instance you need to create instances of the
services Provider and ProviderFactory interfaces. These should contain all of
the necessary functions defined by the interface.
The _ProviderFactory_ instance needs to have the _getId()_ functioned defined
to return a unique name for the Service Provider instance so that it can be
looked up by that name.
The _create()_ function of the _ProviderFactory_ will return a concert instance
of the _Provider_
{{Sample
*src/main/resources/META-INF/services/org.apache.sentry.spi.FooSomeProvider*
file}}
{code:java}
package org.apache.sentry.fake;
public class FooSomeProvider implements SomeProvider {
public void doSomething(){
... does something ...
}
}
{code}
{{Sample
*src/main/resources/META-INF/services/org.apache.sentry.spi.FooSomeProviderFactory*
file}}
{code:java}
package org.apache.sentry.fake;
public class FooSomeProviderFactory implements SomeProviderFactory {
@Override
public String getId() {
return "foo"
}
@Override
public SomeProvider create() {
return new SomeProvider();
}
}
{code}
h3. Create an entry for the _ProviderFactory_ instance in the service
configuration file for the SPI
The service configuration file is placed in the *META-INF/services* directory
of a jar and is named after the _ProviderFactory_ instance of the SPI.
Sample
*src/main/resources/META-INF/services/org.apache.sentry.fake.SomeProviderFactory*
file
{code:java}
org.apache.sentry.fake.FooSomeProviderFactory{code}
h3.
Load the Service instance with the _ProviderManager_
You can then load the service from code with the ProviderManager. Either all
service providers or an individual one.
{code:java}
# Loads instances of all SomeProviderFactory defined
List = ProviderManager.getInstance().load("some-spi");
# Loads only the "foo" instance of the SomeProviderFactory
SomeProviderFactory someProviderFactory =
ProviderManager.getInstance().load("some-spi", "foo");
{code}
h2.
How to Implement a New Service:
h3. Create an SPI implantation
You can create Service by implementing a concrete instance of the SPI
interface. This interface will provider information about what interfaces the
SPI will be looking for when loading instances. It requires the Provider and
the ProviderFactory information as well as a unique name for the Service in the
system.
Sample *src/main/java/org/apache/sentry/fake/SomeSpi.java* file
{code:java}
package org.apache.sentry.fake;
public class SomeSpi implements Spi {
@Override
public String getName() {
return "some-spi";
}
@Override
public Class getProviderClass() {
return SomeProvider.class;
}
@Override
public Class getProviderFactoryClass() {
return SomeProviderFactory.class;
}
}
{code}
As well you must put an entry for the SPI concrete class in the
*META-INF/services/org.apache.sentry.spi.Spi* service configuration file
pointing to that instance.
Sample *src/main/resources/META-INF/services/org.apache.sentry.spi.SomeSpi* file
{code:java}
org.apache.sentry.fake.SomeSpi
org.apache.sentry.fake.SomeOtherSpi{code}
h3.
Create a the Provider and Provider Factory interfaces
You need to create the interfaces referenced in the the SPI class. These extend
the Provider and ProviderFactory interfaces and can be customized to have the
function definitions for how you want your service to operate.
Sample *src/main/java/org/apache/sentry/fake/SomeProvider.java* file
{code:java}
package org.apache.sentry.fake;
public interface SomeProvider extends Provider {
void doSomething();
}
{code}
Sample *src/main/java/org/apache/sentry/fake/SomeProviderFactory.java* file
{code:java}
package org.apache.sentry.fake;
public interface SomeProviderFactory extends ProviderFactory {
void init(SomeConfig config);
}
{code}
was (Author: btowles):
This module is a way to simplify loading of different "plugin" service
implementations within the sentry. It allows for the
[jira] [Commented] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[
https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590517#comment-16590517
]
Brian Towles commented on SENTRY-2367:
--
This module is a way to simplify loading of different "plugin" service
implementations within the sentry. It allows for the easy creation of separate
Services and allows for multiple implementations of that service to be defined
and easily loaded. It is able to get all implementations or a single one
depending on configuration and need.
The Sentry API module takes advantage of the Java Service Loader which was
introduced in Java 7. It uses the Factory pattern in order to get concrete
instances of Services and allow for custom initialization of those concrete
instances.
h3. *Create an Service Instance:*
h3. Create concert classes for the Services Provider and ProviderFactory
interfaces.
In order to create a service instance you need to create instances of the
services Provider and ProviderFactory interfaces. These should contain all of
the necessary functions defined by the interface.
The _ProviderFactory_ instance needs to have the _getId()_ functioned defined
to return a unique name for the Service Provider instance so that it can be
looked up by that name.
The _create()_ function of the _ProviderFactory_ will return a concert instance
of the _Provider_
{{Sample
*src/main/resources/META-INF/services/org.apache.sentry.spi.FooSomeProvider*
file}}
{code:java}
package org.apache.sentry.fake;
public class FooSomeProvider implements SomeProvider {
public void doSomething(){
... does something ...
}
}
{code}
{{Sample
*src/main/resources/META-INF/services/org.apache.sentry.spi.FooSomeProviderFactory*
file}}
{code:java}
package org.apache.sentry.fake;
public class FooSomeProviderFactory implements SomeProviderFactory {
@Override
public String getId() {
return "foo"
}
@Override
public SomeProvider create() {
return new SomeProvider();
}
}
{code}
h3. Create an entry for the _ProviderFactory_ instance in the service
configuration file for the SPI
The service configuration file is placed in the *META-INF/services* directory
of a jar and is named after the _ProviderFactory_ instance of the SPI.
Sample
*src/main/resources/META-INF/services/org.apache.sentry.fake.SomeProviderFactory*
file
{code:java}
org.apache.sentry.fake.FooSomeProviderFactory{code}
h3.
Load the Service instance with the _ProviderManager_
You can then load the service from code with the ProviderManager. Either all
service providers or an individual one.
{code:java}
# Loads instances of all SomeProviderFactory defined
List = ProviderManager.getInstance().load("some-spi");
# Loads only the "foo" instance of the SomeProviderFactory
SomeProviderFactory someProviderFactory =
ProviderManager.getInstance().load("some-spi", "foo");
{code}
h2.
How to Implement a New Service:
h3. Create an SPI implantation
You can create Service by implementing a concrete instance of the SPI
interface. This interface will provider information about what interfaces the
SPI will be looking for when loading instances. It requires the Provider and
the ProviderFactory information as well as a unique name for the Service in the
system.
Sample *src/main/java/org/apache/sentry/fake/SomeSpi.java* file
{code:java}
package org.apache.sentry.fake;
public class SomeSpi implements Spi {
@Override
public String getName() {
return "some-spi";
}
@Override
public Class getProviderClass() {
return SomeProvider.class;
}
@Override
public Class getProviderFactoryClass() {
return SomeProviderFactory.class;
}
}
{code}
As well you must put an entry for the SPI concrete class in the
*META-INF/services/org.apache.sentry.spi.Spi* service configuration file
pointing to that instance.
Sample *src/main/resources/META-INF/services/org.apache.sentry.spi.SomeSpi* file
{code:java}
org.apache.sentry.fake.SomeSpi
org.apache.sentry.fake.SomeOtherSpi{code}
h3.
Create a the Provider and Provider Factory interfaces
You need to create the interfaces referenced in the the SPI class. These extend
the Provider and ProviderFactory interfaces and can be customized to have the
function definitions for how you want your service to operate.
Sample *src/main/java/org/apache/sentry/fake/SomeProvider.java* file
{code:java}
package org.apache.sentry.fake;
public interface SomeProvider extends Provider {
void doSomething();
}
{code}
Sample *src/main/java/org/apache/sentry/fake/SomeProviderFactory.java* file
{code:java}
package org.apache.sentry.fake;
public interface SomeProviderFactory extends ProviderFactory {
void init(SomeConfig config);
}
{code}
> Implement subsystem to allow for pluggable attribute providers and transports
> -
>
> Key: SENTRY-2367
> URL:
[jira] [Updated] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2367: - Fix Version/s: 2.1.0 Affects Version/s: 2.0.1 Status: Patch Available (was: In Progress) > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2367.001.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
[ https://issues.apache.org/jira/browse/SENTRY-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2367: - Attachment: SENTRY-2367.001.patch > Implement subsystem to allow for pluggable attribute providers and transports > - > > Key: SENTRY-2367 > URL: https://issues.apache.org/jira/browse/SENTRY-2367 > Project: Sentry > Issue Type: Sub-task > Components: Core >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Attachments: SENTRY-2367.001.patch > > > Implement a subsystem for Sentry to for the pluggable loading of attribute > providers and transports. This will be done with the Java SPI interface and > mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2367) Implement subsystem to allow for pluggable attribute providers and transports
Brian Towles created SENTRY-2367: Summary: Implement subsystem to allow for pluggable attribute providers and transports Key: SENTRY-2367 URL: https://issues.apache.org/jira/browse/SENTRY-2367 Project: Sentry Issue Type: Sub-task Components: Core Reporter: Brian Towles Assignee: Brian Towles Implement a subsystem for Sentry to for the pluggable loading of attribute providers and transports. This will be done with the Java SPI interface and mechanisms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2335) Allow multiple callbacks to be run when a Signal is received.
[ https://issues.apache.org/jira/browse/SENTRY-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2335: - Attachment: SENTRY-2335.02.patch > Allow multiple callbacks to be run when a Signal is received. > - > > Key: SENTRY-2335 > URL: https://issues.apache.org/jira/browse/SENTRY-2335 > Project: Sentry > Issue Type: Improvement > Components: Core >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Minor > Fix For: 2.1.0 > > Attachments: SENTRY-2335.01.patch, SENTRY-2335.02.patch > > > As Sentry develops, more and more items possibly need to react to signals. To > do things like reload config files without restarting the service, a signal > (like SIGHUP) is ideal. Currently Sentry only allows for one function to be > called when a signal is received, but different parts of the system may need > to react to that same signal. > This is a small change to the SigUtils to allow for multiple handlers for a > single signal. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2335) Allow multiple callbacks to be run when a Signal is received.
[ https://issues.apache.org/jira/browse/SENTRY-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2335: - Fix Version/s: (was: 2.0.1) 2.1.0 > Allow multiple callbacks to be run when a Signal is received. > - > > Key: SENTRY-2335 > URL: https://issues.apache.org/jira/browse/SENTRY-2335 > Project: Sentry > Issue Type: Improvement > Components: Core >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Minor > Fix For: 2.1.0 > > Attachments: SENTRY-2335.01.patch > > > As Sentry develops, more and more items possibly need to react to signals. To > do things like reload config files without restarting the service, a signal > (like SIGHUP) is ideal. Currently Sentry only allows for one function to be > called when a signal is received, but different parts of the system may need > to react to that same signal. > This is a small change to the SigUtils to allow for multiple handlers for a > single signal. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2335) Allow multiple callbacks to be run when a Signal is received.
[ https://issues.apache.org/jira/browse/SENTRY-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2335: - Fix Version/s: 2.0.1 Affects Version/s: 2.0.1 Status: Patch Available (was: In Progress) > Allow multiple callbacks to be run when a Signal is received. > - > > Key: SENTRY-2335 > URL: https://issues.apache.org/jira/browse/SENTRY-2335 > Project: Sentry > Issue Type: Improvement > Components: Core >Affects Versions: 2.0.1 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Minor > Fix For: 2.0.1 > > Attachments: SENTRY-2335.01.patch > > > As Sentry develops, more and more items possibly need to react to signals. To > do things like reload config files without restarting the service, a signal > (like SIGHUP) is ideal. Currently Sentry only allows for one function to be > called when a signal is received, but different parts of the system may need > to react to that same signal. > This is a small change to the SigUtils to allow for multiple handlers for a > single signal. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2335) Allow multiple callbacks to be run when a Signal is received.
Brian Towles created SENTRY-2335: Summary: Allow multiple callbacks to be run when a Signal is received. Key: SENTRY-2335 URL: https://issues.apache.org/jira/browse/SENTRY-2335 Project: Sentry Issue Type: Improvement Components: Core Reporter: Brian Towles Assignee: Brian Towles As Sentry develops, more and more items possibly need to react to signals. To do things like reload config files without restarting the service, a signal (like SIGHUP) is ideal. Currently Sentry only allows for one function to be called when a signal is received, but different parts of the system may need to react to that same signal. This is a small change to the SigUtils to allow for multiple handlers for a single signal. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2225) Generic Attribute Ingestion and Default Implementation
[ https://issues.apache.org/jira/browse/SENTRY-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472570#comment-16472570 ] Brian Towles commented on SENTRY-2225: -- Its not really about re-storage of attributes. This is about collection of attributes (not persistence in the long term) for use in other parts of Sentry. So lets step back a bit and look at what an ABAC system needs, not just for the column masking subsystem. I think we have a bit of a disconnect on the relation of attributes to things like access control and column masking. The ability for a user to tie different sources of attributes into their access control decisions and data presentation systems is very powerful and desired. The [Guide to Attribute Based Access Control (ABAC) Definition and Considerations|https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-162.pdf] talks about this and is a good reference. One of the key parts of this is that the Policy Information Point (PIP) is about the retrieval of attributes and not the system of record for those attributes and the attributes themselves could be volatile and potentially change frequently. For the general collection and aggregation of attributes I do not feel that the permanent storage of attributes is needed or desired. What the PIP side of this becomes is a attribute collection, aggregation, and distribution source but not outright persistence. It collects the attributes from multiple sources based on the plugins developed for attribute collection. It aggregates this into a central attribute collection that it is able to distribute to the bindings or Policy Decision Points (PDP). This allows the access control engine or column masking decision engine to have access to the attributes where the decision is being made for speed and easy lookup. The different attribute collection mechanisms would allow attributes that have a different valid life spans to be taken into account. User or Subject level attributes might have a different valid life span then HMS Object data. It allows for the ability to implement different collection patterns that may need to be put into place, push versus pull as needed by the system providing the attributes. Implementation of attribute collection can be done with a pluggable solution using Java Service Loader SPI framework to allow for multiple attribute collection sources at one time. It would also provide set interfaces and loading mechanism for end user custom provided attribute loading mechanisms as well as additional attribute collection that could come from the project. Once attributes are ingested, column masking and other components in the system can make use of the attributes in order to make decisions. The aggregated set of attributes can be distributed to the server or bindings, what ever is being the decision point and allow for quick local access quickly. The way I see it the initial sources can be the HMS Table Properties I mentioned above as well as the Static File mechanism that could be used more for development and testing. > Generic Attribute Ingestion and Default Implementation > -- > > Key: SENTRY-2225 > URL: https://issues.apache.org/jira/browse/SENTRY-2225 > Project: Sentry > Issue Type: Sub-task > Components: Core >Affects Versions: 2.1.0 >Reporter: Anthony Young-Garner >Priority: Major > Labels: ABAC > > As discussed in the design document linked on SENTRY-2140, attributes and > their mapping to columns are created and stored in an external system. In > order for Sentry to make masking decisions based upon these attributes and > mappings, this information must be ingested from the external system. The > scope of this Jira is to : > # implement the generic extensible framework by which different external > systems may contribute attributes (the specific details of the design are > still under discussion on the parent Jira; whether there is a full plugin > model implemented in Sentry or whether the ingestion process will run > entirely external to Sentry and send the information to Sentry via Thrift API > is not yet decided). > # Implement at least one default implementation (whether this will be an > example implementation only for reference like a static text file or a > full-featured implementation more suitable for production use is under > discussion) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2225) Generic Attribute Ingestion and Default Implementation
[
https://issues.apache.org/jira/browse/SENTRY-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471348#comment-16471348
]
Brian Towles commented on SENTRY-2225:
--
The default llocation we can get attributes from can be HMS rather then a
static file.
HMS has the ability to do Table Properties as a free form key value store. We
can very easily set table and column level attributes in the properties using a
name-spaced approach.
e.g.
sentry.attributes.table.tag = Foo // table level "tag" attribute
sentry.attributes.column.ccnum.label = "PCI,PII" // column level "*label*"
attribute for ccnum column
sentry.attributes.column.ccnum.content-descriptor = "credit_card_number" //
column level "content-descriptor" attribute for ccnum column
The current structure for TABLE_PARAMS in HMS is
{noformat}
`TBL_ID` bigint(20) NOT NULL,
`PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin DEFAULT
NULL,{noformat}
So there is a very large amount of available space for nested structures (like
JSON) as a param value if necessary.
Entry of these properties is easy and done right in SQL with commands like:
*ALTER TABLE table_name SET TBLPROPERTIES ("propertykey" = "property value");*
In the plugin we will be able to respond to HMS changed events from HMSFollower
and populate out the attributes as needed. Or it could implement its own
retrieval method for attributes based on changes happening in HMS, push or
pull. HMS Follower could send notifications of objects changed. Right now it
uses PubSub to do, but that only sends a string. We could augment or replace
with guavas EventBus and have full EventObject passing internal to Sentry. The
messages that HMSFollower gets from include changes to the Table Properties
when they occur and can be used to update attributes on creates, changes, or
removals.
> Generic Attribute Ingestion and Default Implementation
> --
>
> Key: SENTRY-2225
> URL: https://issues.apache.org/jira/browse/SENTRY-2225
> Project: Sentry
> Issue Type: Sub-task
> Components: Core
>Affects Versions: 2.1.0
>Reporter: Anthony Young-Garner
>Priority: Major
> Labels: ABAC
>
> As discussed in the design document linked on SENTRY-2140, attributes and
> their mapping to columns are created and stored in an external system. In
> order for Sentry to make masking decisions based upon these attributes and
> mappings, this information must be ingested from the external system. The
> scope of this Jira is to :
> # implement the generic extensible framework by which different external
> systems may contribute attributes (the specific details of the design are
> still under discussion on the parent Jira; whether there is a full plugin
> model implemented in Sentry or whether the ingestion process will run
> entirely external to Sentry and send the information to Sentry via Thrift API
> is not yet decided).
> # Implement at least one default implementation (whether this will be an
> example implementation only for reference like a static text file or a
> full-featured implementation more suitable for production use is under
> discussion)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2074: - Attachment: SENTRY-2074.04.patch > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2074.01.patch, SENTRY-2074.02.patch, > SENTRY-2074.03.patch, SENTRY-2074.04.patch > > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. > > The one of the primary motivations for this patch is to help on its way to > cleaning up the distribution. Currently the dist module reads all > dependencies no mater what scope they are and drops them into the distributed > libs. This causes things like junit and ant to be pushed into the libs that > are being distributed. With the changes to have direct dependencies always > defined it allows us to take compile and runtime scopes only into account > when dropping the libs needed. > As well this identifies which libraries are provided already by environments > where the plugins/bindings are going into. For example in the hive bindings, > the hive and hadoop libraries need only be defined with "provided" scope, > since with those application we want to use the hadoop and hive libraries > that the applications already provide. > This makes it a lot easier for shading and package shifting of the binding > and plugins for libraries and versions of those libraries that are needed by > the binding and might conflict with versions already in the application which > the binding or plugin is going into. Guava is a major issue with this. Doing > this short of shading based on the cleanup would allow us to rev Guava and > use newer Guava features while not conflicting with the Guava version the > application is using. By having the directly used dependency defined it gives > us control over the exact version we are using and not be dependent on and > having conflicts with the transitive dependencies of the application being > embedded in. > This patch will not really make the development process harder since the > analysis of the dependencies needed automatically runs as part of the build > and a failure occurs telling you which "used but undefined" and which > "defined but unused" libraries are missing or in the pom. There is even an > xml dump of the dependencies part need to put right into the pom. No > additional runs or dependency analysis needs to take place. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2074: - Attachment: SENTRY-2074.03.patch > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2074.01.patch, SENTRY-2074.02.patch, > SENTRY-2074.03.patch > > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. > > The one of the primary motivations for this patch is to help on its way to > cleaning up the distribution. Currently the dist module reads all > dependencies no mater what scope they are and drops them into the distributed > libs. This causes things like junit and ant to be pushed into the libs that > are being distributed. With the changes to have direct dependencies always > defined it allows us to take compile and runtime scopes only into account > when dropping the libs needed. > As well this identifies which libraries are provided already by environments > where the plugins/bindings are going into. For example in the hive bindings, > the hive and hadoop libraries need only be defined with "provided" scope, > since with those application we want to use the hadoop and hive libraries > that the applications already provide. > This makes it a lot easier for shading and package shifting of the binding > and plugins for libraries and versions of those libraries that are needed by > the binding and might conflict with versions already in the application which > the binding or plugin is going into. Guava is a major issue with this. Doing > this short of shading based on the cleanup would allow us to rev Guava and > use newer Guava features while not conflicting with the Guava version the > application is using. By having the directly used dependency defined it gives > us control over the exact version we are using and not be dependent on and > having conflicts with the transitive dependencies of the application being > embedded in. > This patch will not really make the development process harder since the > analysis of the dependencies needed automatically runs as part of the build > and a failure occurs telling you which "used but undefined" and which > "defined but unused" libraries are missing or in the pom. There is even an > xml dump of the dependencies part need to put right into the pom. No > additional runs or dependency analysis needs to take place. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2074: - Description: Using the maven dependency plugin to analyze the dependency usage for each module and put all directly used libraries in the pom. Clean-up the unused ones and adjust the scope of libraries only used for tests and provided libraries for plugins. The one of the primary motivations for this patch is to help on its way to cleaning up the distribution. Currently the dist module reads all dependencies no mater what scope they are and drops them into the distributed libs. This causes things like junit and ant to be pushed into the libs that are being distributed. With the changes to have direct dependencies always defined it allows us to take compile and runtime scopes only into account when dropping the libs needed. As well this identifies which libraries are provided already by environments where the plugins/bindings are going into. For example in the hive bindings, the hive and hadoop libraries need only be defined with "provided" scope, since with those application we want to use the hadoop and hive libraries that the applications already provide. This makes it a lot easier for shading and package shifting of the binding and plugins for libraries and versions of those libraries that are needed by the binding and might conflict with versions already in the application which the binding or plugin is going into. Guava is a major issue with this. Doing this short of shading based on the cleanup would allow us to rev Guava and use newer Guava features while not conflicting with the Guava version the application is using. By having the directly used dependency defined it gives us control over the exact version we are using and not be dependent on and having conflicts with the transitive dependencies of the application being embedded in. This patch will not really make the development process harder since the analysis of the dependencies needed automatically runs as part of the build and a failure occurs telling you which "used but undefined" and which "defined but unused" libraries are missing or in the pom. There is even an xml dump of the dependencies part need to put right into the pom. No additional runs or dependency analysis needs to take place. was:Using the maven dependency plugin to analyze the dependency usage for each module and put all directly used libraries in the pom. Clean-up the unused ones and adjust the scope of libraries only used for tests and provided libraries for plugins. > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2074.01.patch, SENTRY-2074.02.patch > > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. > > The one of the primary motivations for this patch is to help on its way to > cleaning up the distribution. Currently the dist module reads all > dependencies no mater what scope they are and drops them into the distributed > libs. This causes things like junit and ant to be pushed into the libs that > are being distributed. With the changes to have direct dependencies always > defined it allows us to take compile and runtime scopes only into account > when dropping the libs needed. > As well this identifies which libraries are provided already by environments > where the plugins/bindings are going into. For example in the hive bindings, > the hive and hadoop libraries need only be defined with "provided" scope, > since with those application we want to use the hadoop and hive libraries > that the applications already provide. > This makes it a lot easier for shading and package shifting of the binding > and plugins for libraries and versions of those libraries that are needed by > the binding and might conflict with versions already in the application which > the binding or plugin is going into. Guava is a major issue with this. Doing > this short of shading based on the cleanup would allow us to rev Guava and > use newer Guava features while not conflicting with the Guava version the > application is using. By having the directly used dependency defined it gives > us control over the exact version we are using and not be dependent on
[jira] [Updated] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2074: - Attachment: SENTRY-2074.02.patch > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2074.01.patch, SENTRY-2074.02.patch > > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2074: - Fix Version/s: 2.1.0 Status: Patch Available (was: Open) > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Fix For: 2.1.0 > > Attachments: SENTRY-2074.01.patch > > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-2074: - Attachment: SENTRY-2074.01.patch > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > Attachments: SENTRY-2074.01.patch > > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
[ https://issues.apache.org/jira/browse/SENTRY-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329335#comment-16329335 ] Brian Towles commented on SENTRY-2074: -- I have created a patch that has all of the direct dependency libraries explicitly defined. There is also some minor cleanup on ThreadSafe annotations to make it from a consistent package (this is only used by IDEs for checks anyways). As well I added a check in the poms to fail a build if it discovers any direct use dependencies that are not defined in the pom. > Fix maven dependencies to have all directly used libraries defined > -- > > Key: SENTRY-2074 > URL: https://issues.apache.org/jira/browse/SENTRY-2074 > Project: Sentry > Issue Type: Improvement > Components: Build >Affects Versions: 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles >Priority: Major > > Using the maven dependency plugin to analyze the dependency usage for each > module and put all directly used libraries in the pom. Clean-up the unused > ones and adjust the scope of libraries only used for tests and provided > libraries for plugins. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (SENTRY-2116) Allow Data Nucleus connection pooling library to be changed via its normal system properties
Brian Towles created SENTRY-2116: Summary: Allow Data Nucleus connection pooling library to be changed via its normal system properties Key: SENTRY-2116 URL: https://issues.apache.org/jira/browse/SENTRY-2116 Project: Sentry Issue Type: Bug Components: Core Affects Versions: 2.0.0 Reporter: Brian Towles Data Nucleus allows multiple different connection pooling libraries to be used, such as DBCP, C3P0, BoneCP, and HikariCP. Currently we hard code the connection pool library to be used by setting the _datanucleus.connectionPoolingType_ property to be only BoneCP. We are not allowing for any of the custom overrides via system properties to be passed in or used. We do this by passing only override properties into the _JDOPersistenceManagerFactory.getPersistenceManagerFactory_. To allow for flexibility during implementation as well as to possibly get around bugs in the default connection pool lib selected we should allow for the end user to be able to set the connection pool via the standard Data Nucleus system properties if necessary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SENTRY-2090) Move license information downloads to mvn package phase
[
https://issues.apache.org/jira/browse/SENTRY-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278957#comment-16278957
]
Brian Towles commented on SENTRY-2090:
--
package runs when install is run as well.
https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#Lifecycle_Reference
We can put in a profile for a dist build if we want.
> Move license information downloads to mvn package phase
> ---
>
> Key: SENTRY-2090
> URL: https://issues.apache.org/jira/browse/SENTRY-2090
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
>Affects Versions: 2.0.0
>Reporter: Sergio Peña
>
> I noticed that license information is downloaded on newer developer
> environments everytime I run {{mvn install}}:
> {noformat}
> [INFO]
>
>
>
> [INFO] Building Sentry Distribution 2.1.0-SNAPSHOT
>
>
> [INFO]
>
>
>
> [INFO]
>
>
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ sentry-dist ---
>
>
> [INFO] Deleting /home/sergio/Development/Apache/sentry/sentry-dist/target
>
>
> [INFO]
>
>
> [INFO] >>> maven-pmd-plugin:3.7:check (validate) > :pmd @ sentry-dist >>>
>
>
> [INFO]
>
>
> [INFO] --- maven-pmd-plugin:3.7:pmd (pmd) @ sentry-dist ---
>
>
> [INFO]
>
>
> [INFO] <<< maven-pmd-plugin:3.7:check (validate) < :pmd @ sentry-dist <<<
>
>
> [INFO]
>
>
> [INFO]
>
>
> [INFO] --- maven-pmd-plugin:3.7:check (validate) @ sentry-dist ---
>
>
> [INFO]
>
>
> [INFO]
>
>
> [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ sentry-dist
> ---
>
> [INFO]
>
[jira] [Created] (SENTRY-2074) Fix maven dependencies to have all directly used libraries defined
Brian Towles created SENTRY-2074: Summary: Fix maven dependencies to have all directly used libraries defined Key: SENTRY-2074 URL: https://issues.apache.org/jira/browse/SENTRY-2074 Project: Sentry Issue Type: Improvement Components: Build Affects Versions: 2.0.0 Reporter: Brian Towles Assignee: Brian Towles Using the maven dependency plugin to analyze the dependency usage for each module and put all directly used libraries in the pom. Clean-up the unused ones and adjust the scope of libraries only used for tests and provided libraries for plugins. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SENTRY-2030) Enable first-level cache for DataNucleus
[ https://issues.apache.org/jira/browse/SENTRY-2030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227267#comment-16227267 ] Brian Towles commented on SENTRY-2030: -- I think thats reasonable. My main concern would be synchronization in an HA deployment. But honestly I dont feel that nodes being different by up to a minute or so is a major issue, As long as a reasonable TTL is put in place for the cache. What about making it a configurable option that defaults to enabled? > Enable first-level cache for DataNucleus > > > Key: SENTRY-2030 > URL: https://issues.apache.org/jira/browse/SENTRY-2030 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov > > Sentry is using a new persistence manager instance per transaction and > repeatable-read transaction isolation level (except for Oracle). I think we > can safely enable first-level caching (which is at the PersistenceManager > level) which should improve performance without introducing any issues. > [~spena] [~kkalyan] [~btowles][~lina.li] What do you think? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SENTRY-2016) Change default web port to something closer inline to other Sentry ports
Brian Towles created SENTRY-2016: Summary: Change default web port to something closer inline to other Sentry ports Key: SENTRY-2016 URL: https://issues.apache.org/jira/browse/SENTRY-2016 Project: Sentry Issue Type: Improvement Reporter: Brian Towles Priority: Trivial The current web port is 29000 and was set by SENTRY-914. The problem is that the ports are fairly separated and does not allow for a simple port range passthrough for firewalls or network access control lists. If we change the default web-port to be 8039 it would be contiguous with the thrift port it will allow for easier network implementations and consistent grouping. Something like: Default Thrift Port: 8038 Default Web Port: 8039 This is still in unassigned IANA range of 8035-8039 tcp,udp (https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=unassigned=4). And any furture ports could go down to 8035 As well this would still be overridable in the normal way incase of collisions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.013.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch, SENTRY-1918.008.patch, SENTRY-1918.009.patch, > SENTRY-1918.010.patch, SENTRY-1918.011.patch, SENTRY-1918.012.patch, > SENTRY-1918.013.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.011.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch, SENTRY-1918.008.patch, SENTRY-1918.009.patch, > SENTRY-1918.010.patch, SENTRY-1918.011.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157615#comment-16157615 ] Brian Towles commented on SENTRY-1918: -- I have updated this to SentryStateBank which allows us to do state from any component we want. It uses binary states right now but we can change this implementation later if need be. > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch, SENTRY-1918.008.patch, SENTRY-1918.009.patch, > SENTRY-1918.010.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.010.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch, SENTRY-1918.008.patch, SENTRY-1918.009.patch, > SENTRY-1918.010.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.009.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch, SENTRY-1918.008.patch, SENTRY-1918.009.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.008.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch, SENTRY-1918.008.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.007.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch, > SENTRY-1918.007.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.006.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch, SENTRY-1918.006.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: (was: SENTRY-1919.001.patch) > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.005.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1918.005.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.004.patch Adding patch to test pre-commit build applicaiton of patches > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1918.004.patch, SENTRY-1919.001.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (SENTRY-1901) sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist anymore
[ https://issues.apache.org/jira/browse/SENTRY-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles resolved SENTRY-1901. -- Resolution: Done Fix Version/s: 2.0.0 > sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist > anymore > -- > > Key: SENTRY-1901 > URL: https://issues.apache.org/jira/browse/SENTRY-1901 > Project: Sentry > Issue Type: Bug > Components: Build >Affects Versions: 1.8.0, 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles > Fix For: 2.0.0 > > > First we do not have a stable version we are running tests against. We are > trying to run the tests against is 2.0.0-SNAPSHOT version. So im not sure > where the run against 2.0.0-SNAPSHOT is coming from. Does anyone know which > version of sqoop we should be testing against? > As well its downloading it into a thirdparty directory that is directly under > the project but not in the target. it is being excluded from the git buy the > .gitignore and is not cleaned up between builds. I assume this is to save > time on the downloads between builds. This is causing the hardcoded snapshot > build to be available for older source trees but not newer ones since the > SNAPSHOT version requested is no longer available. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.003.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1918.003.patch, > SENTRY-1919.001.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1918.002.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1918.002.patch, SENTRY-1919.001.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Fix Version/s: 2.0.0 Status: Patch Available (was: Open) > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Fix For: 2.0.0 > > Attachments: SENTRY-1919.001.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1918: - Attachment: SENTRY-1919.001.patch > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > Attachments: SENTRY-1919.001.patch > > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154603#comment-16154603 ] Brian Towles commented on SENTRY-1918: -- Im going to add a function to the SentryService in order to pass through a check to the HMSFollower if a FullSnapshot is running. As well the DBUpdateForwarder will check the SentryService (getting it from changes to the SentryServiceFactory) to see if the Full Snapshot is running before returning a full snapshot. If a Full Snapshot is running then it will return an empty result until the full snapshot is done. > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SENTRY-1921) Make SentryServiceFactory.create static and all calling instances use the static call
Brian Towles created SENTRY-1921: Summary: Make SentryServiceFactory.create static and all calling instances use the static call Key: SENTRY-1921 URL: https://issues.apache.org/jira/browse/SENTRY-1921 Project: Sentry Issue Type: Improvement Components: Service Affects Versions: 2.0.0 Reporter: Brian Towles Priority: Minor The SentryServiceFactory.create call is not a static and all the references to the factory end up creating a new instance of the factory. This should be turned into a static call. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SENTRY-1918) NN snapshot should not be served while HMS snapshot is collected
[ https://issues.apache.org/jira/browse/SENTRY-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-1918: Assignee: Brian Towles > NN snapshot should not be served while HMS snapshot is collected > > > Key: SENTRY-1918 > URL: https://issues.apache.org/jira/browse/SENTRY-1918 > Project: Sentry > Issue Type: Bug > Components: Sentry >Affects Versions: 2.0.0 >Reporter: Alexander Kolbasov >Assignee: Brian Towles > > While Sentry collects full HMS snapshot, Namenode still sends requests to > sentry and may request a full snapshot which is useless because a new one > will be available shortly. It is important to prevent this from happening so > that we do not collect two snapshots in memory at the same time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-967) Use the Maven Dependency Plugin to download artifacts for the Sqoop tests
[ https://issues.apache.org/jira/browse/SENTRY-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-967: Attachment: SENTRY-967-003.patch > Use the Maven Dependency Plugin to download artifacts for the Sqoop tests > - > > Key: SENTRY-967 > URL: https://issues.apache.org/jira/browse/SENTRY-967 > Project: Sentry > Issue Type: Improvement > Components: Test >Reporter: Colm O hEigeartaigh >Assignee: Brian Towles >Priority: Trivial > Fix For: 2.0.0 > > Attachments: SENTRY-967-002.patch, SENTRY-967-003.patch, > SENTRY-967.patch > > > Currently the Maven antrun plugin is used to download a copy of Tomcat (6) > and the Sqoop server to a "thirdparty" directory in the > "sentry-tests/sentry-tests-sqoop" directory. > However there are a number of problems with this approach: > a) You are not downloading to a temporary directory (I guess to save having > to download the artifacts on every build). This is not good practice, as a > "mvn clean" doesn't actually clean this directory > b) If the download is corrupted, the corrupted artifact will stay in > "thirdparty", as the directory is not removed as part of the build process. > c) The antrun/bash approach means that bash must be installed on the machine, > which might be problematic (e.g. on windows) > A better approach is to use the Maven dependency plugin to download the > artifacts and copy them to a directory in the "target" directory of the > tests. The artifacts get downloaded to the local maven repository, so after > the first download, the build won't have to redownload anything. It should > work on every platform as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-967) Use the Maven Dependency Plugin to download artifacts for the Sqoop tests
[ https://issues.apache.org/jira/browse/SENTRY-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-967: Attachment: SENTRY-967-002.patch Patch for updated codebase > Use the Maven Dependency Plugin to download artifacts for the Sqoop tests > - > > Key: SENTRY-967 > URL: https://issues.apache.org/jira/browse/SENTRY-967 > Project: Sentry > Issue Type: Improvement > Components: Test >Reporter: Colm O hEigeartaigh >Assignee: Brian Towles >Priority: Trivial > Fix For: 2.0.0 > > Attachments: SENTRY-967-002.patch, SENTRY-967.patch > > > Currently the Maven antrun plugin is used to download a copy of Tomcat (6) > and the Sqoop server to a "thirdparty" directory in the > "sentry-tests/sentry-tests-sqoop" directory. > However there are a number of problems with this approach: > a) You are not downloading to a temporary directory (I guess to save having > to download the artifacts on every build). This is not good practice, as a > "mvn clean" doesn't actually clean this directory > b) If the download is corrupted, the corrupted artifact will stay in > "thirdparty", as the directory is not removed as part of the build process. > c) The antrun/bash approach means that bash must be installed on the machine, > which might be problematic (e.g. on windows) > A better approach is to use the Maven dependency plugin to download the > artifacts and copy them to a directory in the "target" directory of the > tests. The artifacts get downloaded to the local maven repository, so after > the first download, the build won't have to redownload anything. It should > work on every platform as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SENTRY-967) Use the Maven Dependency Plugin to download artifacts for the Sqoop tests
[ https://issues.apache.org/jira/browse/SENTRY-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-967: --- Assignee: Brian Towles (was: Colm O hEigeartaigh) > Use the Maven Dependency Plugin to download artifacts for the Sqoop tests > - > > Key: SENTRY-967 > URL: https://issues.apache.org/jira/browse/SENTRY-967 > Project: Sentry > Issue Type: Improvement > Components: Test >Reporter: Colm O hEigeartaigh >Assignee: Brian Towles >Priority: Trivial > Fix For: 2.0.0 > > Attachments: SENTRY-967.patch > > > Currently the Maven antrun plugin is used to download a copy of Tomcat (6) > and the Sqoop server to a "thirdparty" directory in the > "sentry-tests/sentry-tests-sqoop" directory. > However there are a number of problems with this approach: > a) You are not downloading to a temporary directory (I guess to save having > to download the artifacts on every build). This is not good practice, as a > "mvn clean" doesn't actually clean this directory > b) If the download is corrupted, the corrupted artifact will stay in > "thirdparty", as the directory is not removed as part of the build process. > c) The antrun/bash approach means that bash must be installed on the machine, > which might be problematic (e.g. on windows) > A better approach is to use the Maven dependency plugin to download the > artifacts and copy them to a directory in the "target" directory of the > tests. The artifacts get downloaded to the local maven repository, so after > the first download, the build won't have to redownload anything. It should > work on every platform as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (SENTRY-1901) sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist anymore
[ https://issues.apache.org/jira/browse/SENTRY-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144060#comment-16144060 ] Brian Towles commented on SENTRY-1901: -- [~coheigea] Yeah SENTRY-967 was how I was going to solve it. If you don't mind im going to adopt that JIRA and get it working against the current code base and get it going. > sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist > anymore > -- > > Key: SENTRY-1901 > URL: https://issues.apache.org/jira/browse/SENTRY-1901 > Project: Sentry > Issue Type: Bug > Components: Build >Affects Versions: 1.8.0, 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles > > First we do not have a stable version we are running tests against. We are > trying to run the tests against is 2.0.0-SNAPSHOT version. So im not sure > where the run against 2.0.0-SNAPSHOT is coming from. Does anyone know which > version of sqoop we should be testing against? > As well its downloading it into a thirdparty directory that is directly under > the project but not in the target. it is being excluded from the git buy the > .gitignore and is not cleaned up between builds. I assume this is to save > time on the downloads between builds. This is causing the hardcoded snapshot > build to be available for older source trees but not newer ones since the > SNAPSHOT version requested is no longer available. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1901) sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist anymore
[ https://issues.apache.org/jira/browse/SENTRY-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1901: - Description: First we do not have a stable version we are running tests against. We are trying to run the tests against is 2.0.0-SNAPSHOT version. So im not sure where the run against 2.0.0-SNAPSHOT is coming from. Does anyone know which version of sqoop we should be testing against? As well its downloading it into a thirdparty directory that is directly under the project but not in the target. it is being excluded from the git buy the .gitignore and is not cleaned up between builds. I assume this is to save time on the downloads between builds. This is causing the hardcoded snapshot build to be available for older source trees but not newer ones since the SNAPSHOT version requested is no longer available. was: First we do not have a stable version we are running tests against. We are trying to run the tests against is 2.0.0-SNAPSHOT version but the CDH6.X version seems to be 1.4.6-cdh6.x-SNAPSHOT. So im not sure where the run against 2.0.0-SNAPSHOT is coming from. Does anyone know which version of sqoop we should be testing against? As well its downloading it into a thirdparty directory that is directly under the project but not in the target. it is being excluded from the git buy the .gitignore and is not cleaned up between builds. I assume this is to save time on the downloads between builds. This is causing the hardcoded snapshot build to be available for older source trees but not newer ones since the SNAPSHOT version requested is no longer available. > sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist > anymore > -- > > Key: SENTRY-1901 > URL: https://issues.apache.org/jira/browse/SENTRY-1901 > Project: Sentry > Issue Type: Bug > Components: Build >Affects Versions: 1.8.0, 2.0.0 >Reporter: Brian Towles >Assignee: Brian Towles > > First we do not have a stable version we are running tests against. We are > trying to run the tests against is 2.0.0-SNAPSHOT version. So im not sure > where the run against 2.0.0-SNAPSHOT is coming from. Does anyone know which > version of sqoop we should be testing against? > As well its downloading it into a thirdparty directory that is directly under > the project but not in the target. it is being excluded from the git buy the > .gitignore and is not cleaned up between builds. I assume this is to save > time on the downloads between builds. This is causing the hardcoded snapshot > build to be available for older source trees but not newer ones since the > SNAPSHOT version requested is no longer available. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (SENTRY-1901) sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist anymore
Brian Towles created SENTRY-1901: Summary: sentry-tests-sqoop is pulling hardcoded snapshot version that doesnt exist anymore Key: SENTRY-1901 URL: https://issues.apache.org/jira/browse/SENTRY-1901 Project: Sentry Issue Type: Bug Components: Build Affects Versions: 1.8.0 Reporter: Brian Towles Assignee: Brian Towles First we do not have a stable version we are running tests against. We are trying to run the tests against is 2.0.0-SNAPSHOT version but the CDH6.X version seems to be 1.4.6-cdh6.x-SNAPSHOT. So im not sure where the run against 2.0.0-SNAPSHOT is coming from. Does anyone know which version of sqoop we should be testing against? As well its downloading it into a thirdparty directory that is directly under the project but not in the target. it is being excluded from the git buy the .gitignore and is not cleaned up between builds. I assume this is to save time on the downloads between builds. This is causing the hardcoded snapshot build to be available for older source trees but not newer ones since the SNAPSHOT version requested is no longer available. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1871) Improve performance of ACL lookups for the Sentry Name Node plugin
[ https://issues.apache.org/jira/browse/SENTRY-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1871: - Fix Version/s: 2.0.0 > Improve performance of ACL lookups for the Sentry Name Node plugin > -- > > Key: SENTRY-1871 > URL: https://issues.apache.org/jira/browse/SENTRY-1871 > Project: Sentry > Issue Type: Improvement > Components: Hdfs Plugin >Affects Versions: 1.8.0 >Reporter: Brian Towles >Assignee: Brian Towles > Fix For: 2.0.0 > > > The performance of acl lookups for the HDFS is slow for very large numbers of > paths because the paths are stored as a tree on the NN plugin side. When a > lookup on an ACL occurs (getACLFeature call) it is a O(log n) operation for > each one. This can be changed to a Hash lookup based on path of only paths > with authorization objects attached to them. This would change it to a O(1) > lookup per path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1871) Improve performance of ACL lookups for the Sentry Name Node plugin
[ https://issues.apache.org/jira/browse/SENTRY-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1871: - Affects Version/s: 1.8.0 > Improve performance of ACL lookups for the Sentry Name Node plugin > -- > > Key: SENTRY-1871 > URL: https://issues.apache.org/jira/browse/SENTRY-1871 > Project: Sentry > Issue Type: Improvement > Components: Hdfs Plugin >Affects Versions: 1.8.0 >Reporter: Brian Towles >Assignee: Brian Towles > Fix For: 2.0.0 > > > The performance of acl lookups for the HDFS is slow for very large numbers of > paths because the paths are stored as a tree on the NN plugin side. When a > lookup on an ACL occurs (getACLFeature call) it is a O(log n) operation for > each one. This can be changed to a Hash lookup based on path of only paths > with authorization objects attached to them. This would change it to a O(1) > lookup per path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (SENTRY-1871) Improve performance of ACL lookups for the Sentry Name Node plugin
[ https://issues.apache.org/jira/browse/SENTRY-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles updated SENTRY-1871: - Component/s: Hdfs Plugin > Improve performance of ACL lookups for the Sentry Name Node plugin > -- > > Key: SENTRY-1871 > URL: https://issues.apache.org/jira/browse/SENTRY-1871 > Project: Sentry > Issue Type: Improvement > Components: Hdfs Plugin >Affects Versions: 1.8.0 >Reporter: Brian Towles >Assignee: Brian Towles > Fix For: 2.0.0 > > > The performance of acl lookups for the HDFS is slow for very large numbers of > paths because the paths are stored as a tree on the NN plugin side. When a > lookup on an ACL occurs (getACLFeature call) it is a O(log n) operation for > each one. This can be changed to a Hash lookup based on path of only paths > with authorization objects attached to them. This would change it to a O(1) > lookup per path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (SENTRY-1871) Improve performance of ACL lookups for the Sentry Name Node plugin
[ https://issues.apache.org/jira/browse/SENTRY-1871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Towles reassigned SENTRY-1871: Assignee: Brian Towles > Improve performance of ACL lookups for the Sentry Name Node plugin > -- > > Key: SENTRY-1871 > URL: https://issues.apache.org/jira/browse/SENTRY-1871 > Project: Sentry > Issue Type: Improvement > Components: Hdfs Plugin >Affects Versions: 1.8.0 >Reporter: Brian Towles >Assignee: Brian Towles > Fix For: 2.0.0 > > > The performance of acl lookups for the HDFS is slow for very large numbers of > paths because the paths are stored as a tree on the NN plugin side. When a > lookup on an ACL occurs (getACLFeature call) it is a O(log n) operation for > each one. This can be changed to a Hash lookup based on path of only paths > with authorization objects attached to them. This would change it to a O(1) > lookup per path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
