Re: Review Request 49898: ACL support for Logsearch Solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49898/ --- (Updated July 12, 2016, 3:21 p.m.) Review request for Ambari, Miklos Gergely, Robert Levas, Robert Nettleton, Sumit Mohanty, and Sebastian Toader. Changes --- - increase zk client timeout and defaul solr min/max memory Bugs: AMBARI-17653 https://issues.apache.org/jira/browse/AMBARI-17653 Repository: ambari Description --- Add abillity to use jaas file to authenticate on znode. (that was missing in upload configuration to zk) - Get rid of zkcli of Solr (we can manage everything from solr-client, that will be useful, later we can use Watchers with async zookeeper command, which can wait until a znode is accessible etc.) - add new functionalities for solrCloudCli (and solr_cloud_util) like setup_kerberos and add_sasl_users, with those we can copy all of the data between secure/unsecure znodes (sync...its needed because we have strict permissions, but if we disable kerberos we can not change them back), and we can create new sasl users. (also it set workd permission on secure node)...in the end I not use these features in ambari, using 2 znode (as hbase does) is a bit tricky solution. we have some other components where we are not using secure zookeeper. if we can find a good solution there later then I can use these features (or change them if neededlike, it would be good if we can use a zk super user...in that case we do not need to switch between znodes) - user state json file to track what is the state of the cluster (secure/unsecure), that is used to figure it out an enable/disable kerberos task happening or there is not state change at all - remove a lot of unused j2 file from logsearch module - add retries for atlas/ranger if solr or znode not accessible - better error messages for timing errors Diffs (updated) - ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4ecddf2 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 3428dd8 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/global.config.json.j2 cd51118 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/input.config.json.j2 bc48503 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml.j2 4338ee3 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2 63c590e ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh.j2 713a73a ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2 cb9bea5 ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/core.properties.j2 5185057 ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/solrconfig.xml.j2 0152880 ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml.j2 3b6eac9 ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2 8e3966e ambari-logsearch/ambari-logsearch-portal/src/main/scripts/run.sh.j2 cb78c58 ambari-logsearch/ambari-logsearch-solr-client/pom.xml bc3babe ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java 1e3cca5 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java 33e94d0 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java de90859 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractStateFileZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperConfigCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperRetryCommand.java c9e8558 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckConfigZkCommand.java 819395c ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckZnodeZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CopyZnodeZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateSaslUsersZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateSolrZnodeZkCommand.java PRE-CREATION
Re: Review Request 49898: ACL support for Logsearch Solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49898/#review141874 --- Ship it! Ship It! - Sebastian Toader On July 11, 2016, 6:33 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49898/ > --- > > (Updated July 11, 2016, 6:33 p.m.) > > > Review request for Ambari, Miklos Gergely, Robert Levas, Robert Nettleton, > Sumit Mohanty, and Sebastian Toader. > > > Bugs: AMBARI-17653 > https://issues.apache.org/jira/browse/AMBARI-17653 > > > Repository: ambari > > > Description > --- > > Add abillity to use jaas file to authenticate on znode. (that was missing in > upload configuration to zk) > - Get rid of zkcli of Solr (we can manage everything from solr-client, that > will be useful, later we can use Watchers with async zookeeper command, which > can wait until a znode is accessible etc.) > - add new functionalities for solrCloudCli (and solr_cloud_util) like > setup_kerberos and add_sasl_users, with those we can copy all of the data > between secure/unsecure znodes (sync...its needed because we have strict > permissions, but if we disable kerberos we can not change them back), and we > can create new sasl users. (also it set workd permission on secure node)...in > the end I not use these features in ambari, using 2 znode (as hbase does) is > a bit tricky solution. we have some other components where we are not using > secure zookeeper. if we can find a good solution there later then I can use > these features (or change them if neededlike, it would be good if we can > use a zk super user...in that case we do not need to switch between znodes) > - user state json file to track what is the state of the cluster > (secure/unsecure), that is used to figure it out an enable/disable kerberos > task happening or there is not state change at all > - remove a lot of unused j2 file from logsearch module > - add retries for atlas/ranger if solr or znode not accessible > - better error messages for timing errors > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py > 4ecddf2 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 > 3428dd8 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/global.config.json.j2 > cd51118 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/input.config.json.j2 > bc48503 > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml.j2 > 4338ee3 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2 > 63c590e > ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh.j2 > 713a73a > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2 > cb9bea5 > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/core.properties.j2 > 5185057 > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/solrconfig.xml.j2 > 0152880 > ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml.j2 > 3b6eac9 > > ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2 > 8e3966e > ambari-logsearch/ambari-logsearch-portal/src/main/scripts/run.sh.j2 cb78c58 > ambari-logsearch/ambari-logsearch-solr-client/pom.xml bc3babe > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java > 1e3cca5 > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java > 33e94d0 > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java > de90859 > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractStateFileZkCommand.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperConfigCommand.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperRetryCommand.java > c9e8558 > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckConfigZkCommand.java > 819395c > > ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckZnodeZkCommand.java > PRE-CREATION > >
Review Request 49898: ACL support for Logsearch Solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49898/ --- Review request for Ambari, Miklos Gergely, Robert Levas, Robert Nettleton, Sumit Mohanty, and Sebastian Toader. Bugs: AMBARI-17653 https://issues.apache.org/jira/browse/AMBARI-17653 Repository: ambari Description --- Add abillity to use jaas file to authenticate on znode. (that was missing in upload configuration to zk) - Get rid of zkcli of Solr (we can manage everything from solr-client, that will be useful, later we can use Watchers with async zookeeper command, which can wait until a znode is accessible etc.) - add new functionalities for solrCloudCli (and solr_cloud_util) like setup_kerberos and add_sasl_users, with those we can copy all of the data between secure/unsecure znodes (sync...its needed because we have strict permissions, but if we disable kerberos we can not change them back), and we can create new sasl users. (also it set workd permission on secure node)...in the end I not use these features in ambari, using 2 znode (as hbase does) is a bit tricky solution. we have some other components where we are not using secure zookeeper. if we can find a good solution there later then I can use these features (or change them if neededlike, it would be good if we can use a zk super user...in that case we do not need to switch between znodes) - user state json file to track what is the state of the cluster (secure/unsecure), that is used to figure it out an enable/disable kerberos task happening or there is not state change at all - remove a lot of unused j2 file from logsearch module - add retries for atlas/ranger if solr or znode not accessible - better error messages for timing errors Diffs - ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py 4ecddf2 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 3428dd8 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/global.config.json.j2 cd51118 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/input.config.json.j2 bc48503 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml.j2 4338ee3 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/output.config.json.j2 63c590e ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh.j2 713a73a ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml.j2 cb9bea5 ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/core.properties.j2 5185057 ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/solrconfig.xml.j2 0152880 ambari-logsearch/ambari-logsearch-portal/src/main/resources/log4j.xml.j2 3b6eac9 ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2 8e3966e ambari-logsearch/ambari-logsearch-portal/src/main/scripts/run.sh.j2 cb78c58 ambari-logsearch/ambari-logsearch-solr-client/pom.xml bc3babe ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java 1e3cca5 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java 33e94d0 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java de90859 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractStateFileZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperConfigCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/AbstractZookeeperRetryCommand.java c9e8558 ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckConfigZkCommand.java 819395c ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CheckZnodeZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CopyZnodeZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateSaslUsersZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/CreateSolrZnodeZkCommand.java PRE-CREATION ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/DownloadConfigZkCommand.java 2a0fa31
