Re: Review Request 64091: Log Feeder: handle multiple '=' (split char) in key/value filter
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64091/#review191999 --- Ship it! Ship It! - Miklos Gergely On Nov. 27, 2017, 8:54 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64091/ > --- > > (Updated Nov. 27, 2017, 8:54 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22526 > https://issues.apache.org/jira/browse/AMBARI-22526 > > > Repository: ambari > > > Description > --- > > Logfeeder key/value filter cannot handle multiple occurence of split char -> > in order to solve that we can limit the number of splits > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java > b04a439 > > > Diff: https://reviews.apache.org/r/64091/diff/2/ > > > Testing > --- > > done > > > Thanks, > > Oliver Szabo > >
Re: Review Request 64027: Ambari server becomes unusable when config properties are misconfigured
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64027/#review191729 --- Ship it! Ship It! - Miklos Gergely On Nov. 22, 2017, 12:52 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64027/ > --- > > (Updated Nov. 22, 2017, 12:52 p.m.) > > > Review request for Ambari, Attila Magyar, Balázs Bence Sári, and Miklos > Gergely. > > > Bugs: AMBARI-22499 > https://issues.apache.org/jira/browse/AMBARI-22499 > > > Repository: ambari > > > Description > --- > > Not all numeric values are valid doubles, eg. 86400L is a valid long, but not > a double. > > Changed to `NumberUtils.createNumber` instead of `Double.parseDouble` to > create the appropriate numeric object. > Checking `doubleValue()` is necessary for equal values of different types > (eg. `0.0` vs `0`). > > Also fixed possible NPE in non-numeric case. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > eade91458847125df8e50505cb2af5bd67ee0b3d > > ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java > 8a0a78228e72fc14ff361445363fa61d1cc4e977 > > > Diff: https://reviews.apache.org/r/64027/diff/1/ > > > Testing > --- > > Added unit test: > > ``` > [INFO] Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.31 > s - in org.apache.ambari.server.state.ConfigHelperTest > ``` > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 63803: Log Feeder: do not try to connect AMS, when it is disabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63803/#review190984 --- Ship it! Ship It! - Miklos Gergely On Nov. 14, 2017, 6:15 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63803/ > --- > > (Updated Nov. 14, 2017, 6:15 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22443 > https://issues.apache.org/jira/browse/AMBARI-22443 > > > Repository: ambari > > > Description > --- > > Set ams client to null if any of their properties are null. > > Its already merged/fixed in 2.6 branch > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/LogFeederAMSClient.java > f446446 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/MetricsManager.java > 1094852 > > ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java > 24042a7 > > > Diff: https://reviews.apache.org/r/63803/diff/1/ > > > Testing > --- > > done > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63696: Exclude test scope jars from Log Search / Log Feeder rpm/deb packages
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63696/#review190576 --- Ship it! Ship It! - Miklos Gergely On Nov. 9, 2017, 12:53 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63696/ > --- > > (Updated Nov. 9, 2017, 12:53 p.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Aravindan > Vijayan, Krisztian Kasa, Miklos Gergely, and Robert Nettleton. > > > Bugs: branch-2.6 and trunk > https://issues.apache.org/jira/browse/branch-2.6 > https://issues.apache.org/jira/browse/trunk > > > Repository: ambari > > > Description > --- > > dependency plugin include all jars in target/libs, so we need to remove those > that has test scope > > > Diffs > - > > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml cf64a36 > ambari-logsearch/ambari-logsearch-portal/pom.xml 53d6e8b > > > Diff: https://reviews.apache.org/r/63696/diff/2/ > > > Testing > --- > > UT done, target/libs folder does not contain easymock, checkstyle, > utility,junit jars > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63441: Remove Slider from Ambari stack definition of HDP 3.0
> On Nov. 1, 2017, 5:14 p.m., Jayush Luniya wrote: > > This will break HIVE LLAP as it uses slider. Please don't make any changes > > to HDP-3.0 stack for now. > > Miklos Gergely wrote: > Currently there is no Slider package in the HDP-3.0 yum repos, so even if > this dependency is still there it is impossible to install Hive with Ambari, > as Slider can not be installed for HDP-3.0, the Ambari install will fail at > the point when it looks for the Slider packages. > > Miklos Gergely wrote: > I've also discussed the situation with a Slider developer (Gour Saha) and > he confirmed that Slider will not be present in HDP-3.0 as a separate > service. I don't see why to keep it in the stack definition. Just one more thing: In this patch I was only removing Slider 0.91.0.3.0 from common services, as it was referneced only from HDP-3.0. Slider 0.60.0.2.2 service definition still remains in common services, so it will be available for all the previous HDP stack definitions / other stack definitions. - Miklos --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63441/#review189818 ----------- On Oct. 31, 2017, 1:35 p.m., Miklos Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63441/ > --- > > (Updated Oct. 31, 2017, 1:35 p.m.) > > > Review request for Ambari, Jayush Luniya, Robert Levas, and Sebastian Toader. > > > Bugs: AMBARI-22340 > https://issues.apache.org/jira/browse/AMBARI-22340 > > > Repository: ambari > > > Description > --- > > Slider is no longer a service on it's own in HDP 3.0, it is a part of YARN. > So it should be removed from the stack definition. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/metainfo.xml > db490c9 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml > adeba58 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml > ce06a18 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-log4j.xml > cd9b2a4 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/kerberos.json > 8f07b42 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/metainfo.xml > 168a2df > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/files/hbaseSmokeVerify.sh > 5c320c0 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/__init__.py > 5561e10 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params.py > c03329c > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_linux.py > 350be45 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_windows.py > fd2f358 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/service_check.py > 8582b5e > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider.py > 83c5629 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py > 45c7785 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/templates/storm-slider-env.sh.j2 > 8022a4b > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/service_advisor.py > 5614868 > > ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml > 061587d > > ambari-server/src/main/resources/stacks/HDP/3.0/services/SLIDER/metainfo.xml > 23f558d > > > Diff: https://reviews.apache.org/r/63441/diff/1/ > > > Testing > --- > > Tested on local vagrant cluster > > > Thanks, > > Miklos Gergely > >
Re: Review Request 63441: Remove Slider from Ambari stack definition of HDP 3.0
> On Nov. 1, 2017, 5:14 p.m., Jayush Luniya wrote: > > This will break HIVE LLAP as it uses slider. Please don't make any changes > > to HDP-3.0 stack for now. > > Miklos Gergely wrote: > Currently there is no Slider package in the HDP-3.0 yum repos, so even if > this dependency is still there it is impossible to install Hive with Ambari, > as Slider can not be installed for HDP-3.0, the Ambari install will fail at > the point when it looks for the Slider packages. I've also discussed the situation with a Slider developer (Gour Saha) and he confirmed that Slider will not be present in HDP-3.0 as a separate service. I don't see why to keep it in the stack definition. - Miklos --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63441/#review189818 --- On Oct. 31, 2017, 1:35 p.m., Miklos Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63441/ > --- > > (Updated Oct. 31, 2017, 1:35 p.m.) > > > Review request for Ambari, Jayush Luniya, Robert Levas, and Sebastian Toader. > > > Bugs: AMBARI-22340 > https://issues.apache.org/jira/browse/AMBARI-22340 > > > Repository: ambari > > > Description > --- > > Slider is no longer a service on it's own in HDP 3.0, it is a part of YARN. > So it should be removed from the stack definition. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/metainfo.xml > db490c9 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml > adeba58 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml > ce06a18 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-log4j.xml > cd9b2a4 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/kerberos.json > 8f07b42 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/metainfo.xml > 168a2df > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/files/hbaseSmokeVerify.sh > 5c320c0 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/__init__.py > 5561e10 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params.py > c03329c > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_linux.py > 350be45 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_windows.py > fd2f358 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/service_check.py > 8582b5e > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider.py > 83c5629 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py > 45c7785 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/templates/storm-slider-env.sh.j2 > 8022a4b > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/service_advisor.py > 5614868 > > ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml > 061587d > > ambari-server/src/main/resources/stacks/HDP/3.0/services/SLIDER/metainfo.xml > 23f558d > > > Diff: https://reviews.apache.org/r/63441/diff/1/ > > > Testing > --- > > Tested on local vagrant cluster > > > Thanks, > > Miklos Gergely > >
Re: Review Request 63441: Remove Slider from Ambari stack definition of HDP 3.0
> On Nov. 1, 2017, 5:14 p.m., Jayush Luniya wrote: > > This will break HIVE LLAP as it uses slider. Please don't make any changes > > to HDP-3.0 stack for now. Currently there is no Slider package in the HDP-3.0 yum repos, so even if this dependency is still there it is impossible to install Hive with Ambari, as Slider can not be installed for HDP-3.0, the Ambari install will fail at the point when it looks for the Slider packages. - Miklos --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63441/#review189818 --- On Oct. 31, 2017, 1:35 p.m., Miklos Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63441/ > --- > > (Updated Oct. 31, 2017, 1:35 p.m.) > > > Review request for Ambari, Jayush Luniya, Robert Levas, and Sebastian Toader. > > > Bugs: AMBARI-22340 > https://issues.apache.org/jira/browse/AMBARI-22340 > > > Repository: ambari > > > Description > --- > > Slider is no longer a service on it's own in HDP 3.0, it is a part of YARN. > So it should be removed from the stack definition. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/metainfo.xml > db490c9 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml > adeba58 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml > ce06a18 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-log4j.xml > cd9b2a4 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/kerberos.json > 8f07b42 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/metainfo.xml > 168a2df > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/files/hbaseSmokeVerify.sh > 5c320c0 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/__init__.py > 5561e10 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params.py > c03329c > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_linux.py > 350be45 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_windows.py > fd2f358 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/service_check.py > 8582b5e > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider.py > 83c5629 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py > 45c7785 > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/templates/storm-slider-env.sh.j2 > 8022a4b > > ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/service_advisor.py > 5614868 > > ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml > 061587d > > ambari-server/src/main/resources/stacks/HDP/3.0/services/SLIDER/metainfo.xml > 23f558d > > > Diff: https://reviews.apache.org/r/63441/diff/1/ > > > Testing > --- > > Tested on local vagrant cluster > > > Thanks, > > Miklos Gergely > >
Review Request 63441: Remove Slider from Ambari stack definition of HDP 3.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63441/ --- Review request for Ambari, Jayush Luniya, Robert Levas, and Sebastian Toader. Bugs: AMBARI-22340 https://issues.apache.org/jira/browse/AMBARI-22340 Repository: ambari Description --- Slider is no longer a service on it's own in HDP 3.0, it is a part of YARN. So it should be removed from the stack definition. Diffs - ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/metainfo.xml db490c9 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml adeba58 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml ce06a18 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-log4j.xml cd9b2a4 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/kerberos.json 8f07b42 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/metainfo.xml 168a2df ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/files/hbaseSmokeVerify.sh 5c320c0 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/__init__.py 5561e10 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params.py c03329c ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_linux.py 350be45 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_windows.py fd2f358 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/service_check.py 8582b5e ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider.py 83c5629 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py 45c7785 ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/templates/storm-slider-env.sh.j2 8022a4b ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/service_advisor.py 5614868 ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml 061587d ambari-server/src/main/resources/stacks/HDP/3.0/services/SLIDER/metainfo.xml 23f558d Diff: https://reviews.apache.org/r/63441/diff/1/ Testing --- Tested on local vagrant cluster Thanks, Miklos Gergely
Review Request 63362: Fix Log Search configuration dependencies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63362/ --- Review request for Ambari, Krisztian Kasa, Oliver Szabo, and Robert Nettleton. Bugs: AMBARI-22328 https://issues.apache.org/jira/browse/AMBARI-22328 Repository: ambari Description --- logsearch-service_logs-solrconfig and logsearch-audit_logs-solrconfig config groups are specified with .xml, logsearch-common-properties is not present at all as a dependency. Diffs - ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 4a7b3e0 Diff: https://reviews.apache.org/r/63362/diff/1/ Testing --- Tested on local cluster. Thanks, Miklos Gergely
Re: Review Request 63328: Log Search: Add docker-compose file for creating ZooKeeper containers
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63328/#review189419 --- Ship it! Ship It! - Miklos Gergely On Oct. 27, 2017, 9:54 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63328/ > --- > > (Updated Oct. 27, 2017, 9:54 a.m.) > > > Review request for Ambari, Krisztian Kasa, Laszlo Puskas, Miklos Gergely, > Robert Nettleton, and Sebastian Toader. > > > Bugs: AMBARI-22314 > https://issues.apache.org/jira/browse/AMBARI-22314 > > > Repository: ambari > > > Description > --- > > In order to separate components for logsearch dev env we need to create > separate docker containers for all of them (zookeeper, solr, logsearch, > logfeeder) > > Its much more clear to use a docker-compose file for it, and also we should > use the offical zookeeper/solr images. > > First patch contains the zookeeper compose file. > > > Diffs > - > > ambari-logsearch/docker/docker-compose-zookeeper.yml PRE-CREATION > > > Diff: https://reviews.apache.org/r/63328/diff/1/ > > > Testing > --- > > done, with "docker-compose -f docker-compose-zookeeper.yml up -d" > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63323: Get rid of deprecated Solr usages (infra + logsearch)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63323/#review189289 --- Ship it! Ship It! - Miklos Gergely On Oct. 26, 2017, 9:24 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63323/ > --- > > (Updated Oct. 26, 2017, 9:24 a.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: trunk > https://issues.apache.org/jira/browse/trunk > > > Repository: ambari > > > Description > --- > > As a lot of deprecated solr4j (solr 6+) methods will be deleted in solr 7 I > get rid of most of the solr 6 usages. > > One thign what i missing is the HttpClientUtil kerberos configurer changes, > about that, there is no class that can be compatible with both solr versions > > > Diffs > - > > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/AmbariSolrCloudClientBuilder.java > 54c4ae3 > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/commands/CreateCollectionCommand.java > 53e96fd > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/commands/CreateShardCommand.java > 9b57827 > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/commands/GetShardsCommand.java > e49bfec > > > Diff: https://reviews.apache.org/r/63323/diff/1/ > > > Testing > --- > > done. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63298: Update Solr (Infra Solr + clients) version to 6.6.2 from 6.6.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63298/#review189200 --- Ship it! Ship It! - Miklos Gergely On Oct. 25, 2017, 1:58 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63298/ > --- > > (Updated Oct. 25, 2017, 1:58 p.m.) > > > Review request for Ambari, Laszlo Puskas, Miklos Gergely, Robert Nettleton, > Sandor Magyari, and Sebastian Toader. > > > Bugs: AMBARI-22305 > https://issues.apache.org/jira/browse/AMBARI-22305 > > > Repository: ambari > > > Description > --- > > Update Solr version to 6.6.2 from 6.6.0 (contains some security fixes) > > > Diffs > - > > ambari-infra/pom.xml 908abb4 > ambari-logsearch/docker/Dockerfile 1e4135e > ambari-logsearch/pom.xml 9434365 > > > Diff: https://reviews.apache.org/r/63298/diff/1/ > > > Testing > --- > > done. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63265: Log Feeder: allow applying grok filter on default fields (like type, path, host etc.)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63265/#review189115 --- Ship it! Ship It! - Miklos Gergely On Oct. 24, 2017, 10:30 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63265/ > --- > > (Updated Oct. 24, 2017, 10:30 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22300 > https://issues.apache.org/jira/browse/AMBARI-22300 > > > Repository: ambari > > > Description > --- > > add new "init_default_fields" config param to input block. If that option is > turned on, then logfeeder will apply the default fields before any grok > filter is applied (right now it should just work with grok filter > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java > deff1b2 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java > 6f6bb6d > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputManager.java > 8bf6ae3 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java > 73cf449 > > > Diff: https://reviews.apache.org/r/63265/diff/1/ > > > Testing > --- > > UTs done > > > Thanks, > > Oliver Szabo > >
Review Request 63196: Avoid Log Feeder NPEs for logs of services not present on the node, but present on the cluster
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63196/ --- Review request for Ambari, Krisztian Kasa, Oliver Szabo, and Robert Nettleton. Bugs: AMBARI-22281 https://issues.apache.org/jira/browse/AMBARI-22281 Repository: ambari Description --- Due to a bug Log Feeders throw NPEs while searching for logs of services which are not present on the node, but present on the cluster. The problem was caused by the fact that File.listFiles(FileFilter) returns null if the file it's called on is not an existing directory, and Arrays.sort() throws an NPE if it's argument is null. Diffs - ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java e24a7aa Diff: https://reviews.apache.org/r/63196/diff/1/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Re: Review Request 63162: Disable xmlparser and configEdit API in Infra Solr by default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63162/#review188825 --- Ship it! Ship It! - Miklos Gergely On Oct. 19, 2017, 7:36 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63162/ > --- > > (Updated Oct. 19, 2017, 7:36 p.m.) > > > Review request for Ambari, Krisztian Kasa, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-22273 > https://issues.apache.org/jira/browse/AMBARI-22273 > > > Repository: ambari > > > Description > --- > > Due to a vulnerability found in Solr with CVE-2017-12629 > (https://nvd.nist.gov/vuln/detail/CVE-2017-12629) > 1.) Disable editing with the Config API by adding the > "-Ddisable.configEdit=true" flag to the SOLR_OPTS by default. > 2.) Update all collections to reroute the xmlparser query parser away from > the vulnerable class, but adding this to the Ranger, Atlas, and LogSearch > collections: > > > That wont affect upgrade as with some manual changes these options can be set > properly. This change only for default deployments. (also wont affect 3.0) > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/solrconfig.xml > 7af91df > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/hadoop_logs/conf/solrconfig.xml > 59f778f > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/history/conf/solrconfig.xml > 8244a08 > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-env.sh.j2 > 5cc344e > > ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/atlas-solrconfig.xml > cba4a4e > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2 > 63879e7 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2 > b6a4d1d > > ambari-server/src/main/resources/common-services/RANGER/0.7.0/properties/ranger-solrconfig.xml.j2 > 25dbb7a > > > Diff: https://reviews.apache.org/r/63162/diff/2/ > > > Testing > --- > > done, UTs passes, FT: install Solr with these setting, also check what > happens if we adding the new xml parser. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63118: Log Search server does not handle proxies properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63118/#review188641 --- Ship it! Ship It! - Miklos Gergely On Oct. 18, 2017, 6:47 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63118/ > --- > > (Updated Oct. 18, 2017, 6:47 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-22266 > https://issues.apache.org/jira/browse/AMBARI-22266 > > > Repository: ambari > > > Description > --- > > If logsearch running behind a proxy and the address is not on root, UI cannot > be accessed/handled properly. > > (like: localhost:61888 can be accessed through localhost:8080/logsearch/) > > Regarding to that issue, there are some backend changes that required: > - form login filter needs to be removed, as we have are custom login > entripoint already (that form login filter is basically the same, so we had 2 > different filter on each other, which did the same) > - add useForward (configurable) to our login entrypoint, to make it possible > to send redirects through proxies properly > - on logout: add useReferer, which can be used if logsearch is behind a > proxy, it can use the referer location instead of the root one. > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java > 54cc10c > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java > 2f9cba4 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/authenticate/LogsearchLogoutSuccessHandler.java > c20e383 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchAuthenticationEntryPoint.java > 1831697 > > > Diff: https://reviews.apache.org/r/63118/diff/1/ > > > Testing > --- > > done, works with both proxies and without proxies (for proxies there should > be some ui changes as well) > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63027: Allow wildcard for log directory folder in the path component of Logfeeder input
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63027/#review188284 --- Ship it! Ship It! - Miklos Gergely On Oct. 16, 2017, 1:03 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63027/ > --- > > (Updated Oct. 16, 2017, 1:03 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-21145 > https://issues.apache.org/jira/browse/AMBARI-21145 > > > Repository: ambari > > > Description > --- > > Allow wildcards for handling log directories. Using ant library for this. (on > trunk, the solution will be JDK 8 based, so its a temporal hack for 2.6.1, > for trunk some classes have cleaned up, so the implementation there will be > different) > > The idea is if we have a folder pattern, we will craete a thread group, and > there will be different threads for every matching input (now only for tail > file) > > Also there is 2 new thread introduced: > - one which checks is the monitored file too old, if it is, stop the thread > for that (detach) > - an another one which checks there is any new file available (like log files > with date patterns in its name), and if it is, update the monitoring thread > with that (log path upgrade) > > Introduced some new configuration as well for inputs: > - path_update_interval_min: the period in minutes for checking new files > (default: 5, based on detach values, its possible that a new input wont be > monitored) > - detach_interval_min: the period in minutes for checking which files are too > old (default: 300) > - detach_time_min: the period in minutes when we flag a file is too old > (default: 2000) > > Also fix an issue which cause that we cannot use grok field names more deeply > (use deep_extract param for this one in grok filter) and i updated the docker > container as well to make it work with branch-2.6 (and use different version > for container in order to keep that) > > > Diffs > - > > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java > 7e2da70 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java > 9f54d8a > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java > 3737839 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/AbstractLogFileMonitor.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/LogFileDetachMonitor.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/LogFilePathUpdateMonitor.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandler.java > 0ece637 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java > ffd6cec > ambari-logsearch/docker/Dockerfile dfa1462 > ambari-logsearch/docker/logsearch-docker.sh a2df90f > > > Diff: https://reviews.apache.org/r/63027/diff/1/ > > > Testing > --- > > done, no UTs here, FTs are in progress > > > Thanks, > > Oliver Szabo > >
Re: Review Request 63027: Allow wildcard for log directory folder in the path component of Logfeeder input
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63027/#review188283 --- ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java Lines 85 (patched) <https://reviews.apache.org/r/63027/#comment265275> This variable is not used for anything, so let's get rid of it, and also the function that calculates it in FileUtils ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java Lines 396 (patched) <https://reviews.apache.org/r/63027/#comment265279> No need to expose threadGroup ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java Lines 437 (patched) <https://reviews.apache.org/r/63027/#comment265278> No need to expose multiFolder, and the class may access the variable directly Also no need to expose logFileDetacherThread, and logFilePathUpdaterThread ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java Lines 37 (patched) <https://reviews.apache.org/r/63027/#comment265274> This should go to init(), it doesn't have to be done over and over again till the input is ready ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/AbstractLogFileMonitor.java Lines 31 (patched) <https://reviews.apache.org/r/63027/#comment265277> inputFile and deteachTime could be protected, thus no getter is needed for the extending classes to access them ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/monitor/AbstractLogFileMonitor.java Lines 45 (patched) <https://reviews.apache.org/r/63027/#comment265276> waitInterval should not be exposed at all ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java Lines 98 (patched) <https://reviews.apache.org/r/63027/#comment265273> This is a utility class performing a general operation, do not link it's name to it's purpose, so let's call this something like getFilesByPathPattern ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java Lines 105 (patched) <https://reviews.apache.org/r/63027/#comment265272> There is a potential hazard, that the folderBeforeRegex repeated after the first *, for example: /var/log/*/var/log/... not likely, but for sure instead of replacing the folderBeforeRegex use substring() - Miklos Gergely On Oct. 16, 2017, 1:03 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63027/ > --- > > (Updated Oct. 16, 2017, 1:03 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-21145 > https://issues.apache.org/jira/browse/AMBARI-21145 > > > Repository: ambari > > > Description > --- > > Allow wildcards for handling log directories. Using ant library for this. (on > trunk, the solution will be JDK 8 based, so its a temporal hack for 2.6.1, > for trunk some classes have cleaned up, so the implementation there will be > different) > > The idea is if we have a folder pattern, we will craete a thread group, and > there will be different threads for every matching input (now only for tail > file) > > Also there is 2 new thread introduced: > - one which checks is the monitored file too old, if it is, stop the thread > for that (detach) > - an another one which checks there is any new file available (like log files > with date patterns in its name), and if it is, update the monitoring thread > with that (log path upgrade) > > Introduced some new configuration as well for inputs: > - path_update_interval_min: the period in minutes for checking new files > (default: 5, based on detach values, its possible that a new input wont be > monitored) > - detach_interval_min: the period in minutes for checking which files are too > old (default: 300) > - detach_time_min: the period in minutes when we flag a file is too old > (default: 2000) > > Also fix an issue which cause that we cannot use grok field names more deeply > (use deep_extract param for this one in grok filter) and i updated the docker > container as well to make it work with branch-2.6 (and use different version > for container in order to keep that) > > > Diffs > - > > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 &
Review Request 62618: Use symlinks for Solr Data Manager
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62618/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-22075 https://issues.apache.org/jira/browse/AMBARI-22075 Repository: ambari Description --- Put symlinks in /usr/bin for infra-solr-data-manager (same as zookeeper-client and zookeeper-server), to acces /usr/lib/ambari-infra-solr-client/solrDataManager.py more easily Create symlinks in the post install scripts, to make it possible to create the rpm packages to be created on Mac Diffs - ambari-infra/ambari-infra-assembly/src/main/package/deb/solr-client/postinst c339cd0 ambari-infra/ambari-infra-assembly/src/main/package/rpm/solr-client/postinstall.sh c339cd0 ambari-infra/ambari-infra-solr-client/build.xml 25ff0cb Diff: https://reviews.apache.org/r/62618/diff/1/ Testing --- tested on local cluster Thanks, Miklos Gergely
Re: Review Request 62584: Remove Database auto configuration from Infra Manager
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62584/#review186382
---
Ship it!
Ship It!
- Miklos Gergely
On Sept. 26, 2017, 6:30 p.m., Oliver Szabo wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62584/
> ---
>
> (Updated Sept. 26, 2017, 6:30 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Laszlo Puskas, Miklos Gergely,
> Robert Nettleton, and Sandor Magyari.
>
>
> Bugs: AMBARI-22064
> https://issues.apache.org/jira/browse/AMBARI-22064
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Spring boot introduced a Database autoconfiguration feature. Becuase of that
> feature, our initialization script does not work.
>
> To use that properly we need to set some spring properties
> ("spring.datasource" prefix) which can use the right database schema script
> with discovering the datasorce type of the datasource (which sounds fine),
> but we should not use that feature until we find out how to change the
> default prefixes of the configuration.
>
>
> Diffs
> -
>
>
> ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/InfraManager.java
> 185e344
>
>
> Diff: https://reviews.apache.org/r/62584/diff/1/
>
>
> Testing
> ---
>
> manually, db file created properly
>
>
> Thanks,
>
> Oliver Szabo
>
>
Review Request 62576: Solr Data Manager script should provide non-destructive archive download option
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62576/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-22061 https://issues.apache.org/jira/browse/AMBARI-22061 Repository: ambari Description --- Allow users to save only their data without deleting Diffs - ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py e0356bb Diff: https://reviews.apache.org/r/62576/diff/1/ Testing --- tested locally Thanks, Miklos Gergely
Re: Review Request 62570: Use symlinks for Infra components
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62570/#review186248 --- Ship it! Ship It! - Miklos Gergely On Sept. 26, 2017, 1:09 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62570/ > --- > > (Updated Sept. 26, 2017, 1:09 p.m.) > > > Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Miklos Gergely, > and Robert Nettleton. > > > Bugs: AMBARI-22059 > https://issues.apache.org/jira/browse/AMBARI-22059 > > > Repository: ambari > > > Description > --- > > Put symlinks in /usr/bin for infra-solr-client (same as zookeeper-client and > zookeeper-server), to acces /usr/lib/ambari-infra-solr-client scripts more > easily > > Create symlinks in the post install scripts, to make it possible to create > the rpm packages to be created on Mac > > > Diffs > - > > ambari-infra/ambari-infra-assembly/pom.xml fafef7e > ambari-infra/ambari-infra-assembly/src/main/package/deb/manager/postinst > 21a01fa > > ambari-infra/ambari-infra-assembly/src/main/package/deb/solr-client/postinst > 21a01fa > > ambari-infra/ambari-infra-assembly/src/main/package/rpm/manager/postinstall.sh > PRE-CREATION > > ambari-infra/ambari-infra-assembly/src/main/package/rpm/solr-client/postinstall.sh > PRE-CREATION > ambari-infra/ambari-infra-manager/src/main/resources/infraManager.sh > 65287b2 > ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh > 1d219d6 > > > Diff: https://reviews.apache.org/r/62570/diff/1/ > > > Testing > --- > > done manually. > > > Thanks, > > Oliver Szabo > >
Review Request 62562: Solr Data Manager script should use gzip compression type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62562/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-22056 https://issues.apache.org/jira/browse/AMBARI-22056 Repository: ambari Description --- Support gzip compression and offer it as default. Diffs - ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py 18a4da7 Diff: https://reviews.apache.org/r/62562/diff/1/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Re: Review Request 62551: Make Log Search IT work with the new UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62551/#review186219 --- Ship it! Ship It! - Miklos Gergely On Sept. 25, 2017, 6:56 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62551/ > --- > > (Updated Sept. 25, 2017, 6:56 p.m.) > > > Review request for Ambari, Attila Doroszlai, Miklos Gergely, Robert > Nettleton, and Sandor Magyari. > > > Bugs: AMBARI-22055 > https://issues.apache.org/jira/browse/AMBARI-22055 > > > Repository: ambari > > > Description > --- > > As Log Search has a new UI, integration test framework cannot login (with > Selenium) anyomore during test run (using X11) > > Fix that to make IT tests work with the new UI > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/LogSearchUISteps.java > b40a2bc > > ambari-logsearch/ambari-logsearch-it/src/test/resources/stories/selenium/login.ui.story > 543c211 > > > Diff: https://reviews.apache.org/r/62551/diff/1/ > > > Testing > --- > > done. with maven failsafe plugin and enabling xquartz > > > Thanks, > > Oliver Szabo > >
Review Request 62543: Infra Manager jobs should handle their own configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62543/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-22050 https://issues.apache.org/jira/browse/AMBARI-22050 Repository: ambari Description --- Dummy job configuration should be extracted from InfraManagerBatchConfig Diffs - ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java 98a4901 ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobConfiguration.java PRE-CREATION Diff: https://reviews.apache.org/r/62543/diff/1/ Testing --- Tested locally, still works fine Thanks, Miklos Gergely
Review Request 62542: Solr Data Manager should use --line-delimited option by default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62542/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-22049 https://issues.apache.org/jira/browse/AMBARI-22049 Repository: ambari Description --- Instead of having an option for creating line delimited json file, that should be the default, and a json file should be created if a flag is set. Also the -j short argument form was assigned to the name parameter instead of this. Diffs - ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py d17aec7 Diff: https://reviews.apache.org/r/62542/diff/1/ Testing --- Tested on local cluster. Thanks, Miklos Gergely
Re: Review Request 62530: Infra Manager: Integrate with Spring Boot
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62530/#review186095 --- Ship it! Ship It! - Miklos Gergely On Sept. 24, 2017, 12:31 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62530/ > --- > > (Updated Sept. 24, 2017, 12:31 p.m.) > > > Review request for Ambari, Attila Magyar, Laszlo Puskas, Miklos Gergely, and > Robert Nettleton. > > > Bugs: AMBARI-22046 > https://issues.apache.org/jira/browse/AMBARI-22046 > > > Repository: ambari > > > Description > --- > > Integrate infra manager with spring boot in order to achive easier dependency > and property management. > > Other improvements: > - in memory swagger description storage > - log4j2 support > > > Diffs > - > > ambari-infra/ambari-infra-manager/.gitignore PRE-CREATION > ambari-infra/ambari-infra-manager/docker/bin/start.sh 076c06f > ambari-infra/ambari-infra-manager/pom.xml b9f2a77 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/InfraManager.java > 186f98c > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java > 105f20e > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerDataConfig.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerJerseyResourceConfig.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerServletConfig.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/StaticResourceConfiguration.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/doc/InfraManagerApiDocStorage.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java > 444f0d1 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/ApiDocResource.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/resources/infra-manager.properties > fbeac78 > ambari-infra/ambari-infra-manager/src/main/resources/log4j.xml 0450454 > ambari-infra/ambari-infra-manager/src/main/resources/log4j2.xml > PRE-CREATION > ambari-infra/ambari-infra-manager/src/main/resources/static/index.html > PRE-CREATION > ambari-infra/ambari-infra-manager/src/main/resources/swagger/swagger.html > 8580e1a > ambari-infra/ambari-infra-manager/src/main/resources/webapp/index.html > 3e64867 > > > Diff: https://reviews.apache.org/r/62530/diff/1/ > > > Testing > --- > > done manually. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 62526: Infra Solr client cleanup: make infra-solr-client script usable with symlinks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62526/#review186094 --- Ship it! Ship It! - Miklos Gergely On Sept. 23, 2017, 11:20 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62526/ > --- > > (Updated Sept. 23, 2017, 11:20 p.m.) > > > Review request for Ambari, Attila Magyar, Miklos Gergely, Robert Nettleton, > and Sandor Magyari. > > > Bugs: AMBARI-22045 > https://issues.apache.org/jira/browse/AMBARI-22045 > > > Repository: ambari > > > Description > --- > > Make it possible for infra solr scripts to used by symlinks. > > Right now if a symlink points to the infra solr scripts -> throws an error as > we using the actual directory to calculate the java classpath. The patch > fixes that. > > Also added some small cleanups (e.g.: stderrs are duplicated because of 2 > appenders - i changed the default ones) > > > Diffs > - > > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/AmbariSolrCloudCLI.java > 70bc232 > ambari-infra/ambari-infra-solr-client/src/main/resources/log4j.properties > e8dca12 > ambari-infra/ambari-infra-solr-client/src/main/resources/solrCloudCli.sh > 7bfa864 > ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh > 12e6a77 > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/solr-client-log4j.properties.j2 > 0d0adc3 > > > Diff: https://reviews.apache.org/r/62526/diff/1/ > > > Testing > --- > > tested manually. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 62460: Enable logsearch-web unit tests and make them skippable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62460/#review185874 --- Ship it! Ship It! - Miklos Gergely On Sept. 21, 2017, 10:47 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62460/ > --- > > (Updated Sept. 21, 2017, 10:47 a.m.) > > > Review request for Ambari, Andrii Babiichuk, Attila Magyar, Miklos Gergely, > and Robert Nettleton. > > > Bugs: AMBARI-22019 > https://issues.apache.org/jira/browse/AMBARI-22019 > > > Repository: ambari > > > Description > --- > > The new Log Search web UTs are not enabled by default (as earlier it did not > run in headless mode), now: enable it and make it skippable > > > Diffs > - > > ambari-logsearch/ambari-logsearch-web/pom.xml 7a6b02e > > > Diff: https://reviews.apache.org/r/62460/diff/1/ > > > Testing > --- > > unit tests run and passed, skipped if i addeed -DskipTests > > > Thanks, > > Oliver Szabo > >
Re: Review Request 62456: Add JMX enable/disable option to Infra Solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62456/#review185858 --- Ship it! Ship It! - Miklos Gergely On Sept. 21, 2017, 9:29 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62456/ > --- > > (Updated Sept. 21, 2017, 9:29 a.m.) > > > Review request for Ambari, Attila Magyar, Miklos Gergely, Robert Nettleton, > and Sandor Magyari. > > > Bugs: AMBARI-22013 > https://issues.apache.org/jira/browse/AMBARI-22013 > > > Repository: ambari > > > Description > --- > > - Add jmx enable option to Infra Solr stack (also change the default value > from true to false) > - Add java.rmi.server.hostname property to the solr env file (add this to the > docker environment) to make JMX RMI communication work with JConsole. > - Set the infra solr property ordering in ambari-web > > > Diffs > - > > ambari-logsearch/docker/test-config/solr/solr-env-ssl.sh faaa6ff > ambari-logsearch/docker/test-config/solr/solr-env.sh 11b9fe0 > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > 52a5e6a > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py > e4ea885 > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/properties/infra-solr-env.sh.j2 > 5cc344e > ambari-web/app/data/configs/services/ambari_infra_properties.js aae7fa5 > > > Diff: https://reviews.apache.org/r/62456/diff/1/ > > > Testing > --- > > tested manually on 3 node cluster > > > Thanks, > > Oliver Szabo > >
Re: Review Request 62369: Log Search: remove file patterns from web security config
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62369/#review185530 --- Ship it! Ship It! - Miklos Gergely On Sept. 17, 2017, 1:39 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62369/ > --- > > (Updated Sept. 17, 2017, 1:39 p.m.) > > > Review request for Ambari, Andrii Babiichuk, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-21976 > https://issues.apache.org/jira/browse/AMBARI-21976 > > > Repository: ambari > > > Description > --- > > As the web resources files are mapped under /resources/, no need to map the > file patterns in the security config > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java > 5ba0ac0 > > > Diff: https://reviews.apache.org/r/62369/diff/1/ > > > Testing > --- > > done, manually > > > Thanks, > > Oliver Szabo > >
Re: Review Request 62368: Stack Advisor can reset the number of shards (Log Search) during adding a new service (BP install)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62368/#review185529 --- Ship it! Ship It! - Miklos Gergely On Sept. 17, 2017, 1:07 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62368/ > --- > > (Updated Sept. 17, 2017, 1:07 p.m.) > > > Review request for Ambari, Attila Doroszlai, Miklos Gergely, Robert > Nettleton, and Sebastian Toader. > > > Bugs: AMBARI-21975 > https://issues.apache.org/jira/browse/AMBARI-21975 > > > Repository: ambari > > > Description > --- > > The default value for the number of shards is 5. On UI install, the proper > value is calculated, the max values is 3x number of solr instances. If you > have only 1 Solr and Log Search was installed with BP without any custom > shard config, the shard max value will be 3, but in fact the actual one is 5, > that can cause during adding a new service through the UI can reset the > number of shards. To fix that: set the max number of shard value to 5, in > case you have just 1 solr ( use: max (3x solrs, 5)) > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py > f0d9f78 > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > ff57c7b > > > Diff: https://reviews.apache.org/r/62368/diff/1/ > > > Testing > --- > > done. (updated unit tests) > > > Thanks, > > Oliver Szabo > >
Re: Review Request 62367: Increase the default maximum memory value for Infra Solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62367/#review185528 --- Ship it! Ship It! - Miklos Gergely On Sept. 17, 2017, 12:39 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62367/ > --- > > (Updated Sept. 17, 2017, 12:39 p.m.) > > > Review request for Ambari, Attila Doroszlai, Miklos Gergely, Robert > Nettleton, and Sebastian Toader. > > > Bugs: AMBARI-21974 > https://issues.apache.org/jira/browse/AMBARI-21974 > > > Repository: ambari > > > Description > --- > > Change the max memory value for Infra Solr. The old max was 5G, which is > sometimes too small for a solr node (there are examples where customers using > 30G -> you cannot set that value easily on the UI with the smart configs) > > > Diffs > - > > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/configuration/infra-solr-env.xml > 27f001e > > > Diff: https://reviews.apache.org/r/62367/diff/1/ > > > Testing > --- > > not required. > > > Thanks, > > Oliver Szabo > >
Review Request 62356: Log Feeder should not support german month names
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62356/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21968 https://issues.apache.org/jira/browse/AMBARI-21968 Repository: ambari Description --- Log Feeder grok patterns currently support german month names too. This has almost no usefulness at all, as there are no german language logs, and we don't even support german day names, but it may cause errors as the März contains a ä character which may lead to encoding related errors. Diffs - ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/grok-patterns 920f521 Diff: https://reviews.apache.org/r/62356/diff/1/ Testing --- Done. Thanks, Miklos Gergely
Re: Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- (Updated Sept. 12, 2017, 3:30 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- allow user to add additional filters allow user to specify the name of the archive support tar.bz2 generate data with json.dumps fix typos Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs (updated) - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/7/ Changes: https://reviews.apache.org/r/61898/diff/6-7/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Re: Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- (Updated Sept. 7, 2017, 9:56 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- use flag to indicate line delimited json file creation Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs (updated) - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/6/ Changes: https://reviews.apache.org/r/61898/diff/5-6/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Re: Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- (Updated Sept. 7, 2017, 8:33 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- configurable file format, compression method Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs (updated) - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/5/ Changes: https://reviews.apache.org/r/61898/diff/4-5/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Review Request 62083: Fix Log Search upgrade to 3.0.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62083/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21881 https://issues.apache.org/jira/browse/AMBARI-21881 Repository: ambari Description --- There are 3 problems about the Log Search upgrade to 3.0.0 - *-logsearch-conf attributes were deleted instead of removing the config type - logsearch-common-properties must be created explicitly, because the empty config type will not be created by default - logsearch-output-config was created instead of overwriting logfeeder-output-config Diffs - ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java 9255163 ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java 53518dd Diff: https://reviews.apache.org/r/62083/diff/1/ Testing --- Tested on local cluster. Thanks, Miklos Gergely
Re: Review Request 62034: Integrate LogSearch new UI with the server and get rid of the old one
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62034/#review184505 --- Ship it! Ship It! - Miklos Gergely On Sept. 1, 2017, 7:17 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62034/ > --- > > (Updated Sept. 1, 2017, 7:17 p.m.) > > > Review request for Ambari, Andrii Babiichuk, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-21870 > https://issues.apache.org/jira/browse/AMBARI-21870 > > > Repository: ambari > > > Description > --- > > - get rid of old ui > - renamve web-new to web > - add some small changes to authentication (create an abstract filter for jwt > and move it somewhere else, those changes will be continued in an other patch) > - change login/logout authentication mechanism for the new UI (do not > redirect to /login.html) > > > Diffs > - > > ambari-logsearch/ambari-logsearch-server/build.xml aadacd7 > ambari-logsearch/ambari-logsearch-server/pom.xml e60cc14 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/auth/filter/AbstractJWTFilter.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java > 038ba5c > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/authenticate/LogsearchLogoutSuccessHandler.java > c20e383 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchJWTFilter.java > 1bc7231 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/model/JWTAuthenticationToken.java > dfac191 > ambari-logsearch/ambari-logsearch-web-new/.angular-cli.json > ambari-logsearch/ambari-logsearch-web-new/.editorconfig > ambari-logsearch/ambari-logsearch-web-new/.gitignore 54bfd20 > ambari-logsearch/ambari-logsearch-web-new/README.md 2fb07fb > ambari-logsearch/ambari-logsearch-web-new/e2e/app.e2e-spec.ts > ambari-logsearch/ambari-logsearch-web-new/e2e/app.po.ts > ambari-logsearch/ambari-logsearch-web-new/e2e/tsconfig.e2e.json > ambari-logsearch/ambari-logsearch-web-new/karma.conf.js > ambari-logsearch/ambari-logsearch-web-new/package.json 96733eb > ambari-logsearch/ambari-logsearch-web-new/pom.xml 28e1754 > ambari-logsearch/ambari-logsearch-web-new/protractor.conf.js > ambari-logsearch/ambari-logsearch-web-new/src/app/app.module.ts > > ambari-logsearch/ambari-logsearch-web-new/src/app/classes/queries/audit-logs-query-params.class.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/classes/queries/query-params.class.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/classes/queries/service-logs-histogram-query-params.class.spec.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/classes/queries/service-logs-histogram-query-params.class.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/classes/queries/service-logs-query-params.class.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/accordion-panel/accordion-panel.component.html > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/accordion-panel/accordion-panel.component.less > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/accordion-panel/accordion-panel.component.spec.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/accordion-panel/accordion-panel.component.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.html > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.less > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.spec.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/app.component.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.html > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.less > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dropdown-button.component.spec.ts > > > ambari-logsearch/ambari-logsearch-web-new/src/app/components/dropdown-button/dro
Re: Review Request 61993: Atlas should handle external Zookeeper properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61993/ --- (Updated Aug. 31, 2017, 8:44 p.m.) Review request for Ambari, Oliver Szabo, Robert Nettleton, and Vishal Suvagia. Summary (updated) - Atlas should handle external Zookeeper properly Bugs: AMBARI-21855 https://issues.apache.org/jira/browse/AMBARI-21855 Repository: ambari Description --- Atlas tries to create it's collection using the local ambari-infra Solr even if it is specified to use an external one. Diffs - ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 0305b82 ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 41be001 Diff: https://reviews.apache.org/r/61993/diff/1/ Testing --- Tested on two clusters, one running the external solr, the other installed Atlas, the collections were created. Thanks, Miklos Gergely
Re: Review Request 62015: Ranger should handle external Zookeeper properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62015/ --- (Updated Aug. 31, 2017, 8:41 p.m.) Review request for Ambari, Gautam Borad, Mugdha Varadkar, Oliver Szabo, Robert Nettleton, Vishal Suvagia, and Velmurugan Periasamy. Summary (updated) - Ranger should handle external Zookeeper properly Bugs: AMBARI-21863 https://issues.apache.org/jira/browse/AMBARI-21863 Repository: ambari Description --- Ranger tries to create it's configurations using the local ambari-infra Solr even if it is specified to use an external one. Diffs - ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py bd45558 Diff: https://reviews.apache.org/r/62015/diff/1/ Testing (updated) --- Tested on two clusters, one running the external Zookeeper, the other installed Ranger, the config data were created. Thanks, Miklos Gergely
Re: Review Request 62015: Ranger should handle external Solr properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62015/ --- (Updated Aug. 31, 2017, 8:41 p.m.) Review request for Ambari, Gautam Borad, Mugdha Varadkar, Oliver Szabo, Robert Nettleton, Vishal Suvagia, and Velmurugan Periasamy. Bugs: AMBARI-21863 https://issues.apache.org/jira/browse/AMBARI-21863 Repository: ambari Description --- Ranger tries to create it's configurations using the local ambari-infra Solr even if it is specified to use an external one. Diffs - ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py bd45558 Diff: https://reviews.apache.org/r/62015/diff/1/ Testing (updated) --- Tested on two clusters, one running the external solr, the other installed Ranger, the config data were created. Thanks, Miklos Gergely
Review Request 62015: Ranger should handle external Solr properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62015/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21863 https://issues.apache.org/jira/browse/AMBARI-21863 Repository: ambari Description --- Ranger tries to create it's configurations using the local ambari-infra Solr even if it is specified to use an external one. Diffs - ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py bd45558 Diff: https://reviews.apache.org/r/62015/diff/1/ Testing --- Tested on two clusters, one running the external solr, the other installed Atlas, the config data were created. Thanks, Miklos Gergely
Re: Review Request 61996: Integrate new Log Search UI with maven
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61996/#review184252 --- Ship it! Ship It! - Miklos Gergely On Aug. 30, 2017, 6:10 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61996/ > --- > > (Updated Aug. 30, 2017, 6:10 p.m.) > > > Review request for Ambari, Andrii Babiichuk, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-21857 > https://issues.apache.org/jira/browse/AMBARI-21857 > > > Repository: ambari > > > Description > --- > > Add new pom.xml file for the new logsearch web ui in order to package static > resources in jar (which are generated with yarn and angular-cli) > > The resource generation flow is the following: > - install (download or copy an already downloaded one) node and yarn in > target/webapp-build folder > - copy source files from the ui root folder into target/webapp-build > - run yarn install, then yarn build-prod > - copy target/webapp-build/dist to target/classes/dist > - during package phase target/classes content included in the generated jar > > Next steps from here: > Add the new ui dependency to logsearch server, during the logsearch portal > build, the dist static content will be packaged with the server files. > > > Diffs > - > > ambari-logsearch/ambari-logsearch-web-new/package.json b4902a3 > ambari-logsearch/ambari-logsearch-web-new/pom.xml PRE-CREATION > > > Diff: https://reviews.apache.org/r/61996/diff/1/ > > > Testing > --- > > done, manually, generated jar contains "dist/" folder. anyway, no mvn clean > test support yet. > > > Thanks, > > Oliver Szabo > >
Review Request 61993: Atlas should handle external Solr properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61993/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21855 https://issues.apache.org/jira/browse/AMBARI-21855 Repository: ambari Description --- Atlas tries to create it's collection using the local ambari-infra Solr even if it is specified to use an external one. Diffs - ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py 0305b82 ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py 41be001 Diff: https://reviews.apache.org/r/61993/diff/1/ Testing --- Tested on two clusters, one running the external solr, the other installed Atlas, the collections were created. Thanks, Miklos Gergely
Re: Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- (Updated Aug. 28, 2017, 2:52 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Fix argument long names to follow standards Fix logging issues fix variable names Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs (updated) - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/4/ Changes: https://reviews.apache.org/r/61898/diff/3-4/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Re: Review Request 61937: Fix new logsearch UI rat issues
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61937/#review183922 --- Ship it! Ship It! - Miklos Gergely On Aug. 28, 2017, 11:58 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61937/ > --- > > (Updated Aug. 28, 2017, 11:58 a.m.) > > > Review request for Ambari, Andrii Babiichuk and Miklos Gergely. > > > Bugs: AMBARI-21825 > https://issues.apache.org/jira/browse/AMBARI-21825 > > > Repository: ambari > > > Description > --- > > exclude some files from rat checks: vendor files, .* and README.md files > > > (note: by the way, i do not know the .editorconfig file is required for us or > not) > > > Diffs > - > > ambari-logsearch/pom.xml 2be11ee > > > Diff: https://reviews.apache.org/r/61937/diff/2/ > > > Testing > --- > > ran mvn clean package successfully > > > Thanks, > > Oliver Szabo > >
Re: Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- (Updated Aug. 27, 2017, 3:41 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Fix script name Allow saving to local directory Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs (updated) - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/3/ Changes: https://reviews.apache.org/r/61898/diff/2-3/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Re: Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- (Updated Aug. 25, 2017, 4:20 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- - Renamed script - Added delete mode - added flag to run verbose Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs (updated) - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solrDataManager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/2/ Changes: https://reviews.apache.org/r/61898/diff/1-2/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Re: Review Request 61888: Add configurable management endpoints to Log Search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61888/#review183812 --- Ship it! Ship It! - Miklos Gergely On Aug. 24, 2017, 3:05 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61888/ > --- > > (Updated Aug. 24, 2017, 3:05 p.m.) > > > Review request for Ambari, Balázs Bence Sári, Miklos Gergely, and Sebastian > Toader. > > > Bugs: AMBARI-21804 > https://issues.apache.org/jira/browse/AMBARI-21804 > > > Repository: ambari > > > Description > --- > > Add actuator framework for Log Search, enable a few endpoints: > /ping - health checks, added some custom health indicators as well > /metrics - memory/threads etc. details for the application > /env - environment variables and properties for the app, > (secrets/passwords/credentials are not visiable) > /heapdump + /dump endpoints > /auditevents - currently showing failed logins, needs to be extended later. > > new api endpoint outside of the managment apis: > /api/v1/info - show logsearch/solr/java version details - ambari reads that > from the new info.properties, which is filled during the maven build > > > Diffs > - > > ambari-logsearch/ambari-logsearch-server/pom.xml c0ae16e > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/ApplicationConfig.java > 3d30c2a > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java > 2758388 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java > da0a8bb > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/health/AbstractSolrHealthIndicator.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/health/SolrAuditLogsHealthIndicator.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/health/SolrEventHistoryHealthIndicator.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/health/SolrServiceLogsHealthIndicator.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/InfoManager.java > 2f63492 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/InfoResource.java > e49be90 > > ambari-logsearch/ambari-logsearch-server/src/main/resources/default.properties > 4c74142 > ambari-logsearch/ambari-logsearch-server/src/main/resources/info.properties > PRE-CREATION > > > Diff: https://reviews.apache.org/r/61888/diff/1/ > > > Testing > --- > > done. manually > > > Thanks, > > Oliver Szabo > >
Review Request 61898: Create Utility Script to support Solr Collection Data Retention/Purging/Archiving
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61898/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21810 https://issues.apache.org/jira/browse/AMBARI-21810 Repository: ambari Description --- This script allows the user to set an end date, and save all the previous log entries into HDFS or S3 Diffs - ambari-infra/ambari-infra-solr-client/build.xml 9b8b6cc ambari-infra/ambari-infra-solr-client/pom.xml 3818aba ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/S3Uploader.java PRE-CREATION ambari-infra/ambari-infra-solr-client/src/main/python/solr_collection_data_manager.py PRE-CREATION Diff: https://reviews.apache.org/r/61898/diff/1/ Testing --- Tested on local cluster with - SSL - kerberos - interrupted, then restarted script - HDFS - S3 - the combinations of the above Thanks, Miklos Gergely
Re: Review Request 61850: Log Search: hdfs audit log fields are not mapped
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61850/#review183708 --- Ship it! Ship It! - Miklos Gergely On Aug. 23, 2017, 4:20 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61850/ > --- > > (Updated Aug. 23, 2017, 4:20 p.m.) > > > Review request for Ambari, Miklos Gergely and Sebastian Toader. > > > Bugs: AMBARI-21790 > https://issues.apache.org/jira/browse/AMBARI-21790 > > > Repository: ambari > > > Description > --- > > Serialized json field name was wrong, that caused map existing key fields to > new fields does not work. > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldNameDescriptorImpl.java > e1b71e6 > > > Diff: https://reviews.apache.org/r/61850/diff/1/ > > > Testing > --- > > done, manually > > > Thanks, > > Oliver Szabo > >
Re: Review Request 61197: Log Feeder properties should be handled by one class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61197/ --- (Updated Aug. 2, 2017, 11:11 a.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Fix typo Bugs: AMBARI-21592 https://issues.apache.org/jira/browse/AMBARI-21592 Repository: ambari Description --- Log Feeder properties are scattered all over the code, with their documentation, putting large blocks of codes into several classes. Instead of it they should be handled by one class together with their documentation and default values, having all the rest of the classes asking that one for their value. Diffs (updated) - ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java ba3412b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 30b61a1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 9535260 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java 8050263 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java 10642d1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 091015a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java 7c487ba ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/loglevelfilter/LogLevelFilterHandler.java 79bf5ea ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/LogFeederAMSClient.java fdad9a6 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java 2b47a00 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputS3File.java 9f41a15 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java e3da864 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederPropertiesUtil.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java a244a4e ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SSLUtil.java 573bb1c ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandlerTest.java c07035b ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java 4b4067c Diff: https://reviews.apache.org/r/61197/diff/3/ Changes: https://reviews.apache.org/r/61197/diff/2-3/ Testing --- Tested on docker. The properties are working fine, and the properties info REST API endpoint still return their description. Thanks, Miklos Gergely
Re: Review Request 61277: Log Search Config should be separated into a Server and Log Feeder interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61277/ --- (Updated Aug. 2, 2017, 11:05 a.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Use LogSearchConfigConfigurer in a more Spring-ish way + fix bug about the get services call when there is no service (will be renamed later to cover a broader concept) input configuration at all yet Bugs: AMBARI-21623 https://issues.apache.org/jira/browse/AMBARI-21623 Repository: ambari Description --- Log Search Config is getting too complex, it serves too many kind of things. It should be separated into two interfaces, one for the Server and one for the Log Feeders to make it cleaner. Also fixed a minor issue about shipper configurations without any post map values. Diffs (updated) - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java 76be392 ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactory.java 77b48eb ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeeder.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigServer.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java e308346 ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java b64dae8 ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactoryTest.java f990c5c ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeederClass1.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeederClass2.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigServerClass1.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigServerClass2.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigLogFeederZK.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigServerZK.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java 387d0c6 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java ba3412b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 30b61a1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java 10642d1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/Output.java b370e58 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputSolrTest.java ce040f9 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/LogSearchConfigConfigurer.java 3f6df75 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrCollectionConfigurer.java 6de34ce ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java 15f59e4 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/CreateCollectionHandler.java 8b4c7bf ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java a1181b4 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilter.java 84bddb8 Diff: https://reviews.apache.org/r/61277/diff/2/ Changes: https://reviews.apache.org/r/61277/diff/1-2/ Testing --- Tested on local cluster, still works Thanks, Miklos Gergely
Re: Review Request 61197: Log Feeder properties should be handled by one class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61197/ --- (Updated Aug. 2, 2017, 4:30 a.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Make some logical expressions in OutputSolr cleaner. Bugs: AMBARI-21592 https://issues.apache.org/jira/browse/AMBARI-21592 Repository: ambari Description --- Log Feeder properties are scattered all over the code, with their documentation, putting large blocks of codes into several classes. Instead of it they should be handled by one class together with their documentation and default values, having all the rest of the classes asking that one for their value. Diffs (updated) - ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java ba3412b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 30b61a1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 9535260 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java 8050263 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java 10642d1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 091015a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java 7c487ba ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/loglevelfilter/LogLevelFilterHandler.java 79bf5ea ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/LogFeederAMSClient.java fdad9a6 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java 2b47a00 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputS3File.java 9f41a15 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java e3da864 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederPropertiesUtil.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java a244a4e ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SSLUtil.java 573bb1c ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandlerTest.java c07035b ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java 4b4067c Diff: https://reviews.apache.org/r/61197/diff/2/ Changes: https://reviews.apache.org/r/61197/diff/1-2/ Testing --- Tested on docker. The properties are working fine, and the properties info REST API endpoint still return their description. Thanks, Miklos Gergely
Review Request 61277: Log Search Config should be separated into a Server and Log Feeder interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61277/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21623 https://issues.apache.org/jira/browse/AMBARI-21623 Repository: ambari Description --- Log Search Config is getting too complex, it serves too many kind of things. It should be separated into two interfaces, one for the Server and one for the Log Feeders to make it cleaner. Also fixed a minor issue about shipper configurations without any post map values. Diffs - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java 76be392 ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactory.java 77b48eb ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeeder.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfigServer.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java e308346 ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java b64dae8 ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigFactoryTest.java f990c5c ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeederClass1.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigLogFeederClass2.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigServerClass1.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigServerClass2.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigLogFeederZK.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigServerZK.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java 387d0c6 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java ba3412b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 30b61a1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java 10642d1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/Output.java b370e58 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputSolrTest.java ce040f9 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/LogSearchConfigConfigurer.java 3f6df75 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java a1181b4 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilter.java 84bddb8 Diff: https://reviews.apache.org/r/61277/diff/1/ Testing --- Tested on local cluster, still works Thanks, Miklos Gergely
Review Request 61209: Log Search should stop using deprecated API for solr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61209/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21596 https://issues.apache.org/jira/browse/AMBARI-21596 Repository: ambari Description --- Since we've switched to solrj 6.6.0 some API calls we are using are deprecated now. Let's use non-deprecated API instead. Also suppressed some warnings and fixed the formatting of the pom file. Diffs - ambari-logsearch/ambari-logsearch-server/pom.xml ebca2d5 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrAuditAliasConfigurer.java 1eca94b ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrCollectionConfigurer.java 225f5a3 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java 71f9f29 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/AbstractSolrConfigHandler.java 6f3d8ca ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/CreateCollectionHandler.java b6e9def ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/ReloadCollectionHandler.java 52f3366 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/solr/ResponseDataGenerator.java 74e9c11 Diff: https://reviews.apache.org/r/61209/diff/1/ Testing --- Tested on docker, still works :) Thanks, Miklos Gergely
Review Request 61197: Log Feeder properties should be handled by one class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61197/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21592 https://issues.apache.org/jira/browse/AMBARI-21592 Repository: ambari Description --- Log Feeder properties are scattered all over the code, with their documentation, putting large blocks of codes into several classes. Instead of it they should be handled by one class together with their documentation and default values, having all the rest of the classes asking that one for their value. Diffs - ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java ba3412b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 30b61a1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 9535260 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java 8050263 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java 10642d1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 091015a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java 7c487ba ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/loglevelfilter/LogLevelFilterHandler.java 79bf5ea ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/metrics/LogFeederAMSClient.java fdad9a6 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java 2b47a00 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputS3File.java 9f41a15 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java 596e022 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederPropertiesUtil.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java a244a4e ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SSLUtil.java 573bb1c ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandlerTest.java c07035b ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java 4b4067c Diff: https://reviews.apache.org/r/61197/diff/1/ Testing --- Tested on docker. The properties are working fine, and the properties info REST API endpoint still return their description. Thanks, Miklos Gergely
Re: Review Request 60943: Log Search Solr output properties should be provided by the Config API
> On July 26, 2017, 6:43 p.m., Robert Nettleton wrote: > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java > > Lines 67 (patched) > > <https://reviews.apache.org/r/60943/diff/3/?file=1780214#file1780214line67> > > > > Would it make more sense to have a single "inputConfigExists" method, > > and then have implementations that are specific to the Logfeeder or the > > LogSearch Server? If the function is called from the Log Search Server then it also must be specified which cluster we are asking for, while when it is called from Log Feeder it is obvious. Their default ZK implementation uses different caches too (Log Search Server caches the data of all the clusters, while Log Feeder caches only it's own cluster's shipper configs) Later I'll split the LogSearchConfigAPI into two interfaces (and it's default implementation into two classes) having a common ancestor. Then both interfaces will have an inputConfigExists method with different signature. - Miklos --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60943/#review181476 ----------- On July 20, 2017, 9:54 a.m., Miklos Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60943/ > --- > > (Updated July 20, 2017, 9:54 a.m.) > > > Review request for Ambari, Oliver Szabo and Robert Nettleton. > > > Bugs: AMBARI-21507 > https://issues.apache.org/jira/browse/AMBARI-21507 > > > Repository: ambari > > > Description > --- > > The Solr output properties should be persisted into the Config API by the Log > Search Server as the output is initialized, and fetched by he Log Feeders > from there > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java > 6c5cefd > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/OutputConfigMonitor.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/outputconfig/OutputProperties.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/outputconfig/OutputSolrProperties.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java > 28844d5 > > ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java > 5934fa6 > > ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java > fdd8ed6 > > ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/outputconfig/impl/OutputSolrPropertiesImpl.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java > 59c2a22 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java > 5bf074c > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogEntryParseTester.java > 5356159 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java > 09fc3f5 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java > f1002ae > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/Output.java > 65b9e19 > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputManager.java > 4d6c43b > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java > 162a7f8 > > ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputSolrTest.java > 8985110 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java > c569a27 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrEventHistor
Re: Review Request 61162: Ooozie quicklink configuration doesn't work with https
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61162/#review181540 --- Ship it! Ship It! - Miklos Gergely On July 26, 2017, 9:22 p.m., Balázs Bence Sári wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61162/ > --- > > (Updated July 26, 2017, 9:22 p.m.) > > > Review request for Ambari, Andrii Tkach, Jaimin Jetly, Miklos Gergely, Oliver > Szabo, Robert Levas, Sebastian Toader, and Yusaku Sako. > > > Bugs: AMBARI-21579 > https://issues.apache.org/jira/browse/AMBARI-21579 > > > Repository: ambari > > > Description > --- > > Ooize quick links didn't work whith SSL enables die to two reasons: > > 1. Typos in the quicklink definitions (quicklinks.json) - fixed in this patch > > 2. An Oozie bug that prohibits changing the oozie-site/oozie.base.url > property to the https address. The port for the quick link address was taken > from this property. > > Issue #2 has been worked around in the patch by the using the > oozie-site/oozie.https.port to determine the port for the https quick link. > Code has been slightly changed to support custom regex to parse the https > port for the quicklink. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/state/quicklinks/Port.java > 7bd4b81 > > ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/quicklinks/quicklinks.json > 6353220 > > ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/quicklinks/quicklinks.json > 81e7cbe > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/quicklinks/quicklinks.json > f83bb5c > > ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/quicklinks/quicklinks.json > 81e7cbe > > ambari-server/src/test/java/org/apache/ambari/server/stack/QuickLinksConfigurationModuleTest.java > e37eaf0 > ambari-server/src/test/resources/parent_quicklinks.json 0aca8e3 > ambari-web/app/views/common/quick_view_link_view.js e39f230 > ambari-web/test/views/common/quick_link_view_test.js 48cdb28 > > > Diff: https://reviews.apache.org/r/61162/diff/1/ > > > Testing > --- > > - manually tested the fix > - wrote to javascript unit tests > - extended a java unit tests > - unit tests pass in ambari-web (21223 passing / 128 pending) > - java unit tests pass in ambari-server > - 13 python unit test fail in ambari-server to an unrelated commit (the tests > fail in CI builds too) > > > Thanks, > > Balázs Bence Sári > >
Re: Review Request 60943: Log Search Solr output properties should be provided by the Config API
/UpgradeCatalog300Test.java d7bdf75 Diff: https://reviews.apache.org/r/60943/diff/3/ Changes: https://reviews.apache.org/r/60943/diff/2-3/ Testing --- Tested on docker and vagrant. Also tested with external Log Search server. Thanks, Miklos Gergely
Re: Review Request 60966: Log Search docker test environment build front/backend only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60966/ --- (Updated July 19, 2017, 12:22 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Added documentation Bugs: AMBARI-21516 https://issues.apache.org/jira/browse/AMBARI-21516 Repository: ambari Description --- Building the logsearch project for testing on docker takes some time, while often only the backend / frontend was modified. It should be configurable what to build. Diffs (updated) - ambari-logsearch/docker/logsearch-docker.sh 4d53fa1 Diff: https://reviews.apache.org/r/60966/diff/2/ Changes: https://reviews.apache.org/r/60966/diff/1-2/ Testing --- Done. Thanks, Miklos Gergely
Review Request 60966: Log Search docker test environment build front/backend only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60966/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21516 https://issues.apache.org/jira/browse/AMBARI-21516 Repository: ambari Description --- Building the logsearch project for testing on docker takes some time, while often only the backend / frontend was modified. It should be configurable what to build. Diffs - ambari-logsearch/docker/logsearch-docker.sh 4d53fa1 Diff: https://reviews.apache.org/r/60966/diff/1/ Testing --- Done. Thanks, Miklos Gergely
Re: Review Request 60943: Log Search Solr output properties should be provided by the Config API
/diff/1-2/ Testing --- Tested on docker and vagrant. Also tested with external Log Search server. Thanks, Miklos Gergely
Review Request 60943: Log Search Solr output properties should be provided by the Config API
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60943/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21507 https://issues.apache.org/jira/browse/AMBARI-21507 Repository: ambari Description --- The Solr output properties should be persisted into the Config API by the Log Search Server as the output is initialized, and fetched by he Log Feeders from there Diffs - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java 6c5cefd ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/OutputConfigMonitor.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/outputconfig/OutputProperties.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/outputconfig/OutputSolrProperties.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java 28844d5 ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java 5934fa6 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java fdd8ed6 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/outputconfig/impl/OutputSolrPropertiesImpl.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java 59c2a22 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 5bf074c ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogEntryParseTester.java 5356159 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputConfigUploader.java 09fc3f5 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java f1002ae ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/Output.java 65b9e19 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputManager.java 4d6c43b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java 162a7f8 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java c569a27 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrEventHistoryPropsConfig.java 975e6a7 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrPropsConfig.java ceddf7e ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java e5039d5 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/LogSearchConfigConfigurer.java c34dce6 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrCollectionConfigurer.java f2d022e ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java 3eea08f ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java 308ef1f ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java b30b6ef ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/CreateCollectionHandler.java 752a1e1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java 2c143c0 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/output.config.json 55fd36c ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/output.config.json.j2 214e5ba ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py fc49824 Diff: https://reviews.apache.org/r/60943/diff/1/ Testing --- Tested on docker and vagrant. Also tested with external Log Search server. Thanks, Miklos Gergely
Re: Review Request 60813: Add lucene index migration script to infra solr client package
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60813/#review180412 --- Fix it, then Ship it! Ship It! ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 75 (patched) <https://reviews.apache.org/r/60813/#comment255595> use shift 2 ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 84 (patched) <https://reviews.apache.org/r/60813/#comment255596> use shift 2 ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 93 (patched) <https://reviews.apache.org/r/60813/#comment255597> use if [ -z $... ] ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 98 (patched) <https://reviews.apache.org/r/60813/#comment255598> use if [ -z $... ] ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 102 (patched) <https://reviews.apache.org/r/60813/#comment255599> use if [ -z $... ] ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 110 (patched) <https://reviews.apache.org/r/60813/#comment255593> This supposed to be $INDEX_DIR ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh Lines 140 (patched) <https://reviews.apache.org/r/60813/#comment255594> Fix the indentations of the cases - Miklos Gergely On July 12, 2017, 7:03 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60813/ > --- > > (Updated July 12, 2017, 7:03 p.m.) > > > Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, > and Sebastian Toader. > > > Bugs: AMBARI-21459 > https://issues.apache.org/jira/browse/AMBARI-21459 > > > Repository: ambari > > > Description > --- > > Add script to migrate lucene index from version 5.x to lucene 6.x. > > The bash cli tool implemeted without getopts as it has some issues on > different evironments. > > Backup files not implemented as solr 6 has already a snapshot tool. > > For more specific runs (e.g. using hdfs directory factory or exorcise corrupt > indices from files) check index and upgrade index tool can be run directly as > well. > > Note: migration takes about 1 minute per 1 GB > > > Diffs > - > > ambari-infra/ambari-infra-solr-client/build.xml a54e336 > ambari-infra/ambari-infra-solr-client/pom.xml d103003 > ambari-infra/ambari-infra-solr-client/src/main/resources/solrIndexHelper.sh > PRE-CREATION > > > Diff: https://reviews.apache.org/r/60813/diff/1/ > > > Testing > --- > > done. run script manually with different use cases > > > Thanks, > > Oliver Szabo > >
Review Request 60798: Log Feeder should support logs without date (time only)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60798/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21447 https://issues.apache.org/jira/browse/AMBARI-21447 Repository: ambari Description --- If there is only the time of the day present for a log entry we should handle it similarly as the entries without year component: if the time is already passed, then it is for today, otherwise it is for yesterday. Diffs - ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperDate.java 305688b Diff: https://reviews.apache.org/r/60798/diff/1/ Testing --- Tested on local cluster with HDF logs (Registry, Streamline) Thanks, Miklos Gergely
Re: Review Request 60744: Add python functions to get the Ambari version of the agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60744/ --- (Updated July 10, 2017, 2:37 p.m.) Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, and Robert Nettleton. Changes --- fix summary Summary (updated) - Add python functions to get the Ambari version of the agent Bugs: AMBARI-21435 https://issues.apache.org/jira/browse/AMBARI-21435 Repository: ambari Description --- In the mpacks we may want to know the version of Ambari, for example the Log Search handles it's configuration in different ways depending on it's version Diffs - ambari-common/src/main/python/ambari_commons/get_ambari_version.py PRE-CREATION Diff: https://reviews.apache.org/r/60744/diff/3/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Re: Review Request 60744: Add python functions to get the Ambari version of the server / agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60744/ --- (Updated July 10, 2017, 2:27 p.m.) Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, and Robert Nettleton. Changes --- added documentation removed get_ambari_version_server, it can be obtained like this: >>> from ambari_server.serverConfiguration import get_ambari_properties, >>> get_ambari_version >>> p = get_ambari_properties() >>> get_ambari_version(p) Bugs: AMBARI-21435 https://issues.apache.org/jira/browse/AMBARI-21435 Repository: ambari Description --- In the mpacks we may want to know the version of Ambari, for example the Log Search handles it's configuration in different ways depending on it's version Diffs (updated) - ambari-common/src/main/python/ambari_commons/get_ambari_version.py PRE-CREATION Diff: https://reviews.apache.org/r/60744/diff/3/ Changes: https://reviews.apache.org/r/60744/diff/2-3/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Re: Review Request 60744: Add python functions to get the Ambari version of the server / agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60744/ --- (Updated July 10, 2017, 1:48 p.m.) Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Oliver Szabo, and Robert Nettleton. Changes --- fix file handling Bugs: AMBARI-21435 https://issues.apache.org/jira/browse/AMBARI-21435 Repository: ambari Description --- In the mpacks we may want to know the version of Ambari, for example the Log Search handles it's configuration in different ways depending on it's version Diffs (updated) - ambari-common/src/main/python/ambari_commons/get_ambari_version.py PRE-CREATION Diff: https://reviews.apache.org/r/60744/diff/2/ Changes: https://reviews.apache.org/r/60744/diff/1-2/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Review Request 60744: Add python functions to get the Ambari version of the server / agent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60744/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21435 https://issues.apache.org/jira/browse/AMBARI-21435 Repository: ambari Description --- In the mpacks we may want to know the version of Ambari, for example the Log Search handles it's configuration in different ways depending on it's version Diffs - ambari-common/src/main/python/ambari_commons/get_ambari_version.py PRE-CREATION Diff: https://reviews.apache.org/r/60744/diff/1/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Re: Review Request 60689: Upgrade Infra Solr version from 5.5.2 to 6.6.x
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60689/#review179860 --- Ship it! Ship It! - Miklos Gergely On July 6, 2017, 6:41 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60689/ > --- > > (Updated July 6, 2017, 6:41 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-21400 > https://issues.apache.org/jira/browse/AMBARI-21400 > > > Repository: ambari > > > Description > --- > > - upgrade solr version to 6.6.0 > - update custom solr start script (add 6.6.0 custom changes) > - add new solr client command (--remove-admin-handlers) -> its required for > solr 6, using AdminHandlers class was deprecated in Solr 5, in Solr 6, it > does not exists, so throws runtime expection in solr node. This command will > run before solr starts. > - handle ambari upgrade scenario > > (note: this change does not contain index migration changes, there will be a > command for that as well, but that should be some manual step) > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py > 1c5432b > ambari-infra/ambari-infra-assembly/src/main/resources/solr bf504d9 > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/AmbariSolrCloudCLI.java > e3a1e79 > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/AmbariSolrCloudClient.java > 9479679 > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/commands/RemoveAdminHandlersCommand.java > PRE-CREATION > > ambari-infra/ambari-infra-solr-client/src/main/java/org/apache/ambari/infra/solr/commands/SetClusterPropertyZkCommand.java > 34597c6 > > ambari-infra/ambari-infra-solr-plugin/src/test/java/org/apache/ambari/infra/security/InfraRuleBasedAuthorizationPluginTest.java > ee84969 > ambari-infra/pom.xml 4f5c29c > > ambari-logsearch/ambari-logsearch-server/src/main/configsets/audit_logs/conf/solrconfig.xml > 7af91df > > ambari-logsearch/ambari-logsearch-server/src/main/configsets/hadoop_logs/conf/solrconfig.xml > 59f778f > > ambari-logsearch/ambari-logsearch-server/src/main/configsets/history/conf/solrconfig.xml > 8244a08 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java > d99694b > ambari-logsearch/docker/Dockerfile 2b8fd5d > ambari-logsearch/docker/bin/start.sh 28ebf65 > ambari-logsearch/pom.xml 82943e4 > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java > 5fdc885 > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/params.py > 5f547f3 > > ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/package/scripts/setup_infra_solr.py > f3dbcf3 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2 > 63879e7 > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2 > b6a4d1d > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > a342baa > ambari-server/src/test/python/stacks/2.4/AMBARI_INFRA/test_infra_solr.py > 87304cd > > > Diff: https://reviews.apache.org/r/60689/diff/1/ > > > Testing > --- > > unit tests done. + manual testing, there are some more use cases ... > > > Thanks, > > Oliver Szabo > >
Review Request 60695: Add REST end point for the documentation of the Log Feeder shipper properties
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60695/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21423 https://issues.apache.org/jira/browse/AMBARI-21423 Repository: ambari Description --- Add a REST API end point similar to the property description endpoint which provides info about the fields of the shipper configuration. Diffs - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/ShipperConfigElementDescription.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/ShipperConfigTypeDescription.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/ConditionsImpl.java 8bbff8f ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/FieldsImpl.java 68cd0e2 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/FilterDescriptorImpl.java 4e11715 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/FilterGrokDescriptorImpl.java 995f76b ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/FilterKeyValueDescriptorImpl.java 8e89990 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/InputConfigImpl.java a4eba8e ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/InputDescriptorImpl.java 54b4b9b ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/InputFileBaseDescriptorImpl.java 51c7ec8 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/InputS3FileDescriptorImpl.java 277a57c ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java 5fdbbab ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapDateDescriptorImpl.java 2e54e7a ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldCopyDescriptorImpl.java 4a8d746 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldDescriptorImpl.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldNameDescriptorImpl.java bd32018 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldValueDescriptorImpl.java 599e152 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java 3c21fd8 ambari-logsearch/ambari-logsearch-logfeeder/docs/filter.md 129279b ambari-logsearch/ambari-logsearch-logfeeder/docs/input.md 661eeb8 ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 7ec439a ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/filter/FilterJSONTest.java 7abf177 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/common/ShipperConfigDescriptionStorage.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java 6d1382d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/InfoManager.java f6d0449 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/response/ShipperConfigDescriptionData.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/InfoResource.java 6ea0bab Diff: https://reviews.apache.org/r/60695/diff/1/ Testing --- Tested on docker, the end point works fine. Thanks, Miklos Gergely
Review Request 60682: Move the Log Search ZK config root to the connect string
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60682/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21413 https://issues.apache.org/jira/browse/AMBARI-21413 Repository: ambari Description --- Each path that are looked up contain the root in LogSearchConfigZK, so it makes sense to move it into the zk connect string. Diffs - ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java 6d362031ea0a4b48dc8651daa6772e8bf4f1b411 Diff: https://reviews.apache.org/r/60682/diff/1/ Testing --- Tested on docker, each REST API end point still works fine. Thanks, Miklos Gergely
Re: Review Request 60634: Create property descriptions for internal Log Feeder configs
/util/LogFeederUtil.java 0fdc21f ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogfeederHDFSUtil.java c975b99 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SSLUtil.java d963de3 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/filter/FilterJSONTest.java 8f75c3a ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputManagerTest.java e9bbe7e ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java 1461352 ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/logfeeder.properties 19027d1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/LogSearchConfigConfigurer.java 978e91a Diff: https://reviews.apache.org/r/60634/diff/1/ Testing --- Tested on docker. Thanks, Miklos Gergely
Review Request 60634: Create property descriptions for internal Log Feeder configs
-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogfeederHDFSUtil.java c975b99 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SSLUtil.java d963de3 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/filter/FilterJSONTest.java 8f75c3a ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputManagerTest.java e9bbe7e ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java 1461352 ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/logfeeder.properties 19027d1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/LogSearchConfigConfigurer.java 978e91a Diff: https://reviews.apache.org/r/60634/diff/1/ Testing --- Tested on docker. Thanks, Miklos Gergely
Re: Review Request 60565: Force to use JDK 8 for Ambari Server maven build
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60565/#review179371 --- Ship it! Ship It! - Miklos Gergely On June 30, 2017, 10:37 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60565/ > --- > > (Updated June 30, 2017, 10:37 a.m.) > > > Review request for Ambari, Attila Doroszlai, Attila Magyar, Laszlo Puskas, > and Sebastian Toader. > > > Bugs: AMBARI-21380 > https://issues.apache.org/jira/browse/AMBARI-21380 > > > Repository: ambari > > > Description > --- > > add source/target versions for compiler plugin and uses 1.8 there > > > Diffs > - > > ambari-server/pom.xml 4b75d9b > > > Diff: https://reviews.apache.org/r/60565/diff/1/ > > > Testing > --- > > done, failing with jdk 7 > > > Thanks, > > Oliver Szabo > >
Review Request 60568: Log Feeder input config attribute "tail" should be clearer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60568/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21387 https://issues.apache.org/jira/browse/AMBARI-21387 Repository: ambari Description --- tail = true should mean that only the latest log file is loaded tail = false should mean that the previous rolled over log files are loaded too, but not followed AbstractInputFile must be prepared for previous rolled over files to check in even when the next part is actively handled, as the checking in is done when the data was moved to solr, so all file related fields were moved to maps, so the checkin can identify which file's status is checked in. Log files are sorted alphabetically after identified, thus ensureing that the first one is the active one. In practice they were sorted already, but the File.listFiles function doesn't ensures it according to it's documentation, so it's better to do so explicitly. The thread that is responsible for deleting unused checkpoint files should do one more check before deleting a checkpoint file because the file it belongs to is no more there, or it's file key doesn't matches the one in the checkpoint file: first it should look up if there is any other file in the same folder which have the same file key, in which case it is assumed that the file was renamed since. As the code that's looking for a checkpoint file to determine the next line to load does that by the name of the checkpoint file, which is generated from the file key it will successfuly identify it irrelevant of the name of the file. I know, it's a bit complicated :) But I couldn't put it into words less complicated. Also fixed Log Search Config ZK bug not to handle not node data change related events. Diffs - ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java 26375e1 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java ab50eb7 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java c36f96b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java fc40ca4 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 19894ae ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java 2b19503 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputManagerTest.java e9bbe7e Diff: https://reviews.apache.org/r/60568/diff/1/ Testing --- Tested on local vagrant cluster, by loading previous files again and again, and by renaming rolled over files. In both cases no duplicate data were loaded, checkpoints were handled correctly. Thanks, Miklos Gergely
Re: Review Request 60389: Extract Log Search part from stack advisor to a separate service advisor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60389/ --- (Updated June 30, 2017, 10:34 a.m.) Review request for Ambari, Alejandro Fernandez, Oliver Szabo, and Robert Nettleton. Changes --- return an empty list of validators instead of null Bugs: AMBARI-21333 https://issues.apache.org/jira/browse/AMBARI-21333 Repository: ambari Description --- Extract Log Search service advisor from the stack advisor. Also created an empty Ambari Infra service advisor. Diffs (updated) - ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/service_advisor.py PRE-CREATION ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 726514b ambari-server/src/test/python/common-services/LOGSEARCH/test_service_advisor.py PRE-CREATION ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py c28ee2a Diff: https://reviews.apache.org/r/60389/diff/3/ Changes: https://reviews.apache.org/r/60389/diff/2-3/ Testing --- Tested locally. Thanks, Miklos Gergely
Re: Review Request 60501: Create property descriptions for internal Log Search configs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60501/#review179245 --- Ship it! Ship It! - Miklos Gergely On June 28, 2017, 3:51 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60501/ > --- > > (Updated June 28, 2017, 3:51 p.m.) > > > Review request for Ambari, Andrii Babiichuk, Miklos Gergely, and Robert > Nettleton. > > > Bugs: AMBARI-21366 > https://issues.apache.org/jira/browse/AMBARI-21366 > > > Repository: ambari > > > Description > --- > > - added new annotation to create descrptions for all internal logsearch > properties (currently only for logsearch.properties, not logfeeder.properties) > - create /info endpoints to get access the property descriptions > (/api/v1/info/properties) > - speed up doc generation (for this change and for swagger as well...as it > generated every time you hit the rest api call, now it will generated during > startup on a thread, from that point you will be able to acces the docs from > the memory) > > Also some UI changes: > - there was a /api/v1/pulbic/config call, instead of it, uses > /api/v1/info/auth with returning a simple map json (not that weird name value > structure)...use that endpoint in login.html > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchPropertyDescription.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/LogSearch.java > 45281b6 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/common/ApiDocStorage.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java > a2a7f5e > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/common/PropertyDescriptionStorage.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/ApiDocConfig.java > 9d4e96e > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java > 54cc10c > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/LogSearchHttpHeaderConfig.java > cb8c097 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java > 441b628 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java > 5981bcc > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionPropsConfig.java > 7d37efd > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrEventHistoryPropsConfig.java > a117501 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrKerberosConfig.java > 7cf79b0 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java > 482438a > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java > 73de0ee > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/InfoManager.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/PublicManager.java > cbab651 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/response/PropertyDescriptionData.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/InfoResource.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/PublicResource.java > df83d44 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/SwaggerResource.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/util/SSLUtil.java > d4b6544 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchKRBAuthenticationFilter.java > 1b77753 >
Re: Review Request 60514: Use JDK 8 maven compiler plugin for Log Search and Infra projects
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60514/#review179243 --- Ship it! Ship It! - Miklos Gergely On June 28, 2017, 7:34 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60514/ > --- > > (Updated June 28, 2017, 7:34 p.m.) > > > Review request for Ambari, Attila Doroszlai, Miklos Gergely, Robert > Nettleton, and Sebastian Toader. > > > Bugs: AMBARI-21369 > https://issues.apache.org/jira/browse/AMBARI-21369 > > > Repository: ambari > > > Description > --- > > Start to use jdk 8 compiler maven plugin for Log Search and Infra > > > Diffs > - > > ambari-infra/ambari-infra-manager/pom.xml b7708c2 > ambari-infra/ambari-infra-solr-plugin/pom.xml c890cec > ambari-infra/pom.xml 07adb3e > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml ae2150e > ambari-logsearch/ambari-logsearch-server/pom.xml fc4029b > ambari-logsearch/pom.xml af1dc6a > > > Diff: https://reviews.apache.org/r/60514/diff/1/ > > > Testing > --- > > done. (fails with jdk 7) > > > Thanks, > > Oliver Szabo > >
Re: Review Request 60389: Extract Log Search part from stack advisor to a separate service advisor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60389/ --- (Updated June 26, 2017, 9:58 a.m.) Review request for Ambari, Alejandro Fernandez, Oliver Szabo, and Robert Nettleton. Changes --- Fix unit tests Bugs: AMBARI-21333 https://issues.apache.org/jira/browse/AMBARI-21333 Repository: ambari Description --- Extract Log Search service advisor from the stack advisor. Also created an empty Ambari Infra service advisor. Diffs (updated) - ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/service_advisor.py PRE-CREATION ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 726514b ambari-server/src/test/python/common-services/LOGSEARCH/test_service_advisor.py PRE-CREATION ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py c28ee2a Diff: https://reviews.apache.org/r/60389/diff/2/ Changes: https://reviews.apache.org/r/60389/diff/1-2/ Testing --- Tested locally. Thanks, Miklos Gergely
Review Request 60389: Extract Log Search part from stack advisor to a separate service advisor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60389/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21333 https://issues.apache.org/jira/browse/AMBARI-21333 Repository: ambari Description --- Extract Log Search service advisor from the stack advisor. Also created an empty Ambari Infra service advisor. Diffs - ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/service_advisor.py PRE-CREATION ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/service_advisor.py PRE-CREATION ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 726514b Diff: https://reviews.apache.org/r/60389/diff/1/ Testing --- Tested locally. Thanks, Miklos Gergely
Re: Review Request 60196: Fail to create solr clients in Log Search / Log Feeder with openjdk8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60196/#review178433 --- Ship it! Ship It! - Miklos Gergely On June 20, 2017, 7:46 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60196/ > --- > > (Updated June 20, 2017, 7:46 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-21277 > https://issues.apache.org/jira/browse/AMBARI-21277 > > > Repository: ambari > > > Description > --- > > remvoe commons-httpclient dependency from logsearch/logfeeder > > > Diffs > - > > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml ce784cb > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java > d37a3bb > ambari-logsearch/ambari-logsearch-server/pom.xml 7cd90eb > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java > df00c15 > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrCollectionConfigurer.java > f2d022e > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java > d99694b > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/CreateCollectionHandler.java > 752a1e1 > > > Diff: https://reviews.apache.org/r/60196/diff/3/ > > > Testing > --- > > done. manually checked the applications. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 59944: Add ability to Log Search to test a log entry if it is parseable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59944/ --- (Updated June 20, 2017, 11:58 a.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- return Map<String, Object> instead of string Bugs: AMBARI-21210 https://issues.apache.org/jira/browse/AMBARI-21210 Repository: ambari Description --- Add a REST API endpoint to the server, and a command line option to the Log Feeder to check if the provided log entry is parseable. Diffs (updated) - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/LogSearchConfig.java 4cbf21f ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/FilterGrokDescriptor.java e85ce97 ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass1.java d7e3c0a ambari-logsearch/ambari-logsearch-config-api/src/test/java/org/apache/ambari/logsearch/config/api/LogSearchConfigClass2.java 198c133 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/LogSearchConfigZK.java 4d10a5b ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/FilterGrokDescriptorImpl.java 7f40b7f ambari-logsearch/ambari-logsearch-logfeeder/pom.xml ce784cb ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java 8d7c69f ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederCommandLine.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/ConfigHandler.java 726ff27 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/common/LogEntryParseTester.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java 70aea65 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java 27d16c4 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/AliasUtil.java 5049b62 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java 94d6558 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/LogFeederUtil.java d8a1fbb ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh 53cd17f ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/logconfig/LogConfigHandlerTest.java 4123dad ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetrcisManagerTest.java 8ee6d00 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/metrics/MetricsManagerTest.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/pom.xml 7cd90eb ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java 2ab5f0a ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java a0db92f ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/ShipperConfigResource.java 71da326 Diff: https://reviews.apache.org/r/59944/diff/2/ Changes: https://reviews.apache.org/r/59944/diff/1-2/ Testing --- Tested on local cluster both the REST API endpoint, the command line tool, and also the original system so there aren't any regression. Thanks, Miklos Gergely
Re: Review Request 60196: Fail to create solr clients in Log Search / Log Feeder with openjdk8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60196/#review178224 --- Ship it! Ship It! - Miklos Gergely On June 19, 2017, 11:52 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60196/ > --- > > (Updated June 19, 2017, 11:52 a.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-21277 > https://issues.apache.org/jira/browse/AMBARI-21277 > > > Repository: ambari > > > Description > --- > > remvoe commons-httpclient dependency from logsearch/logfeeder > > > Diffs > - > > ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 49122e8 > ambari-logsearch/ambari-logsearch-portal/pom.xml a439266 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java > df00c15 > > > Diff: https://reviews.apache.org/r/60196/diff/2/ > > > Testing > --- > > done. manually checked the applications. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 59828: Ability to anonymize data during log processing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59828/ --- (Updated June 19, 2017, 10:34 a.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- fix description Bugs: AMBARI-21181 https://issues.apache.org/jira/browse/AMBARI-21181 Repository: ambari Description (updated) --- Have a new mapper called map_anonymize which accepts a pattern containing elements. After the mapping if the pattern is found in the field, than the parts matching the elements are replaced with a configurable hiding character ( default '*' ). Example: pattern: the secret is / which is confidential log message: Bla bla bla the secret is SECRET1 / SECRET2 which is confidential bla bla after the mapping: Bla bla bla the secret is *** / *** which is confidential bla bla This patch also fixes the checkpoint file writing, which wasn't atomic. The checkpoint files are now written under a temporary name first, than atomically renamed to the proper name, thus making it impossible to have a half written file. Diffs - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapAnonymizeDescriptor.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapDateDescriptor.java f88435f ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldDescriptor.java db086c5 ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldValueDescriptor.java cf37e62 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java 32aded8 ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 906359a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 2bc4e68 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java fba596d ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 8c76785 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java 4bf162b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java ffd6cec ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json e2ed625 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperAnonymizeTest.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/log4j.xml e641018 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java dcb0393 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapField.java b18439c ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java 18744e2 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-hst.json 4ccef74 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-zookeeper.json d3685a4 Diff: https://reviews.apache.org/r/59828/diff/2/ Testing --- Tested on local cluster, and by the new unit tests added. Thanks, Miklos Gergely
Re: Review Request 59828: Ability to anonymize data during log processing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59828/ --- (Updated June 19, 2017, 10:31 a.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Use * instead of X by default to hide sensitive log message parts Bugs: AMBARI-21181 https://issues.apache.org/jira/browse/AMBARI-21181 Repository: ambari Description (updated) --- Have a new mapper called map_anonymize which accepts a pattern containing elements. After the mapping if the pattern is found in the field, than the parts matching the elements are replaced with a configurable hiding character ( default '*' ). Example: pattern: the secret is / which is confidential log message: Bla bla bla the secret is SECRET1 / SECRET2 which is confidential bla bla after the mapping: Bla bla bla the secret is *** / *** which is confidential bla bla This patch also fixes the checkpoint file writing, which wasn't atomic. The checkpoint files are now written under a temporary name first, than atomically renamed to the proper name, thus making it impossible to have a half written file. Diffs (updated) - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapAnonymizeDescriptor.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapDateDescriptor.java f88435f ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldDescriptor.java db086c5 ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldValueDescriptor.java cf37e62 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java 32aded8 ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 906359a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 2bc4e68 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java fba596d ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 8c76785 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java 4bf162b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java ffd6cec ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json e2ed625 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperAnonymizeTest.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/log4j.xml e641018 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java dcb0393 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapField.java b18439c ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java 18744e2 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-hst.json 4ccef74 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-zookeeper.json d3685a4 Diff: https://reviews.apache.org/r/59828/diff/2/ Changes: https://reviews.apache.org/r/59828/diff/1-2/ Testing --- Tested on local cluster, and by the new unit tests added. Thanks, Miklos Gergely
Review Request 59947: Remove unnecessary Log Search stack definition parts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59947/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21212 https://issues.apache.org/jira/browse/AMBARI-21212 Repository: ambari Description --- There are some parts of the Log Search stack definition which can be removed as it is no longer in use: - NiFi input configuration - it is in NiFi stack definition now - dependencies to the *-logsearch-conf config types - they no longer exist Diffs - ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml ab66696 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py 6738c5c ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-nifi.json.j2 fe5c426 Diff: https://reviews.apache.org/r/59947/diff/1/ Testing --- Tested on local cluster, works as before. Thanks, Miklos Gergely
Re: Review Request 59888: Fix ambari-solr-plugin folder structure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59888/#review177282 --- Ship it! Ship It! - Miklos Gergely On June 7, 2017, 5:51 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59888/ > --- > > (Updated June 7, 2017, 5:51 p.m.) > > > Review request for Ambari, Attila Doroszlai, Laszlo Puskas, Miklos Gergely, > Robert Nettleton, and Sebastian Toader. > > > Bugs: AMBARI-21196 > https://issues.apache.org/jira/browse/AMBARI-21196 > > > Repository: ambari > > > Description > --- > > move solr plugin files from "org.apache.ambari.infra.security" folder to > org/apache/ambari/infra/security > > (funny fact: it worked with that folder name, as the class was generated into > the proper place beacase of the package name) > > > Diffs > - > > > ambari-infra/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraKerberosHostValidator.java > > > ambari-infra/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraRuleBasedAuthorizationPlugin.java > > > ambari-infra/ambari-infra-solr-plugin/src/main/java/org.apache.ambari.infra.security/InfraUserRolesLookupStrategy.java > > > > Diff: https://reviews.apache.org/r/59888/diff/1/ > > > Testing > --- > > done > > > Thanks, > > Oliver Szabo > >
Re: Review Request 59846: Get rid deprecated jdk install in the Dockerfile of Log Search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59846/#review177163 --- Ship it! Ship It! - Miklos Gergely On June 6, 2017, 5:14 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59846/ > --- > > (Updated June 6, 2017, 5:14 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: trunk > https://issues.apache.org/jira/browse/trunk > > > Repository: ambari > > > Description > --- > > JDK download location does not exists anymore. (also uses a hash for jdk > version, from now, its required) > > > Diffs > - > > ambari-logsearch/docker/Dockerfile d399fc6 > > > Diff: https://reviews.apache.org/r/59846/diff/1/ > > > Testing > --- > > done, manually. > > > Thanks, > > Oliver Szabo > >
Review Request 59828: Ability to anonymize data during log processing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59828/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21181 https://issues.apache.org/jira/browse/AMBARI-21181 Repository: ambari Description --- Have a new mapper called map_anonymize which accepts a pattern containing elements. After the mapping if the pattern is found in the field, than the parts matching the elements are replaced with a configurable hiding character ( default 'X' ). Example: pattern: the secret is / which is confidential log message: Bla bla bla the secret is SECRET1 / SECRET2 which is confidential bla bla after the mapping: Bla bla bla the secret is XXX / XXX which is confidential bla bla This patch also fixes the checkpoint file writing, which wasn't atomic. The checkpoint files are now written under a temporary name first, than atomically renamed to the proper name, thus making it impossible to have a half written file. Diffs - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapAnonymizeDescriptor.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapDateDescriptor.java f88435f ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldDescriptor.java db086c5 ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/MapFieldValueDescriptor.java cf37e62 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapAnonymizeDescriptorImpl.java PRE-CREATION ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/PostMapValuesAdapter.java 32aded8 ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md 906359a ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java 2bc4e68 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java fba596d ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputManager.java 8c76785 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java 4bf162b ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/FileUtil.java ffd6cec ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json e2ed625 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperAnonymizeTest.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/log4j.xml e641018 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java dcb0393 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapField.java b18439c ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldAnonymize.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java 18744e2 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-hst.json 4ccef74 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-zookeeper.json d3685a4 Diff: https://reviews.apache.org/r/59828/diff/1/ Testing --- Tested on local cluster, and by the new unit tests added. Thanks, Miklos Gergely
Re: Review Request 59689: Fix swagger sytax errors in Log Search REST API doc
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59689/#review176516 --- Ship it! Ship It! - Miklos Gergely On May 31, 2017, 6:53 p.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59689/ > --- > > (Updated May 31, 2017, 6:53 p.m.) > > > Review request for Ambari, Miklos Gergely and Robert Nettleton. > > > Bugs: AMBARI-21159 > https://issues.apache.org/jira/browse/AMBARI-21159 > > > Repository: ambari > > > Description > --- > > Fix sytax errors regarding to generated swagger.yaml file: > - info did not contain version > - getClusterForServiceLog operationId is duplicated > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/conf/ApiDocConfig.java > 630e73a > > ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/AuditLogsResource.java > 5312da8 > > > Diff: https://reviews.apache.org/r/59689/diff/1/ > > > Testing > --- > > done. tried out with swagger online editor > > > Thanks, > > Oliver Szabo > >
Re: Review Request 59683: Design Ambari Infra Component & workflows
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59683/#review176444 --- Ship it! Ship It! - Miklos Gergely On May 31, 2017, 10:31 a.m., Oliver Szabo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59683/ > --- > > (Updated May 31, 2017, 10:31 a.m.) > > > Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian > Toader. > > > Bugs: AMBARI-21155 > https://issues.apache.org/jira/browse/AMBARI-21155 > > > Repository: ambari > > > Description > --- > > - Add design overview to README.md of Infra Manager > - fix one of the rest api endpoint > - fix swagger.yaml generation (version is missing if Info object is not used) > - add a static swagger.yaml > - extend to Dummy Job example: add listeners and write output data into the > filesystem > > > Diffs > - > > ambari-infra/ambari-infra-manager/README.md d3527c4 > ambari-infra/ambari-infra-manager/docs/api/swagger.yaml PRE-CREATION > ambari-infra/ambari-infra-manager/docs/images/batch-1.png PRE-CREATION > ambari-infra/ambari-infra-manager/docs/images/batch-2.png PRE-CREATION > ambari-infra/ambari-infra-manager/docs/images/batch-3.png PRE-CREATION > ambari-infra/ambari-infra-manager/docs/images/batch-4.png PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java > 77f7008 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java > 22e2263 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java > c3d8db6 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java > f495795 > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java > PRE-CREATION > > ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java > 7023957 > > > Diff: https://reviews.apache.org/r/59683/diff/1/ > > > Testing > --- > > done. > > > Thanks, > > Oliver Szabo > >
Re: Review Request 58322: Small fixes in Log Search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58322/ --- (Updated May 30, 2017, 4:18 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- actualize Bugs: AMBARI-20727 https://issues.apache.org/jira/browse/AMBARI-20727 Repository: ambari Description --- Remove some unused codes, suppress some warnings with annotations, fix generics. Diffs (updated) - ambari-logsearch/ambari-logsearch-appender/pom.xml e14d576 ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/LogSearchApiSteps.java bcfc4db ambari-logsearch/ambari-logsearch-logfeeder/pom.xml 5d6f8b6 ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/log4j.xml 8a3d26d ambari-logsearch/ambari-logsearch-server/pom.xml b505c12 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/converter/AbstractLogRequestFacetQueryConverter.java d5e807b ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/ListCollectionHandler.java 124ce40 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java 99d2675 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java cddfc85 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java cb9e806 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/authenticate/LogsearchAuthSuccessHandler.java cf9d200 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchKrbFilter.java f4e70d7 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/web/model/JWTAuthenticationToken.java 35175b6 ambari-logsearch/ambari-logsearch-server/src/main/resources/log4j.xml 547841a ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java 44fbd4d ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-log4j.xml.j2 a818fe0 ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-log4j.xml.j2 63f9f0a ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java c949ca2 Diff: https://reviews.apache.org/r/58322/diff/3/ Changes: https://reviews.apache.org/r/58322/diff/2-3/ Testing --- Tested on local cluster Thanks, Miklos Gergely
Re: Review Request 59633: Create .md files to describe Log Search input configurations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59633/ --- (Updated May 30, 2017, 1:48 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- fixed minor issues in the documentation + removed unused parameter start_position Bugs: AMBARI-21144 https://issues.apache.org/jira/browse/AMBARI-21144 Repository: ambari Description (updated) --- Create .md files to describe Log Search input configurations Also removed unused input parameter start_position Diffs (updated) - ambari-logsearch/ambari-logsearch-config-api/src/main/java/org/apache/ambari/logsearch/config/api/model/inputconfig/InputDescriptor.java c41da93 ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/InputDescriptorImpl.java 94dcc2a ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapDateDescriptorImpl.java 9daad2b ambari-logsearch/ambari-logsearch-logfeeder/README.md d00eb57 ambari-logsearch/ambari-logsearch-logfeeder/docs/filter.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/docs/input.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/docs/inputConfig.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/AbstractInputFile.java cfa1903 ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperDate.java 5d34c06 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/input/InputFileTest.java 3a5f31e ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/samples/config/config_audit.json b7d0a6e ambari-logsearch/ambari-logsearch-logfeeder/src/test/resources/samples/config/config_service.json 2e96fd7 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInput.java e0f7564 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java 3147402 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/global.config.json 7ac29dd ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/global.config.json.j2 f337527 Diff: https://reviews.apache.org/r/59633/diff/2/ Changes: https://reviews.apache.org/r/59633/diff/1-2/ Testing (updated) --- Tested on local cluster Thanks, Miklos Gergely
Review Request 59633: Create .md files to describe Log Search input configurations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59633/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21144 https://issues.apache.org/jira/browse/AMBARI-21144 Repository: ambari Description --- Create .md files to describe Log Search input configurations Diffs - ambari-logsearch/ambari-logsearch-logfeeder/docs/filter.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/docs/input.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/docs/inputConfig.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/docs/postMapValues.md PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperDate.java 5d34c06 Diff: https://reviews.apache.org/r/59633/diff/1/ Testing --- Nothing to test Thanks, Miklos Gergely
Re: Review Request 59563: Log Search REST API validate input configurations and log level filters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59563/ --- (Updated May 25, 2017, 2:21 p.m.) Review request for Ambari, Oliver Szabo and Robert Nettleton. Changes --- Fix issue where Map<String, List<..>> wasn't validated properly. Bugs: AMBARI-21119 https://issues.apache.org/jira/browse/AMBARI-21119 Repository: ambari Description --- Log Search server REST API should validate that the received input configuration / log level filter is in the valid format. Diffs (updated) - ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldNameDescriptorImpl.java 333cb67 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java 44d91a9 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerConditions.java 9cd9710 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFields.java 5f570da ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilter.java 0190c01 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterDeserializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterGrok.java a8c4a7a ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterJson.java 3c0ed17 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterKeyValue.java dcee25d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInput.java fe83fe4 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputConfig.java e3dc0d1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputDeserializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputFile.java 5c547ad ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputFileBase.java df21d0d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputS3File.java 8e9acf0 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerLogLevelFilter.java 2a00802 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerLogLevelFilterMap.java 3088db1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java dcacceb ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldCopy.java b0bea83 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldName.java 000b29d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldValue.java 6152de5 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValues.java 5f361c9 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesList.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListDeserializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesListSerializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesSerializer.java 7543677 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/ShipperConfigResource.java a7d99c9 Diff: https://reviews.apache.org/r/59563/diff/2/ Changes: https://reviews.apache.org/r/59563/diff/1-2/ Testing --- Tested in local docker environment. Thanks, Miklos Gergely
Review Request 59563: Log Search REST API validate input configurations and log level filters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59563/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21119 https://issues.apache.org/jira/browse/AMBARI-21119 Repository: ambari Description --- Log Search server REST API should validate that the received input configuration / log level filter is in the valid format. Diffs - ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldNameDescriptorImpl.java 333cb67 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/manager/ShipperConfigManager.java 44d91a9 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerConditions.java 9cd9710 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFields.java 5f570da ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilter.java 0190c01 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterDeserializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterGrok.java a8c4a7a ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterJson.java 3c0ed17 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerFilterKeyValue.java dcee25d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInput.java fe83fe4 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputConfig.java e3dc0d1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputDeserializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputFile.java 5c547ad ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputFileBase.java df21d0d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerInputS3File.java 8e9acf0 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerLogLevelFilter.java 2a00802 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerLogLevelFilterMap.java 3088db1 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapDate.java dcacceb ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldCopy.java b0bea83 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldName.java 000b29d ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerMapFieldValue.java 6152de5 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValues.java 5f361c9 ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/model/common/LSServerPostMapValuesDeserializer.java PRE-CREATION ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/rest/ShipperConfigResource.java a7d99c9 Diff: https://reviews.apache.org/r/59563/diff/1/ Testing --- Tested in local docker environment. Thanks, Miklos Gergely
Review Request 59451: Add Log Feeder input config to HDP 3.0 hooks too
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59451/ --- Review request for Ambari, Oliver Szabo and Robert Nettleton. Bugs: AMBARI-21090 https://issues.apache.org/jira/browse/AMBARI-21090 Repository: ambari Description --- HDP 3.0 clusters need their input configs too Diffs - ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/params.py 5dcd39b ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py e9f2283 Diff: https://reviews.apache.org/r/59451/diff/1/ Testing --- done on local cluster. Thanks, Miklos Gergely
