Re: Review Request 55511: Ambari views : Adding new FileSystem support to views should work without code changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55511/#review161682 --- Ship it! Ship It! - DIPAYAN BHOWMICK On Jan. 13, 2017, 5:57 p.m., Nitiraj Rathore wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55511/ > --- > > (Updated Jan. 13, 2017, 5:57 p.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav > Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin > Rajeev. > > > Bugs: AMBARI-19528 > https://issues.apache.org/jira/browse/AMBARI-19528 > > > Repository: ambari > > > Description > --- > > Before this patch : > Right now when new file-system support has to be added to views, we need to > add code to include several more key value pairs into the conf to make it > work. > Usually these key value pairs are file-system class names, secret key and > access key. > > In this patch: > 1. load the complete core-site and hdfs-site into the conf for creating > filesystem. This will load any secret key, access key configured. It will > also load any other necessary parameters. > 2. Create a new view setting's parameter per view which will have semi colon > separated key=value pairs that needs to be loaded into conf for connecting to > the filesystem. This will cover any extra parameters. > 3. changes in pom.xml -> included Amazon S3 related jars and excluded > conflicting jars. > > > Diffs > - > > ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java > 10f216e > > ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java > 6a4f04a > ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java > 7f7c19b > contrib/views/commons/pom.xml 4b7ee3a > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/FileOperationService.java > bc3d11d > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/HdfsService.java > 91eebcf > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UploadService.java > 97253ad > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UserService.java > dc303ce > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/ViewPropertyHelper.java > PRE-CREATION > contrib/views/files/pom.xml ad9af4f > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java > 96d3541 > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FileBrowserService.java > adaa6c9 > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java > 051e40d > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/HelpService.java > 92af2d5 > contrib/views/files/src/main/resources/view.xml 640cee9 > contrib/views/hive-next/pom.xml 2e7beab > > contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/internal/HdfsApiSupplier.java > 1cc1211 > > contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/files/FileService.java > 654bfcc > > contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/utils/SharedObjectsFactory.java > dd0b715 > contrib/views/hive-next/src/main/resources/view.xml 1107cd1 > contrib/views/hive20/pom.xml 6fc30f0 > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/internal/HdfsApiSupplier.java > e66b9ab > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/browser/FileService.java > 96e9554 > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/files/FileService.java > a3623e9 > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/utils/SharedObjectsFactory.java > 5bc6070 > contrib/views/hive20/src/main/resources/view.xml 315d0a9 > contrib/views/jobs/src/main/resources/ui/.gitignore 503a8a9 > contrib/views/pig/pom.xml d7d2de9 > > contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/files/FileService.java > 509b20e > > contrib/views/pig/src/main/java/org/apache/ambari/view/pig/utils/UserLocalObjects.java > 8293899 > contrib/views/pig/src/main/resources/view.xml da8ffdd > contrib/views/pom.xml 13cfb23 > contrib/views/utils/pom.xml c3cdab5 > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java > d505e9a > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsApi.java > c7ae952 > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java > 8bc3ca5 > contrib/views/wfmanager/pom.xml
Re: Review Request 55511: Ambari views : Adding new FileSystem support to views should work without code changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55511/#review161558 --- Ship it! Ship It! - Gaurav Nagar On Jan. 13, 2017, 5:57 p.m., Nitiraj Rathore wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55511/ > --- > > (Updated Jan. 13, 2017, 5:57 p.m.) > > > Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav > Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin > Rajeev. > > > Bugs: AMBARI-19528 > https://issues.apache.org/jira/browse/AMBARI-19528 > > > Repository: ambari > > > Description > --- > > Before this patch : > Right now when new file-system support has to be added to views, we need to > add code to include several more key value pairs into the conf to make it > work. > Usually these key value pairs are file-system class names, secret key and > access key. > > In this patch: > 1. load the complete core-site and hdfs-site into the conf for creating > filesystem. This will load any secret key, access key configured. It will > also load any other necessary parameters. > 2. Create a new view setting's parameter per view which will have semi colon > separated key=value pairs that needs to be loaded into conf for connecting to > the filesystem. This will cover any extra parameters. > 3. changes in pom.xml -> included Amazon S3 related jars and excluded > conflicting jars. > > > Diffs > - > > ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java > 10f216e > > ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java > 6a4f04a > ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java > 7f7c19b > contrib/views/commons/pom.xml 4b7ee3a > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/FileOperationService.java > bc3d11d > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/HdfsService.java > 91eebcf > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UploadService.java > 97253ad > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UserService.java > dc303ce > > contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/ViewPropertyHelper.java > PRE-CREATION > contrib/views/files/pom.xml ad9af4f > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java > 96d3541 > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FileBrowserService.java > adaa6c9 > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java > 051e40d > > contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/HelpService.java > 92af2d5 > contrib/views/files/src/main/resources/view.xml 640cee9 > contrib/views/hive-next/pom.xml 2e7beab > > contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/internal/HdfsApiSupplier.java > 1cc1211 > > contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/files/FileService.java > 654bfcc > > contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/utils/SharedObjectsFactory.java > dd0b715 > contrib/views/hive-next/src/main/resources/view.xml 1107cd1 > contrib/views/hive20/pom.xml 6fc30f0 > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/internal/HdfsApiSupplier.java > e66b9ab > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/browser/FileService.java > 96e9554 > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/files/FileService.java > a3623e9 > > contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/utils/SharedObjectsFactory.java > 5bc6070 > contrib/views/hive20/src/main/resources/view.xml 315d0a9 > contrib/views/jobs/src/main/resources/ui/.gitignore 503a8a9 > contrib/views/pig/pom.xml d7d2de9 > > contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/files/FileService.java > 509b20e > > contrib/views/pig/src/main/java/org/apache/ambari/view/pig/utils/UserLocalObjects.java > 8293899 > contrib/views/pig/src/main/resources/view.xml da8ffdd > contrib/views/pom.xml 13cfb23 > contrib/views/utils/pom.xml c3cdab5 > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java > d505e9a > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsApi.java > c7ae952 > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java > 8bc3ca5 > contrib/views/wfmanager/pom.xml 95b988f
Review Request 55511: Ambari views : Adding new FileSystem support to views should work without code changes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55511/ --- Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin Rajeev. Bugs: AMBARI-19528 https://issues.apache.org/jira/browse/AMBARI-19528 Repository: ambari Description --- Before this patch : Right now when new file-system support has to be added to views, we need to add code to include several more key value pairs into the conf to make it work. Usually these key value pairs are file-system class names, secret key and access key. In this patch: 1. load the complete core-site and hdfs-site into the conf for creating filesystem. This will load any secret key, access key configured. It will also load any other necessary parameters. 2. Create a new view setting's parameter per view which will have semi colon separated key=value pairs that needs to be loaded into conf for connecting to the filesystem. This will cover any extra parameters. 3. changes in pom.xml -> included Amazon S3 related jars and excluded conflicting jars. Diffs - ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java 10f216e ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java 6a4f04a ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java 7f7c19b contrib/views/commons/pom.xml 4b7ee3a contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/FileOperationService.java bc3d11d contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/HdfsService.java 91eebcf contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UploadService.java 97253ad contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UserService.java dc303ce contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/ViewPropertyHelper.java PRE-CREATION contrib/views/files/pom.xml ad9af4f contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java 96d3541 contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FileBrowserService.java adaa6c9 contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java 051e40d contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/HelpService.java 92af2d5 contrib/views/files/src/main/resources/view.xml 640cee9 contrib/views/hive-next/pom.xml 2e7beab contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/internal/HdfsApiSupplier.java 1cc1211 contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/files/FileService.java 654bfcc contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/utils/SharedObjectsFactory.java dd0b715 contrib/views/hive-next/src/main/resources/view.xml 1107cd1 contrib/views/hive20/pom.xml 6fc30f0 contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/internal/HdfsApiSupplier.java e66b9ab contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/browser/FileService.java 96e9554 contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/files/FileService.java a3623e9 contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/utils/SharedObjectsFactory.java 5bc6070 contrib/views/hive20/src/main/resources/view.xml 315d0a9 contrib/views/jobs/src/main/resources/ui/.gitignore 503a8a9 contrib/views/pig/pom.xml d7d2de9 contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/files/FileService.java 509b20e contrib/views/pig/src/main/java/org/apache/ambari/view/pig/utils/UserLocalObjects.java 8293899 contrib/views/pig/src/main/resources/view.xml da8ffdd contrib/views/pom.xml 13cfb23 contrib/views/utils/pom.xml c3cdab5 contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java d505e9a contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsApi.java c7ae952 contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java 8bc3ca5 contrib/views/wfmanager/pom.xml 95b988f contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/FileServices.java ab286b2 contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java 327d8fc contrib/views/wfmanager/src/main/resources/view.xml c5d4a5b Diff: https://reviews.apache.org/r/55511/diff/ Testing --- manual testing done on amazon s3 cluster and normal hdfs cluster. Thanks, Nitiraj Rathore
