Hi Radu, Excellent work on this one. Apologies for being a bit distant. Loads of work myself but glad to be re-engaging with you now. Let me take a look at this review request and I will get back to you as soon as I can. Do you have reports substantiated on the OODT wiki briefly describing the key contributions of the entire work which is proposed here? Thanks, and again. Great work and drive. Lewis
On Thu, Jul 30, 2015 at 2:54 PM, Radu Manole <manole.v.r...@gmail.com> wrote: > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36953/ > Review request for oodt, Lewis McGibbney and Chris Mattmann. > By Radu Manole. > *Repository: * oodt > Description > > Separated the FileManager logic from rpc communication createing interfaces > for server/client side (FileManagerCient, FileManagerServer).Refactor > existing XMLRPC server/client so it would use communication interfaces. > Created Avro schemas and protocol for ipc communication, (classes that have > the prefixe Avro), with a Factory class AvroTypeFactory that transforms from > oodt to avro and from avro to oodt classes. Implement avro server/client. For > changeability I created a Factory (RpcCommunicationFactory) that can be > configurable to use either avro or xmlrpc for IPC. Because of this, anywhere > in project where on instantiation of server/client rewrite to use this > factory. > > Testing > > Used existing tests, applied them for avro ipc. > > Diffs > > - > trunk/app/fmbrowser/src/main/java/org/apache/oodt/cas/filemgr/browser/model/CasDB.java > (1693501) > - > trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/action/FilemgrUniquenessChecker.java > (1693501) > - > trunk/crawler/src/main/java/org/apache/oodt/cas/crawl/comparator/FilemgrUniquenessCheckComparator.java > (1693501) > - trunk/crawler/src/site/xdoc/user/index.xml (1693501) > - trunk/filemgr/pom.xml (1693501) > - trunk/filemgr/src/main/avro/types/AvroComplexQuery.avsc > (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroElement.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroExtractorSpec.avsc > (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroFileTransferStatus.avsc > (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroFilterAlgor.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroMetadata.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroMimeType.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroProduct.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroProductPage.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroProductType.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroQuery.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroQueryCriteria.avsc > (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroQueryFilter.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroQueryResult.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroReference.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/AvroTypeHandler.avsc (PRE-CREATION) > - trunk/filemgr/src/main/avro/types/protocol.avdl (PRE-CREATION) > - trunk/filemgr/src/main/bin/filemgr (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/LuceneCatalog.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/AbstractDeleteProductCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/DeleteProductByIdCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/FileManagerCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/GetFirstPageCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/GetLastPageCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/GetNextPageCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/GetNumProductsCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/GetPrevPageCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/GetProductPercentTransferredCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/IngestProductCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/RetrieveFilesCliAction.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/AvroLocalDataTransferer.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/InPlaceDataTransferer.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/LocalDataTransferer.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/RemoteDataTransferer.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/ingest/LocalCache.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/ingest/StdIngester.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/metadata/ProductMetKeys.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/metadata/extractors/examples/TikaAutoDetectExtractor.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/AvroFileManagerClient.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/AvroFileManagerServer.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/FileManager.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/FileManagerClient.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/FileManagerServer.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/XmlRpcFileManagerClient.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/XmlRpcFileManagerServer.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/rpc/AvroFileManagerClientFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/rpc/AvroFileManagerServerFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/rpc/FileManagerClientFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/rpc/FileManagerServerFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/rpc/XmlRpcFileManagerClientFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/system/rpc/XmlRpcFileManagerServerFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/CatalogSearch.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/DeleteProduct.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/ExpImpCatalog.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/MetadataBasedProductMover.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/MetadataDumper.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/ProductDumper.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/QueryTool.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/SolrIndexer.java > (1693501) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/AvroTypeFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/RpcCommunicationFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java > (1693501) > - trunk/filemgr/src/main/resources/.~lock.CAS File Manager User > Guide.doc# (PRE-CREATION) > - trunk/filemgr/src/main/resources/filemgr.properties (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/TestFileManagerCli.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/UseMockClientCmdLineActionStore.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/DummyFileManagerClient.java > (PRE-CREATION) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestAddProductTypeCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestDeleteProductByIdCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestDeleteProductByNameCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestDumpMetadataCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetCurrentTransferCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetCurrentTransfersCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetFilePercentTransferredCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetFirstPageCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetLastPageCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetNextPageCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetNumProductsCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetPrevPageCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetProductByIdCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetProductByNameCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetProductPercentTransferredCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestGetProductTypeByNameCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestHasProductCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestIngestProductCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestLuceneQueryCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/action/TestSqlQueryCliAction.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestCachedIngester.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestLocalCache.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestRmiCache.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestStdIngester.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/structs/type/TestTypeHandler.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/MockFileManagerClient.java > (PRE-CREATION) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestRpcFileManager.java > (PRE-CREATION) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestRpcFileManagerClient.java > (PRE-CREATION) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/rpc/TestServerClientFactories.java > (PRE-CREATION) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestExpImpCatalog.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestMetadataBasedProductMover.java > (1693501) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/util/TestAvroTypesFactory.java > (PRE-CREATION) > - > trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/util/TestXmlRpcStructFactory.java > (1693501) > - trunk/filemgr/src/test/resources/createProduct/test.txt > (PRE-CREATION) > - trunk/filemgr/src/test/resources/filemgr.properties (1693501) > - > trunk/mvn/archetypes/radix/src/main/resources/archetype-resources/filemgr/src/main/resources/bin/filemgr > (1693501) > - > trunk/pcs/core/src/main/java/org/apache/oodt/pcs/util/FileManagerUtils.java > (1693501) > - trunk/pge/src/main/java/org/apache/oodt/cas/pge/staging/FileStager.java > (1693501) > - trunk/pge/src/main/java/org/apache/oodt/cas/pge/util/XmlHelper.java > (1693501) > - > trunk/pge/src/test/java/org/apache/oodt/cas/pge/staging/TestFileStager.java > (1693501) > - > trunk/profile/src/main/java/org/apache/oodt/profile/handlers/cas/CASProfileHandler.java > (1693501) > - > trunk/webapp/components/src/main/java/org/apache/oodt/cas/webcomponents/filemgr/FileManagerConn.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/CASProductHandler.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataDeliveryServlet.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DatasetDeliveryServlet.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/services/CasProductJaxrsService.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/servlets/CasProductJaxrsServlet.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rdf/RDFDatasetServlet.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rdf/RDFProductServlet.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductServlet.java > (1693501) > - > trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java > (1693501) > > View Diff <https://reviews.apache.org/r/36953/diff/> > -- *Lewis*