[GitHub] chrismattmann commented on issue #101: Solrj incorrect version causes DRAT index to Fail
chrismattmann commented on issue #101: Solrj incorrect version causes DRAT index to Fail URL: https://github.com/apache/drat/issues/101#issuecomment-343789678 fixed in https://github.com/apache/drat/commit/3ca8328aeb0d6239df350fdd75e8fac0e2f7ef90 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr
chrismattmann commented on issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr URL: https://github.com/apache/drat/issues/100#issuecomment-343789644 fixed by upgrading to Apache OODT 1.2.1 (soon to be released hopefully). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr
chrismattmann commented on issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr URL: https://github.com/apache/drat/issues/100#issuecomment-343789442 tested on https://drat.apache.org/demo/ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04
chrismattmann commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04 URL: https://github.com/apache/drat/issues/104#issuecomment-343789528 Make sure you have the these [environment variables](https://github.com/apache/drat/wiki/Installation#useful-environment-variables) set. Try building after that. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr
chrismattmann commented on issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr URL: https://github.com/apache/drat/issues/100#issuecomment-343789412 fixed in https://github.com/apache/drat/commit/3ca8328aeb0d6239df350fdd75e8fac0e2f7ef90 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #103: Homepage not working
chrismattmann commented on issue #103: Homepage not working URL: https://github.com/apache/drat/issues/103#issuecomment-343794683 agree @ottlinger - please come on over to #98 and let's discuss. Can you help put up an "under construction page." Until #98 is done? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #98: Make website at drat.apache.org
chrismattmann commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-348292065 ping @shivikathapar @nipunhere again This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nipunhere commented on issue #98: Make website at drat.apache.org
nipunhere commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-351820397 Hi @chrismattmann, @wmburke I have just generated a PR for the website here: https://github.com/apache/drat/pull/110 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nipunhere opened a new pull request #110: Website first draft
nipunhere opened a new pull request #110: Website first draft URL: https://github.com/apache/drat/pull/110 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger closed issue #103: Homepage not working
ottlinger closed issue #103: Homepage not working URL: https://github.com/apache/drat/issues/103 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann opened a new issue #101: Solrj incorrect version causes DRAT index to Fail
chrismattmann opened a new issue #101: Solrj incorrect version causes DRAT index to Fail URL: https://github.com/apache/drat/issues/101 As described here: https://s.apache.org/FHwY Figure out what to do, either upgrade to Apache OODT 1.9-SNAPSHOT and figure out a workaround or work with the Apache OODT PMC to roll a 1.2.1 that fixes the Solrj pinned version. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann opened a new issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr
chrismattmann opened a new issue #100: Issues with Default Tomcat Installation via RADIX on Ubuntu and Solr URL: https://github.com/apache/drat/issues/100 Solr fails to start on Tomcat in DRAT on Ubuntu specifically drat-vm.apache.org. Details described here: https://lists.apache.org/list.html?dev@drat.apache.org:2017-11 Seems to work fine on Mac. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed issue #102: DRAT Workflow Page doesn't properly return to Home Page when under a ProxyPath
chrismattmann closed issue #102: DRAT Workflow Page doesn't properly return to Home Page when under a ProxyPath URL: https://github.com/apache/drat/issues/102 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThejanW commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04
ThejanW commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04 URL: https://github.com/apache/drat/issues/104#issuecomment-344315257 Oh! didn't notice that. thanks @chrismattmann :+1: This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04
chrismattmann commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04 URL: https://github.com/apache/drat/issues/104#issuecomment-344319099 awesome. Can you confirm this fixed your issue @ThejanW and that it builds? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger commented on issue #98: Make website at drat.apache.org
ottlinger commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-344315047 Can anyone supply me with the following information about DRAT: * Twitter-handle (if existing) * Facebook-URL (if existing) * Instagram-URL (if existing) * LinkedIn-URL (if any) * Mailing list (develo...@drat.apache.org?! or us...@drat.apache.org) * Base location of the main repository - or is https://github.com/apache/drat the main repo and no ASF-repo exists at all? I want to tweak the current page before it goes live via [INFRA-15489](https://issues.apache.org/jira/browse/INFRA-15489) Thanks for your help. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger opened a new pull request #106: https://github.com/apache/drat/issues/98 Fix link to statement
ottlinger opened a new pull request #106: https://github.com/apache/drat/issues/98 Fix link to statement URL: https://github.com/apache/drat/pull/106 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed pull request #106: https://github.com/apache/drat/issues/98 Fix link to statement
chrismattmann closed pull request #106: https://github.com/apache/drat/issues/98 Fix link to statement URL: https://github.com/apache/drat/pull/106 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/index.html b/index.html index 7d083e9..0380f27 100644 --- a/index.html +++ b/index.html @@ -34,7 +34,7 @@ - Apache DRAT mission statement + Apache DRAT mission statement What does DRAT stand for? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #98: Make website at drat.apache.org
chrismattmann commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-345927385 ping @shivikathapar and @nipunhere we have a temp website up, can we get going on the actual website thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ThejanW commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04
ThejanW commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04 URL: https://github.com/apache/drat/issues/104#issuecomment-345941271 @chrismattmann I am sorry, I couldn't comment on this soon enough. Yes, it built fine, once I exported the correct variables, thanks @chrismattmann. A future user might benefit from this issue. I'm attaching the build log I generated with mvn -X [build.log](https://github.com/apache/drat/files/1490400/build.log) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed issue #104: DRAT Proteus build failure on Ubuntu 16.04
chrismattmann closed issue #104: DRAT Proteus build failure on Ubuntu 16.04 URL: https://github.com/apache/drat/issues/104 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04
chrismattmann commented on issue #104: DRAT Proteus build failure on Ubuntu 16.04 URL: https://github.com/apache/drat/issues/104#issuecomment-346076477 fixed This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #109: Fresh install doesn't create the tomcat/temp directory
chrismattmann commented on issue #109: Fresh install doesn't create the tomcat/temp directory URL: https://github.com/apache/drat/issues/109#issuecomment-346250211 yea @ThejanW i think there is an even simpler fix here. We just need to update the assembly.xml to create that directory and include a REMOVE.log file in there. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann opened a new issue #109: Fresh install doesn't create the tomcat/temp directory
chrismattmann opened a new issue #109: Fresh install doesn't create the tomcat/temp directory URL: https://github.com/apache/drat/issues/109 Fresh install on Ubuntu does not create the tomcat/temp directory and Proteus won't work on start b/c of this: ``` INFO: Cloning Git / SVN project: [oodt] remote repo: [https://github.com/apache/oodt.git] into /home/mattmann/drat/deploy/tomcat/temp java.io.IOException: /home/mattmann/drat/deploy/tomcat/temp doesn't exist. at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:163) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153) at drat.proteus.DratStartForm.parseAsVersionControlledRepo(DratStartForm.java:207) at drat.proteus.DratStartForm.handleSubmit(DratStartForm.java:127) at drat.proteus.DratStartForm.onSubmit(DratStartForm.java:88) at org.apache.wicket.markup.html.form.Form.delegateSubmit(Form.java:1313) at org.apache.wicket.markup.html.form.Form.process(Form.java:976) at org.apache.wicket.markup.html.form.Form.onFormSubmitted(Form.java:797) at org.apache.wicket.markup.html.form.Form.onFormSubmitted(Form.java:708) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.wicket.RequestListenerInterface.internalInvoke(RequestListenerInterface.java:258) at org.apache.wicket.RequestListenerInterface.invoke(RequestListenerInterface.java:216) at org.apache.wicket.core.request.handler.ListenerInterfaceRequestHandler.invokeListener(ListenerInterfaceRequestHandler.java:241) at org.apache.wicket.core.request.handler.ListenerInterfaceRequestHandler.respond(ListenerInterfaceRequestHandler.java:234) at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:895) at org.apache.wicket.request.RequestHandlerStack.execute(RequestHandlerStack.java:64) at org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:265) at org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:222) at org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:293) at org.apache.wicket.protocol.http.WicketFilter.processRequestCycle(WicketFilter.java:261) at org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:203) at org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:284) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger opened a new pull request #108: Optimize assets and add docs.
ottlinger opened a new pull request #108: Optimize assets and add docs. URL: https://github.com/apache/drat/pull/108 Minified * JS * CSS * images to improve SEO. Added basic docs about it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #98: Make website at drat.apache.org
chrismattmann commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-344074440 hey @ottlinger I think we can use the same repo just in a branch. We do that in OODT and it works out great. OK site-dev branch is up! http://github.com/apache/drat/tree/site-dev Have at it! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #98: Make website at drat.apache.org
chrismattmann commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-344078189 Thanks for the great work @ottlinger I've filed https://issues.apache.org/jira/browse/INFRA-15489 to track pushing this to drat.apache.org via Gitpubsub This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #105: https://github.com/apache/drat/issues/98 Add page skeleton
chrismattmann commented on issue #105: https://github.com/apache/drat/issues/98 Add page skeleton URL: https://github.com/apache/drat/pull/105#issuecomment-344078873 thanks @ottlinger we appreciate your contributions and we will make it as easy as possible on you promise. Thanks again. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger commented on issue #98: Make website at drat.apache.org
ottlinger commented on issue #98: Make website at drat.apache.org URL: https://github.com/apache/drat/issues/98#issuecomment-344072532 @chrismattmann would a separate repository make sense as it may function as a transition repo?! Or would you create a separate drat-site-repo instead?! RAT has a separate repository for that thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger opened a new pull request #105: https://github.com/apache/drat/issues/98 Add page skeleton
ottlinger opened a new pull request #105: https://github.com/apache/drat/issues/98 Add page skeleton URL: https://github.com/apache/drat/pull/105 Adding first page skeleton to prove infra is working. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #105: https://github.com/apache/drat/issues/98 Add page skeleton
chrismattmann commented on issue #105: https://github.com/apache/drat/issues/98 Add page skeleton URL: https://github.com/apache/drat/pull/105#issuecomment-344076918 thanks so much @ottlinger appreciate this! I will file an INFRA ticket now to gitpubsub thisstand by I'll link here. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger commented on issue #105: https://github.com/apache/drat/issues/98 Add page skeleton
ottlinger commented on issue #105: https://github.com/apache/drat/issues/98 Add page skeleton URL: https://github.com/apache/drat/pull/105#issuecomment-344077385 In case of changes I need to resync my fork, right?! A separate repository would have allowed write access for me but I can easily file PRs for changes. Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189425897 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -104,8 +115,20 @@ public void crawl() throws Exception { } @Override - public void index() throws IOException, DratWrapperException { -simpleDratExec(INDEX_CMD, this.path); + public void index() throws IOException, DratWrapperException, InstantiationException, SolrServerException { + solrIndex(); + } + + private synchronized void solrIndex() throws InstantiationException, SolrServerException, IOException { + DratLog idl = new DratLog("INDEXING"); + idl.logInfo("Starting", null); + System.setProperty(FileConstants.SOLR_INDEXER_CONFIG,FileConstants.SOLR_INDEXER_CONFIG_PATH); + SolrIndexer sIndexer = new SolrIndexer(FileConstants.SOLR_DRAT_URL,FileConstants.FILEMGR_URL); + sIndexer.indexAll(false); + sIndexer.commit(); + sIndexer.optimize(); + idl.logInfo("Comepleted",null); Review comment: typo should read `s/Comepleted/Completed/g`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189425907 ## File path: proteus/src/main/java/backend/FileConstants.java ## @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=PathUtils.replaceEnvVariables("[FILEMGR_URL]"); + public static final String SOLR_DRAT_URL=PathUtils.replaceEnvVariables("[SOLR_DRAT_URL]"); + public static final String CLIENT_URL=PathUtils.replaceEnvVariables("[WORKFLOW_URL]"); + public static final String OPSUI_URL=PathUtils.replaceEnvVariables("[OPSUI_URL]"); Review comment: can you add this to recommended environment variables in the DRAT wiki? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189587883 ## File path: proteus/src/main/java/backend/FileConstants.java ## @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=PathUtils.replaceEnvVariables("[FILEMGR_URL]"); + public static final String SOLR_DRAT_URL=PathUtils.replaceEnvVariables("[SOLR_DRAT_URL]"); + public static final String CLIENT_URL=PathUtils.replaceEnvVariables("[WORKFLOW_URL]"); + public static final String OPSUI_URL=PathUtils.replaceEnvVariables("[OPSUI_URL]"); Review comment: ok, got it, so let's just call CLIENT_URL WORKFLOW_URL instead to be consistent and I'm fine with it... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
chrismattmann commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-392183660 @IMS94 are you done testing can I merge? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] IMS94 opened a new pull request #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
IMS94 opened a new pull request #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130 This PR includes changes required for OODT 1.9 update along with the fix for #112. I think this should not be merged since OODT 1.9 is not released yet and is in SNAPSHOT stage still. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-392121883 @chrismattmann I will do one more test run with DRAT and confirm. Then you can merge. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
chrismattmann commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-392119391 I am fine to merge this even with a 1.9-SNAPSHOT dependency. If there aren't any objections in the next 24 hours I will merge... This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #128: "bin/oodt stop" won't stop tomcat causing system to go out of memory
chrismattmann commented on issue #128: "bin/oodt stop" won't stop tomcat causing system to go out of memory URL: https://github.com/apache/drat/issues/128#issuecomment-391891060 hmmm this sounds like a big problem @IMS94 how should we fix it? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on issue #129: Replaced Processwrapper indexing method with SolrIndexer
ahmedifhaam commented on issue #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#issuecomment-390250812 Yes tested on drat repository it self working fine This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189308433 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -104,8 +109,18 @@ public void crawl() throws Exception { } @Override - public void index() throws IOException, DratWrapperException { -simpleDratExec(INDEX_CMD, this.path); + public void index() throws IOException, DratWrapperException, InstantiationException, SolrServerException { +//simpleDratExec(INDEX_CMD, this.path); Review comment: if you commented this out, just take it out. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189311225 ## File path: webapps/solr-webapp/src/main/webapp/WEB-INF/web.xml ## @@ -36,8 +36,9 @@ solr/home - ../solr java.lang.String + ../solr + Review comment: This is an extraneous change not material to this PR so please remove it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189307034 ## File path: proteus/src/main/java/backend/FileConstants.java ## @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=replaceEnvVar("http://${OODT_HOST}:9000;); Review comment: Suggestion here would simply be to read this as FILEMGR_URL which is the standard used in DRAT and OODT, see: https://github.com/apache/drat/wiki/Installation#useful-environment-variables. Same goes for rest of the vars (e.g., WORKFLOW_URL etc.) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189308674 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -104,8 +109,18 @@ public void crawl() throws Exception { } @Override - public void index() throws IOException, DratWrapperException { -simpleDratExec(INDEX_CMD, this.path); + public void index() throws IOException, DratWrapperException, InstantiationException, SolrServerException { +//simpleDratExec(INDEX_CMD, this.path); + solrIndex(); + } + + private void solrIndex() throws InstantiationException, SolrServerException, IOException { + System.setProperty(FileConstants.SOLR_INDEXER_CONFIG,FileConstants.SOLR_INDEXER_CONFIG_PATH); Review comment: TODO: eventually make this work in a multi-user, multi-context environment if necessary. It may well support that but every time I see a System property there's a concern it will bind us to some particular configuration. I think this is fine for now though. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on issue #129: Replaced Processwrapper indexing method with SolrIndexer
ahmedifhaam commented on issue #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#issuecomment-390250812 Yes tested on drat repository it self.Working fine compared the results from the last master same results This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189400645 ## File path: proteus/src/main/java/backend/FileConstants.java ## @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=replaceEnvVar("http://${OODT_HOST}:9000;); Review comment: This is done in the next commit still a new env var is needed SOLR_DRAT_URL This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann closed pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/proteus/src/main/java/backend/FileConstants.java b/proteus/src/main/java/backend/FileConstants.java index f5a69d7..651060e 100644 --- a/proteus/src/main/java/backend/FileConstants.java +++ b/proteus/src/main/java/backend/FileConstants.java @@ -20,6 +20,8 @@ import org.apache.oodt.cas.metadata.util.PathUtils; import java.io.File; +import java.util.regex.Matcher; +import java.util.regex.Pattern; /** * Created by stevenfrancus on 10/13/15. @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=PathUtils.replaceEnvVariables("[FILEMGR_URL]"); + public static final String SOLR_DRAT_URL=PathUtils.replaceEnvVariables("[SOLR_DRAT_URL]"); + public static final String CLIENT_URL=PathUtils.replaceEnvVariables("[WORKFLOW_URL]"); + public static final String OPSUI_URL=PathUtils.replaceEnvVariables("[OPSUI_URL]"); + + public static final String SOLR_INDEXER_CONFIG = "SOLR_INDEXER_CONFIG"; + private static String getDratDirectory() { final String DRAT_HOME = PathUtils.replaceEnvVariables("[DRAT_HOME]"); return DRAT_HOME.substring(0, DRAT_HOME.lastIndexOf(DRAT) + DRAT.length()); @@ -41,4 +51,6 @@ private static String getDratDirectory() { public static String buildDratSubdirectoryPath(String additionalPath) { return DRAT_SUPER_DIR + additionalPath; } + + } diff --git a/proteus/src/main/java/backend/ProcessDratWrapper.java b/proteus/src/main/java/backend/ProcessDratWrapper.java index e2c4188..10f1e20 100644 --- a/proteus/src/main/java/backend/ProcessDratWrapper.java +++ b/proteus/src/main/java/backend/ProcessDratWrapper.java @@ -22,11 +22,17 @@ import org.apache.commons.exec.PumpStreamHandler; import org.apache.commons.io.FileUtils; import org.apache.commons.lang.ArrayUtils; +import org.apache.commons.lang.time.DurationFormatUtils; +import org.apache.solr.client.solrj.SolrServerException; import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer; +import org.python.modules.synchronize; +import org.apache.oodt.cas.crawl.CrawlerLauncher; import org.apache.oodt.cas.filemgr.structs.Product; import org.apache.oodt.cas.filemgr.structs.ProductPage; import org.apache.oodt.cas.filemgr.structs.ProductType; +import org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient; import org.apache.oodt.cas.filemgr.tools.DeleteProduct; +import org.apache.oodt.cas.filemgr.tools.SolrIndexer; import org.apache.oodt.cas.metadata.util.PathUtils; import org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient; import org.apache.oodt.pcs.util.FileManagerUtils; @@ -36,9 +42,14 @@ import java.io.ByteArrayOutputStream; import java.io.File; import java.io.FileNotFoundException; +import java.io.FileOutputStream; import java.io.IOException; import java.net.URL; import java.nio.charset.Charset; +import java.time.Instant; +import java.time.ZoneId; +import java.time.ZonedDateTime; +import java.time.format.DateTimeFormatter; import java.util.ArrayList; import java.util.Arrays; import java.util.List; @@ -104,8 +115,20 @@ public void crawl() throws Exception { } @Override - public void index() throws IOException, DratWrapperException { -simpleDratExec(INDEX_CMD, this.path); + public void index() throws IOException, DratWrapperException, InstantiationException, SolrServerException { + solrIndex(); + } + + private synchronized void solrIndex() throws InstantiationException, SolrServerException, IOException { + DratLog idl = new DratLog("INDEXING"); + idl.logInfo("Starting", null); + System.setProperty(FileConstants.SOLR_INDEXER_CONFIG,FileConstants.SOLR_INDEXER_CONFIG_PATH); + SolrIndexer sIndexer = new SolrIndexer(FileConstants.SOLR_DRAT_URL,FileConstants.FILEMGR_URL); + sIndexer.indexAll(false); + sIndexer.commit(); + sIndexer.optimize(); + idl.logInfo("Completed",null); + } @Override @@ -173,6 +196,7 @@ public void reset() throws Exception { public void go() throws Exception { // before go, always reset + this.reset(); this.crawl(); this.index(); @@ -422,5 +446,62 @@ private synchronized void
[GitHub] chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
chrismattmann commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189496263 ## File path: proteus/src/main/java/backend/FileConstants.java ## @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=PathUtils.replaceEnvVariables("[FILEMGR_URL]"); + public static final String SOLR_DRAT_URL=PathUtils.replaceEnvVariables("[SOLR_DRAT_URL]"); + public static final String CLIENT_URL=PathUtils.replaceEnvVariables("[WORKFLOW_URL]"); + public static final String OPSUI_URL=PathUtils.replaceEnvVariables("[OPSUI_URL]"); Review comment: @ahmedifhaam This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189400979 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -104,8 +109,18 @@ public void crawl() throws Exception { } @Override - public void index() throws IOException, DratWrapperException { -simpleDratExec(INDEX_CMD, this.path); + public void index() throws IOException, DratWrapperException, InstantiationException, SolrServerException { +//simpleDratExec(INDEX_CMD, this.path); + solrIndex(); + } + + private void solrIndex() throws InstantiationException, SolrServerException, IOException { + System.setProperty(FileConstants.SOLR_INDEXER_CONFIG,FileConstants.SOLR_INDEXER_CONFIG_PATH); Review comment: Noted This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189401238 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -175,7 +190,12 @@ public void go() throws Exception { // before go, always reset this.reset(); this.crawl(); + +LOG.info("Indexing starts"); Review comment: Added this functionality using a small class for clarity, for duration using this format "MM-dd T HH-mm-ss" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer
ahmedifhaam commented on a change in pull request #129: Replaced Processwrapper indexing method with SolrIndexer URL: https://github.com/apache/drat/pull/129#discussion_r189504251 ## File path: proteus/src/main/java/backend/FileConstants.java ## @@ -32,7 +34,15 @@ public static final String DRAT_PATH = buildDratSubdirectoryPath("/deploy/bin/drat"); public static final String DRAT_TEMP_UNZIPPED_PATH = buildDratSubdirectoryPath("/deploy/data/staging"); public static final String DRAT_TEMP_LOG_OUTPUT = buildDratSubdirectoryPath("/deploy/data/drat_output.log"); + public static final String SOLR_INDEXER_CONFIG_PATH = buildDratSubdirectoryPath("/deploy/filemgr/etc/indexer.properties"); + public static final String FILEMGR_URL=PathUtils.replaceEnvVariables("[FILEMGR_URL]"); + public static final String SOLR_DRAT_URL=PathUtils.replaceEnvVariables("[SOLR_DRAT_URL]"); + public static final String CLIENT_URL=PathUtils.replaceEnvVariables("[WORKFLOW_URL]"); + public static final String OPSUI_URL=PathUtils.replaceEnvVariables("[OPSUI_URL]"); Review comment: CLIENT_URL and OPSUI URL are not used but for other functions it's used as its declared here https://github.com/apache/drat/blob/master/distribution/src/main/resources/bin/drat i thought its better to add all at once This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r194242488 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -128,17 +157,43 @@ private synchronized void solrIndex() throws InstantiationException, SolrServerE sIndexer.commit(); sIndexer.optimize(); idl.logInfo("Completed",null); - } @Override - public void map() throws IOException, DratWrapperException { -simpleDratExec(MAP_CMD); + public void map() { +setStatus(MAP_CMD); +DratLog mapLog = new DratLog("MAPPING"); +WorkflowRestResource restResource = new WorkflowRestResource(); +DynamicWorkflowRequestWrapper requestBody = new DynamicWorkflowRequestWrapper(); +requestBody.taskIds = new ArrayList<>(); +requestBody.taskIds.add("urn:drat:MimePartitioner"); +LOG.info("STARTING MAPPING"); +mapLog.logInfo("STARTING", " (dynamic workflow with task urn:drat:MimePartitioner"); +String resp = restResource.performDynamicWorkFlow(requestBody); +if(resp.equals("OK")) { +mapLog.logInfo("STARTED SUCCESSFULLY, urn:drat:MimePartitioner dynamic workflow"); +}else { +mapLog.logSevere("FAILED", "Dynamic workflow starting failed "+resp); +} } @Override - public void reduce() throws IOException, DratWrapperException { -simpleDratExec(REDUCE_CMD); + public void reduce() throws IOException { +setStatus(REDUCE_CMD); +DratLog mapLog = new DratLog("REDUCING"); +WorkflowRestResource restResource = new WorkflowRestResource(); +DynamicWorkflowRequestWrapper requestBody = new DynamicWorkflowRequestWrapper(); +requestBody.taskIds = new ArrayList<>(); +requestBody.taskIds.add("urn:drat:RatAggregator"); Review comment: this should be a static final String key like REDUCE_TASK at the top. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r194242484 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -128,17 +157,43 @@ private synchronized void solrIndex() throws InstantiationException, SolrServerE sIndexer.commit(); sIndexer.optimize(); idl.logInfo("Completed",null); - } @Override - public void map() throws IOException, DratWrapperException { -simpleDratExec(MAP_CMD); + public void map() { +setStatus(MAP_CMD); +DratLog mapLog = new DratLog("MAPPING"); +WorkflowRestResource restResource = new WorkflowRestResource(); +DynamicWorkflowRequestWrapper requestBody = new DynamicWorkflowRequestWrapper(); +requestBody.taskIds = new ArrayList<>(); +requestBody.taskIds.add("urn:drat:MimePartitioner"); Review comment: create a static final key for this at the top (like MAPPER_TASK from before). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
ahmedifhaam commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#issuecomment-396139086 @chrismattmann It's done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#issuecomment-396289231 LGTM! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193808064 ## File path: proteus/src/main/java/drat/proteus/workflow/rest/WorkflowRestResource.java ## @@ -0,0 +1,55 @@ +package drat.proteus.workflow.rest; + + +import java.io.IOException; +import java.util.logging.Logger; + + +import org.apache.oodt.cas.metadata.Metadata; +import org.apache.oodt.cas.metadata.util.PathUtils; +import org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient; +import org.apache.oodt.pcs.util.WorkflowManagerUtils; +import org.wicketstuff.rest.annotations.MethodMapping; +import org.wicketstuff.rest.annotations.parameters.RequestBody; +import org.wicketstuff.rest.contenthandling.json.webserialdeserial.GsonWebSerialDeserial; +import org.wicketstuff.rest.resource.AbstractRestResource; +import org.wicketstuff.rest.utils.http.HttpMethod; + +import backend.FileConstants; + Review comment: add simple class docs This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#issuecomment-395477671 @ahmedifhaam please document the above on the wiki with respect to the new API you added. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807320 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -257,16 +330,24 @@ private synchronized boolean hasAggregateRatLog() { } private boolean mapsStillRunning() throws Exception { -String args[] = { FileConstants.WORKFLOW_PATH, "--url", -"http://localhost:9001;, "--operation", "--getWorkflowInsts" }; -String cmd = Joiner.on(" ").join(args); -LOG.info("Maps Still Running: Executing: " + cmd); -String output = execToString(cmd); -LOG.info("Output from maps still running: " + output); -List items = parseWorkflows(output); -return stillRunning(items); +//String args[] = { FileConstants.WORKFLOW_PATH, "--url", Review comment: remove all of this This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193806972 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -111,15 +115,51 @@ public synchronized void setStatus(String status) { @Override public void crawl() throws Exception { -simpleDratExec(CRAWL_CMD, this.path); +//simpleDratExec(CRAWL_CMD, this.path); + crawlTemp(); + } + + + public void crawlTemp() throws Exception{ +DratLog crawlLog = new DratLog("CRAWLING"); +try{ + setStatus(CRAWL_CMD); + + crawlLog.logInfo("Configuring"); + String beanRepo = System.getProperty("org.apache.oodt.cas.crawl.bean.repo", + FileConstants.CRAWLER_CONFIG); + String crawlerId = "MetExtractorProductCrawler"; + System.setProperty("DRAT_EXCLUDE",""); + FileSystemXmlApplicationContext appContext = new FileSystemXmlApplicationContext("file:"+beanRepo); + +MetExtractorProductCrawler crawler = new MetExtractorProductCrawler(); +crawler.setApplicationContext(appContext); +crawler.setId(crawlerId); + crawler.setMetExtractor("org.apache.oodt.cas.metadata.extractors.CopyAndRewriteExtractor"); +crawler.setMetExtractorConfig(FileConstants.MET_EXT_CONFIG_PATH); +crawler.setFilemgrUrl(FileConstants.FILEMGR_URL); + crawler.setClientTransferer("org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory"); +crawler.setPreCondIds(Arrays.asList("RegExExcludeComparator")); +crawler.setProductPath(this.path); +crawlLog.logInfo("STARTING ",null); +crawler.crawl(); +crawlLog.logInfo("COMPLETED",null); + }catch (Exception ex) { +// LOG.severe(ex.getLocalizedMessage()); Review comment: if it's commented out, remove it This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807770 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -446,39 +556,39 @@ private synchronized void wipeSolrCore(String coreName) { + e.getLocalizedMessage()); } } - + private class DratLog{ - private static final String MODULE = "DRAT_LOG"; + private static final String MODULE = "DRAT_LOG"; long startTime =0; private long lastActionTime=-1L; private long timeDiff =-1L; private ZonedDateTime zdt; private String action; public DratLog(String action) { this.action = action; - + } - + private void logWarning(String status,String desc) { LOG.warning(getMsg(status,desc)); } - + private void logWarning(String desc) { LOG.warning(MODULE+" : "+desc); } - + private void logInfo(String status,String desc) { LOG.info(getMsg(status,desc)); } - + private void logInfo(String desc) { LOG.info(MODULE+" : "+desc); } - + private void logSevere(String status,String desc) { LOG.fine(getMsg(status,desc)); } - Review comment: a bunch of extraneous white space and newline changes. Don't couple these to your actual PR. They are extraneous. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807228 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -128,17 +168,47 @@ private synchronized void solrIndex() throws InstantiationException, SolrServerE sIndexer.commit(); sIndexer.optimize(); idl.logInfo("Completed",null); - + } @Override public void map() throws IOException, DratWrapperException { -simpleDratExec(MAP_CMD); +//simpleDratExec(MAP_CMD); +setStatus(MAP_CMD); +DratLog mapLog = new DratLog("MAPPING"); +WorkflowRestResource restResource = new WorkflowRestResource(); +DynamicWorkflowRequestWrapper requestBody = new DynamicWorkflowRequestWrapper(); +requestBody.taskIds = new ArrayList<>(); +requestBody.taskIds.add("urn:drat:MimePartitioner"); +LOG.info("STARTING MAPPING"); +mapLog.logInfo("STARTING", " (dynamic workflow with task urn:drat:MimePartitioner"); +String resp = (String)restResource.performDynamicWorkFlow(requestBody); +if(resp.equals("OK")) { +mapLog.logInfo("STARTED SUCCESSFULLY, urn:drat:MimePartitioner dynamic workflow"); +}else { +mapLog.logSevere("FAILED", "Dynamic workflow starting failed "+resp); +} } + @Override public void reduce() throws IOException, DratWrapperException { -simpleDratExec(REDUCE_CMD); +//simpleDratExec(REDUCE_CMD); Review comment: remove it This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807586 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -310,36 +391,65 @@ private boolean mapsStillRunning() throws Exception { return items; } - @VisibleForTesting - protected boolean stillRunning(List items) { -List mapperItems = filterMappers(items); -LOG.info("Checking mappers: inspecting [" -+ String.valueOf(mapperItems.size()) + "] mappers."); -for (WorkflowItem mapperItem : mapperItems) { - if (isRunning(mapperItem.getStatus())) { -LOG.info("Mapper: [" + mapperItem.getId() + "] still running."); -return true; + protected boolean stillRunning(List instances){ + List mapperInstances = filterMappers(instances); + LOG.info("Checking mappers: inspecting [" + + String.valueOf(mapperInstances.size()) + "] mappers."); + for(WorkflowInstance mapperInstance:mapperInstances){ + if(isRunning(mapperInstance.getState().getName())){ + LOG.info("Mapper: [" + mapperInstance.getId() + "] still running."); + return true; + } } -} + return false; -return false; } - @VisibleForTesting - protected List filterMappers(List items) { -List mappers = new ArrayList(); -if (items != null && items.size() > 0) { - for (WorkflowItem item : items) { -if (item.getCurrentTask().equals(MAPPER_TASK)) { - LOG.info("Adding mapper: [" + item.getCurrentTask() + "]"); - mappers.add(item); -} else { - LOG.info("Filtering task: [" + item.getCurrentTask() + "]"); -} +// @VisibleForTesting Review comment: remove all of this if it's commented out This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807960 ## File path: proteus/src/main/java/drat/proteus/workflow/rest/DynamicWorkflowRequestWrapper.java ## @@ -0,0 +1,7 @@ +package drat.proteus.workflow.rest; + +import java.util.List; + Review comment: add some simple class docs. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807542 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -310,36 +391,65 @@ private boolean mapsStillRunning() throws Exception { return items; } - @VisibleForTesting - protected boolean stillRunning(List items) { -List mapperItems = filterMappers(items); -LOG.info("Checking mappers: inspecting [" -+ String.valueOf(mapperItems.size()) + "] mappers."); -for (WorkflowItem mapperItem : mapperItems) { - if (isRunning(mapperItem.getStatus())) { -LOG.info("Mapper: [" + mapperItem.getId() + "] still running."); -return true; + protected boolean stillRunning(List instances){ Review comment: I think here you are reformatting white space. Don't do this. It makes it hard to see if this is a real change or not. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807922 ## File path: proteus/src/main/java/drat/proteus/workflow/rest/DynamicWorkflowRequestWrapper.java ## @@ -0,0 +1,7 @@ +package drat.proteus.workflow.rest; Review comment: need to add Apache license header from other files here. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193806844 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -111,15 +115,51 @@ public synchronized void setStatus(String status) { @Override public void crawl() throws Exception { -simpleDratExec(CRAWL_CMD, this.path); +//simpleDratExec(CRAWL_CMD, this.path); + crawlTemp(); + } + + + public void crawlTemp() throws Exception{ Review comment: this method name doesn't make sense to me. I would name it performCrawl(). The crawl() DRAT API method on L117 is fine - and then this actually _performs_ the crawl. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193807091 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -128,17 +168,47 @@ private synchronized void solrIndex() throws InstantiationException, SolrServerE sIndexer.commit(); sIndexer.optimize(); idl.logInfo("Completed",null); - + } @Override public void map() throws IOException, DratWrapperException { -simpleDratExec(MAP_CMD); +//simpleDratExec(MAP_CMD); Review comment: remove it This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
chrismattmann commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193808013 ## File path: proteus/src/main/java/drat/proteus/workflow/rest/WorkflowRestResource.java ## @@ -0,0 +1,55 @@ +package drat.proteus.workflow.rest; Review comment: Apache license header from other files here. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
ahmedifhaam commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193855319 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -72,7 +75,8 @@ private static final String RESET_CMD = "reset"; private static final String STATUS_IDLE = "idle"; - private static final String MAPPER_TASK = "urn:drat:RatCodeAudit"; +// private static final String MAPPER_TASK = "urn:drat:RatCodeAudit"; Review comment: This part is even confusing for me too, because my current branch on github looks like this ... ![image](https://user-images.githubusercontent.com/26105024/41120268-a2e76324-6ab2-11e8-9355-520524509035.png) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
ahmedifhaam commented on a change in pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#discussion_r193855319 ## File path: proteus/src/main/java/backend/ProcessDratWrapper.java ## @@ -72,7 +75,8 @@ private static final String RESET_CMD = "reset"; private static final String STATUS_IDLE = "idle"; - private static final String MAPPER_TASK = "urn:drat:RatCodeAudit"; +// private static final String MAPPER_TASK = "urn:drat:RatCodeAudit"; Review comment: This part is even confusing for me too, because my current branch on github looks like this ... ![image](https://user-images.githubusercontent.com/26105024/41120268-a2e76324-6ab2-11e8-9355-520524509035.png) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger opened a new pull request #132: Correct git clone URL and changes to https instead of http in Readme
ottlinger opened a new pull request #132: Correct git clone URL and changes to https instead of http in Readme URL: https://github.com/apache/drat/pull/132 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger opened a new issue #134: Upgrade to JDK8
ottlinger opened a new issue #134: Upgrade to JDK8 URL: https://github.com/apache/drat/issues/134 Is it possible to update to Java8 in all submodules? I stumbled upon 1.6-compiler level in the pom.xml files. Is there a reason for that? Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger opened a new pull request #133: Streamline usage of junit among the project
ottlinger opened a new pull request #133: Streamline usage of junit among the project URL: https://github.com/apache/drat/pull/133 The project uses various junit versions - the next step would be to define them in the parent-pom. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ottlinger commented on issue #133: Streamline usage of junit among the project and add Travis CI build support
ottlinger commented on issue #133: Streamline usage of junit among the project and add Travis CI build support URL: https://github.com/apache/drat/pull/133#issuecomment-397740877 This pull request allows to build on Travis, once this file is merged a ticket needs to be opened in ASF's INFRA-jira to enable "official" builds with Travis from ASF-drat repository. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #134: Upgrade to JDK8
chrismattmann commented on issue #134: Upgrade to JDK8 URL: https://github.com/apache/drat/issues/134#issuecomment-397725862 no reason it shouldn't be JDK8 please upgrade. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed pull request #133: Streamline usage of junit among the project and add Travis CI build support
chrismattmann closed pull request #133: Streamline usage of junit among the project and add Travis CI build support URL: https://github.com/apache/drat/pull/133 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 000..25a82e7 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,13 @@ +language: java +jdk: + - oraclejdk8 + - oraclejdk9 + +#before_script: +# - pip install --user codecov + +#after_success: +# - codecov + +#addons: +# srcclr: true diff --git a/crawler/pom.xml b/crawler/pom.xml index b11d3f2..622b889 100644 --- a/crawler/pom.xml +++ b/crawler/pom.xml @@ -59,7 +59,7 @@ junit junit - 3.8.2 + ${junit.version} test diff --git a/extensions/pom.xml b/extensions/pom.xml index a4b08f5..4b45ffa 100644 --- a/extensions/pom.xml +++ b/extensions/pom.xml @@ -77,7 +77,7 @@ junit junit - 3.8.2 + ${junit.version} diff --git a/filemgr/pom.xml b/filemgr/pom.xml index b4a9285..6ac033d 100644 --- a/filemgr/pom.xml +++ b/filemgr/pom.xml @@ -60,7 +60,7 @@ junit junit - 3.8.2 + ${junit.version} test diff --git a/pcs/pom.xml b/pcs/pom.xml index 5be580e..548c271 100644 --- a/pcs/pom.xml +++ b/pcs/pom.xml @@ -60,7 +60,7 @@ junit junit - 3.8.2 + ${junit.version} test diff --git a/pom.xml b/pom.xml index 013787b..781a844 100644 --- a/pom.xml +++ b/pom.xml @@ -13,6 +13,7 @@ UTF-8 UTF-8 1.2.2 +4.12 @@ -48,6 +49,7 @@ +clean install ${basedir}/src/main/java ${basedir}/src/test @@ -56,8 +58,8 @@ maven-compiler-plugin 3.1 - 1.6 - 1.6 + 1.8 + 1.8 diff --git a/proteus/pom.xml b/proteus/pom.xml index 18fe642..5f2d5e7 100644 --- a/proteus/pom.xml +++ b/proteus/pom.xml @@ -39,7 +39,6 @@ 7.8.0 - 4.12 2.3 UTF-8 2.2.3 @@ -181,8 +180,8 @@ maven-compiler-plugin 3.1 - 1.7 - 1.7 + 1.8 + 1.8 UTF-8 true true diff --git a/resmgr/pom.xml b/resmgr/pom.xml index 1c246cb..a561867 100644 --- a/resmgr/pom.xml +++ b/resmgr/pom.xml @@ -60,7 +60,7 @@ junit junit - 3.8.2 + ${junit.version} test diff --git a/workflow/pom.xml b/workflow/pom.xml index 99934c4..9c48fb5 100644 --- a/workflow/pom.xml +++ b/workflow/pom.xml @@ -83,7 +83,7 @@ junit junit - 3.8.2 + ${junit.version} test This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #133: Streamline usage of junit among the project and add Travis CI build support
chrismattmann commented on issue #133: Streamline usage of junit among the project and add Travis CI build support URL: https://github.com/apache/drat/pull/133#issuecomment-398454278 thanks @ottlinger please file an INFRA ticket and report back the ISSUE id to the list. Thanks. I just merged this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #132: Correct git clone URL and changes to https instead of http in Readme
chrismattmann commented on issue #132: Correct git clone URL and changes to https instead of http in Readme URL: https://github.com/apache/drat/pull/132#issuecomment-397513032 great work thanks @ottlinger This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed pull request #132: Correct git clone URL and changes to https instead of http in Readme
chrismattmann closed pull request #132: Correct git clone URL and changes to https instead of http in Readme URL: https://github.com/apache/drat/pull/132 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/README.md b/README.md index 5307072..8482acd 100644 --- a/README.md +++ b/README.md @@ -21,5 +21,5 @@ See the wiki for more information on installing and running DRAT: You can clone the wiki by running `git clone https://github.com/apache/drat.wiki.git` -Visit our new website [drat.apache.org](http://drat.apache.org/) at the [Apache Software Foundation](http://www.apache.org/). +Visit our new website [drat.apache.org](https://drat.apache.org/) at the [Apache Software Foundation](https://www.apache.org/). diff --git a/provision.sh b/provision.sh index 21e117c..10698cf 100755 --- a/provision.sh +++ b/provision.sh @@ -27,7 +27,7 @@ echo "[vagrant provisioning] Installing drat..." mkdir -p /usr/local/drat/deploy mkdir -p /usr/local/drat/src cd /usr/local/drat/src -git clone -q https://github.com/chrismattmann/drat.git . +git clone -q https://github.com/apache/drat.git . mvn install cp -R distribution/target/dms-distribution-0.1-bin.tar.gz ../deploy/ cd ../deploy/ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #128: "bin/oodt stop" won't stop tomcat causing system to go out of memory
chrismattmann commented on issue #128: "bin/oodt stop" won't stop tomcat causing system to go out of memory URL: https://github.com/apache/drat/issues/128#issuecomment-392280642 got it. Well let's try and make this robust thanks @IMS94 i'm available to test This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
ahmedifhaam commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#issuecomment-392968578 Also adde rest api on proteus to request a dynamicworkflow execution with tasks A post request on http://localhost:8080/proteus/workflowservice/dynamic with {taskIds:["urn:drat:MimePartitioner"]} will return ok and anything goes wrong with an exception message please do check that once as well This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
ahmedifhaam commented on issue #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131#issuecomment-392968578 Also added rest api on proteus to request a dynamicworkflow execution with tasks A post request on http://localhost:8080/proteus/workflowservice/dynamic with {taskIds:["urn:drat:MimePartitioner"]} will return ok and anything goes wrong with an exception message please do check that once as well This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam opened a new pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger
ahmedifhaam opened a new pull request #131: Changed the map and reduce function to work with XmlRpcWorkflowManger URL: https://github.com/apache/drat/pull/131 Please do check once,I will remove unused methods in the next commit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-394578021 @chrismattmann No, I have the `workflowInstanceId` being null problem in `StdPgeTaskInstance` for task `MimeTypePartitioner`. Once I fix that in OODT side, we can merge this. Otherwise, @ahmedifhaam will encounter carrying on the GSoC project and DRAT master will be unstable. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
chrismattmann commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-394562082 @IMS94 closer to being done? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-394578021 @chrismattmann No, I have the `workflowInstanceId` being null problem in `StdPgeTaskInstance` for task `MimeTypePartitioner`. Once I fix that in OODT side, we can merge this. Otherwise, @ahmedifhaam will encounter problems when carrying on the GSoC project and DRAT master will be unstable. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #134: Upgrade to JDK8
chrismattmann commented on issue #134: Upgrade to JDK8 URL: https://github.com/apache/drat/issues/134#issuecomment-401645832 +1 thanks @buggtb This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] buggtb commented on issue #134: Upgrade to JDK8
buggtb commented on issue #134: Upgrade to JDK8 URL: https://github.com/apache/drat/issues/134#issuecomment-400663725 Be worth testing JDK 10 also, because in some other projects JDK 10 broke Tomcat for a start. I'll give it a whirl this week at some point. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #116: Set up DRAT to weekly scan the ASF full Github
chrismattmann commented on issue #116: Set up DRAT to weekly scan the ASF full Github URL: https://github.com/apache/drat/issues/116#issuecomment-400663558 yep we have one, drat.apache.org. Current projects running there: http://drat.apache.org/demo/ - Proteus live scan, pre-release 1.0 http://drat.apache.org/asfgit/ - Proteus D3 viz, pre-release 1.0 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge)
IMS94 commented on issue #130: Updates for OODT 1.9 and fix for #112 (Do not merge) URL: https://github.com/apache/drat/pull/130#issuecomment-399723389 @chrismattmann I'm in the process of doing that. Will need few more days. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ahmedifhaam opened a new pull request #135: Initial Commit of adding new UI designed with vue
ahmedifhaam opened a new pull request #135: Initial Commit of adding new UI designed with vue URL: https://github.com/apache/drat/pull/135 -Only works for local paths -Only intergrated proteus UI Viz features have to intergrated This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #135: Initial Commit of adding new UI designed with vue
chrismattmann commented on issue #135: Initial Commit of adding new UI designed with vue URL: https://github.com/apache/drat/pull/135#issuecomment-401850573 this includes generated JS and CSS minified. We need the original source, not the minified stuff. I can't review this. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lewismc opened a new pull request #127: ISSUE-126 Upgrade Tika to 1.18
lewismc opened a new pull request #127: ISSUE-126 Upgrade Tika to 1.18 URL: https://github.com/apache/drat/pull/127 This issue addresses #126 but it is blocked by a release and upgrade of OODT here as well. Currently OODT 1.2.2 uses Tika 1.7 and 1.10 dependencies. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] lewismc commented on issue #127: ISSUE-126 Upgrade Tika to 1.18
lewismc commented on issue #127: ISSUE-126 Upgrade Tika to 1.18 URL: https://github.com/apache/drat/pull/127#issuecomment-385859860 Tika upgrade in OODT available at https://github.com/apache/oodt/pull/61 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #123: Upgrade OODT to 1.2.2 and Tomcat to 9.0.7
chrismattmann commented on issue #123: Upgrade OODT to 1.2.2 and Tomcat to 9.0.7 URL: https://github.com/apache/drat/issues/123#issuecomment-385436119 solved in #124 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #124: ISSUE-123
chrismattmann commented on issue #124: ISSUE-123 URL: https://github.com/apache/drat/pull/124#issuecomment-385436073 LGTM thanks @lewismc ! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann closed issue #120: Fix trademark language on website
chrismattmann closed issue #120: Fix trademark language on website URL: https://github.com/apache/drat/issues/120 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #120: Fix trademark language on website
chrismattmann commented on issue #120: Fix trademark language on website URL: https://github.com/apache/drat/issues/120#issuecomment-385436480 addressed in #121 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrismattmann commented on issue #125: Clean site of old artifacts
chrismattmann commented on issue #125: Clean site of old artifacts URL: https://github.com/apache/drat/pull/125#issuecomment-385436381 thanks @ottlinger ! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services