Maybe even something like this should be done but for the contrib modules. https://github.com/apache/incubator-streams/blob/master/streams-contrib/streams-provider-rss/src/test/java/org/apache/streams/rss/test/Top100FeedsTest.java
That would be the something like examples. Wdyt? Renato M. 2014-05-22 12:05 GMT+02:00 Renato Marroquín Mogrovejo < [email protected]>: > Hi Steve, > > Comments inline. > > 2014-05-21 20:59 GMT+02:00 Steve Blackmon <[email protected]>: > > I will happily accept pull requests to streams-examples, although I >> don't think it would need to change for an update to the provider to >> be applied (except perhaps to add that verb filtering step) >> > > This is what I meant. > > >> I am willing to donate all of the examples, but I don't think they >> belong in the main repo. What if there were 50 examples many >> containing integration tests? Build would take forever - plus I don't >> think that a single example failing a test should break the build. >> Mahout suffered from really long-builds and handled it by separating >> the examples out from core. >> > > Well, over in Apache Giraph they have over 20 graph example algorithms and > the way they test them is just by doing some simple computations, I don't > know how long it takes, but I think it makes sense to at least try it and > see how long it takes. > Over in Gora we have a test suite with the different operations supported > that is run over the different data stores, and depending on your machine, > it might take from 10 - 20 mins. There was a time where the HBase tests > were taking around 30 mins by themselves but we fixed them and are now run > in less than 5 mins. > Maybe your examples could be converted into a test suite? I mean reading > from twitter(or a file) and persisting into different places. But anyhow, > your examples should be referenced somewhere as it is what it makes sense > for a newcomer :D > > When the project was hosting in SVN, we had the ability to create and >> checkout different roots (the master pom was in a different root for >> example). If we can do the same using Git I propose we put the >> examples in as a flat multi-module project that way. >> > > +1 > > > Renato M. > > Any thoughts? >> >> Steve Blackmon >> [email protected] >> >> On Wed, May 21, 2014 at 11:59 AM, Renato Marroquín Mogrovejo >> <[email protected]> wrote: >> > I tried updating the site through ApacheCMS but I guess I don't have the >> > rights to do it ;) >> > This is what I got: >> > >> > The operation is forbidden by the server: Commit failed (details >> follow):: >> > Changing file >> > >> '/usr/local/cms/wc/streams/rmarroquin-NlOO6g/trunk/content/source-repository.mdtext' >> > is forbidden by the server: Access to >> > >> '/repos/asf/!svn/txr/1596615-yz8u/incubator/streams/site/trunk/content/source-repository.mdtext' >> > forbidden at /usr/local/cms/webgui/lib/ASF/CMS/WC/Commit.pm line 64 >> > >> > So what is the best/easy way to update the site? >> > >> > @Steve >> > Yeah, I finally got the time to try this out :) and looks very promising >> > for sure! >> > Is anybody working on STREAMS-57? Could I pick it up? I think this >> should >> > be pretty straight forward. >> > And are there any plans to put your examples into the code base? It's >> quite >> > puzzling going from repo to another (plus the one from the site) to >> > accomplish this. And then if I could work on STREAMS-57, would I have >> to do >> > the pull request into W2OGroup/streams-examples repo? I don't think so >> > right? how would this work? >> > Thanks! >> > >> > >> > Renato M. >> > >> > >> > 2014-05-21 17:42 GMT+02:00 Steve Blackmon <[email protected]>: >> > >> >> Cool I'm glad to hear you got it running. That exception occurs because >> >> twitter sends a preamble listing all of your friends when you open a >> >> stream. The provider doesn't have logic to handle it yet, so it tries >> to >> >> build an activity and throws that exception. >> >> >> >> STREAMS-57 would suppress this warning - although once non-content >> >> activities with new verbs like follow are generated we will want a >> >> filtering mechanism in place so it remains easy to build indexes of >> just >> >> shares and posts. >> >> >> >> On Wed, May 21, 2014 at 9:23 AM, Renato Marroquín Mogrovejo < >> >> [email protected]> wrote: >> >> > Hey Matt, >> >> > >> >> > So the examples worked :D at least the one retrieving the twitter >> user's >> >> > history. The one trying to track users activity keeps on getting this >> >> > exception: >> >> > >> >> > 16:09:11.151 [pool-2-thread-2] DEBUG o.a.s.t.p.TwitterTypeConverter - >> >> > ACTIVITY >> >> > org.apache.streams.exceptions.ActivitySerializerException: >> unrecognized >> >> type >> >> > at >> >> > >> >> >> >> >> org.apache.streams.twitter.serializer.TwitterJsonActivitySerializer.deserialize(TwitterJsonActivitySerializer.java:74) >> >> > at >> >> > >> >> >> >> >> org.apache.streams.twitter.processor.TwitterTypeConverter.convert(TwitterTypeConverter.java:65) >> >> > at >> >> > >> >> >> >> >> org.apache.streams.twitter.processor.TwitterTypeConverter.process(TwitterTypeConverter.java:146) >> >> > at >> >> > >> >> >> >> >> org.apache.streams.local.tasks.StreamsProcessorTask.run(StreamsProcessorTask.java:72) >> >> > at >> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> >> > at >> >> > >> >> >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >> > at >> >> > >> >> >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >> > at java.lang.Thread.run(Thread.java:744) >> >> > >> >> > But everything seems to work in spite of that exception :) Cool >> stuff! So >> >> > can I open a JIRA issue for updating the site and putting the patch >> there >> >> > then? >> >> > I will write another email with some other issues. >> >> > Thanks Matt! >> >> > >> >> > >> >> > Renato M. >> >> > >> >> > >> >> > 2014-05-21 15:59 GMT+02:00 Matt Franklin <[email protected]>: >> >> > >> >> >> On Wed, May 21, 2014 at 9:00 AM, Renato Marroquín Mogrovejo < >> >> >> [email protected]> wrote: >> >> >> >> >> >> > Hi Matt, >> >> >> > >> >> >> > Thank for pointing me to these examples. I will try them out and >> >> report >> >> >> > back. >> >> >> > But I have more questions now. I have found: >> >> >> > >> >> >> > https://github.com/apache/incubator-streams --> referenced by the >> >> >> examples >> >> >> > you sent me and more active >> >> >> > https://github.com/apache/streams --> referenced by the website >> >> >> > >> >> >> >> >> >> Oops. Yes it is the first one. The second one was a mirror of SVN. >> >> >> >> >> >> >> >> >> > >> >> >> > Which one is the official one? I guess the first one right. >> Shouldn't >> >> we >> >> >> > change this on the website? does that mean that Apache Streams >> have >> >> two >> >> >> git >> >> >> > repos? I can fix that on the website if you point me to the right >> >> place. >> >> >> > >> >> >> >> >> >> Website is in SVN using Apache CMS. >> >> >> >> >> >> >> >> >> > >> >> >> > >> >> >> > Renato M. >> >> >> > >> >> >> > >> >> >> > >> >> >> > 2014-05-21 2:14 GMT+02:00 Matt Franklin <[email protected] >> >: >> >> >> > >> >> >> > > On Tue, May 20, 2014 at 4:28 PM, Renato Marroquín Mogrovejo < >> >> >> > > [email protected]> wrote: >> >> >> > > >> >> >> > > > Sorry I pressed the sent button before I finished writing the >> >> email. >> >> >> > > > Anyways . . . Tomcat 7.0.39, ServiceMix 5.0.1 and Cassandra >> 2.0.4. >> >> >> > > > But I am getting this exception: >> >> >> > > > >> >> >> > > >> >> >> > > We are unfortunately in the middle of converging the new model >> and >> >> the >> >> >> > old >> >> >> > > model. This runtime should be working, but it needs some work to >> >> >> ensure >> >> >> > > that the new components in streams-contrib work. If you are >> looking >> >> to >> >> >> > get >> >> >> > > started with a newer runtime, streams-runtime-local is a good >> place. >> >> >> > > >> >> >> > > Steve posted a link to some examples he wrote for how to get >> >> started: >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> https://github.com/w2ogroup/streams-examples/tree/master/twitter-history-elasticsearch >> >> >> > > >> >> >> > > We need to create some of our own examples (or ask him kindly to >> >> donate >> >> >> > > some to us), but until the site is fixed, this is a good start. >> I >> >> will >> >> >> > > create an issue for the Caml runtime issue you found here. >> >> >> > > >> >> >> > > >> >> >> > > > >> >> >> > > > May 20, 2014 10:15:29 PM >> org.apache.catalina.startup.ContextConfig >> >> >> > > > getServletContainerInitializer >> >> >> > > > SEVERE: The ServletContentInitializer >> >> >> > > > [org.springframework.web.SpringServletContainerInitializer] >> could >> >> not >> >> >> > be >> >> >> > > > created >> >> >> > > > java.lang.ClassNotFoundException: >> >> >> > > > org.springframework.web.SpringServletContainerInitializer >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1713) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1558) >> >> >> > > > at java.lang.Class.forName0(Native Method) >> >> >> > > > at java.lang.Class.forName(Class.java:270) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.startup.ContextConfig.getServletContainerInitializer(ContextConfig.java:1659) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.startup.ContextConfig.processServletContainerInitializers(ContextConfig.java:1569) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1277) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:878) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:369) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90) >> >> >> > > > at >> >> >> > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5269) >> >> >> > > > at >> >> >> > > > >> >> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) >> >> >> > > > >> >> >> > > > That let me to [2]. Is this related to this? or is it just me >> >> doing >> >> >> > > things >> >> >> > > > wrong here? >> >> >> > > > Thanks for any pointers! >> >> >> > > > >> >> >> > > > >> >> >> > > > Renato M. >> >> >> > > > >> >> >> > > > 2014-05-20 22:25 GMT+02:00 Renato Marroquín Mogrovejo < >> >> >> > > > [email protected]>: >> >> >> > > > >> >> >> > > > > Hey guys, >> >> >> > > > > >> >> >> > > > > This is a long overdue email but with little steps I will >> start >> >> >> going >> >> >> > > :) >> >> >> > > > > So I have been trying to run the whole project based on[1]. >> As >> >> >> there >> >> >> > is >> >> >> > > > no >> >> >> > > > > version for any of the pieces involved in running this (if >> there >> >> is >> >> >> > > > please >> >> >> > > > > point me to them, if not, I am happy to put them on the site >> >> >> > > somewhere). >> >> >> > > > I >> >> >> > > > > chose >> >> >> > > > > >> >> >> > > > > >> >> >> > > > > >> >> >> > > > > [1] http://streams.incubator.apache.org/streams-usage.html >> >> >> > > > > [2] >> >> >> > > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> http://stackoverflow.com/questions/7692497/tomcat-wont-load-my-meta-inf-services-javax-servlet-servletcontainerinitializ >> >> >> > > > > >> >> >> > > > >> >> >> > > >> >> >> > >> >> >> >> >> >> > >
