I think a flat list with good naming would make it easier to find a matching example.
topnwords ? There are already two related examples in Malhar: wordcount and twitter Can some of the examples be consolidated into a single example package, fileioXXX/fileoutput look like candidates. On Sat, Mar 11, 2017 at 6:36 PM, Lakshmi Velineni <[email protected]> wrote: > Hi, > > I have come up with the following list of applications to move from > datatorrent/examples into apex-malhar/examples. There were many examples > that do a similar functionality of moving data around from different > sources to destinations, so I separated them all into a separate folder > called "Ingestion". > > csvformatter > > dedup > > dynamic-partition > > enricher > > exactly-once > > filter > > innerjoin > > maprapp > > parser > > partition > > recordreader > > throttle > > topnwords > > transform > > unifiers > > *Ingestion/* > > cassandrainput > > cassandraoutput > > fileio > > fileio-multidir > > fileio-simple > > fileoutput > > filetojdbc > > hdfs2kafka > > hdfs-sync > > jdbcingest > > jdbctijdbc > > jmsActiveMQ > > jmsSqs > > kafka > > s3-to-hdfs-sync > > s3output > > > Let me know your thoughts. > > > Thanks > > Lakshmi Prasanna > > > > > > > > On Thu, Feb 23, 2017 at 10:34 AM, Amol Kekre <[email protected]> wrote: > > > Yes, should merge samples into examples. Ideally the names of the > examples > > should be more descriptive in terms of "how to" as opposed to a title. PI > > demo for example has lots of ways to do compute. So it could be named as > > "pi - distributed compute" in examples. Similarly if others can be named > to > > bring out features to look at from the examples, it would make examples > > more useful to readers. > > > > Thks > > Amol > > > > > > > > E:[email protected] | M: 510-449-2606 | Twitter: @*amolhkekre* > > > > www.datatorrent.com | apex.apache.org > > > > *Join us at Apex Big Data World-San Jose > > <http://www.apexbigdata.com/san-jose.html>, April 4, 2017!* > > [image: http://www.apexbigdata.com/san-jose-register.html] > > <http://www.apexbigdata.com/san-jose-register.html> > > > > On Thu, Feb 23, 2017 at 10:07 AM, Sanjay Pujare <[email protected]> > > wrote: > > > > > + for renaming to examples. While we are at it, how about merging > > "samples" > > > also in the new "examples" ? > > > > > > On Thu, Feb 23, 2017 at 9:47 AM, Munagala Ramanath < > [email protected]> > > > wrote: > > > > > > > +1 for renaming to "examples" > > > > > > > > Ram > > > > > > > > On Thu, Feb 23, 2017 at 9:12 AM, Lakshmi Velineni < > > > [email protected] > > > > > > > > > wrote: > > > > > > > > > I am ready to bring the examples over into the demos folder. I was > > > > > wondering if anybody has any input on Thomas's suggestion to rename > > the > > > > > demos folder to examples. I would rather do that first and then > bring > > > the > > > > > examples over instead of doing it the other way around as that > would > > > lead > > > > > to refactoring the new examples again. > > > > > > > > > > Thanks > > > > > > > > > > On Wed, Jan 25, 2017 at 8:12 AM, Lakshmi Velineni < > > > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Since the examples have little history I was planning to have two > > > > > > commits for every example, one for the code as the primary author > > of > > > > > > the example and another containing pom.xml and other changes to > > make > > > > > > it work under malhar. > > > > > > > > > > > > Thanks > > > > > > > > > > > > On Wed, Nov 2, 2016 at 9:49 PM, Lakshmi Velineni > > > > > > <[email protected]> wrote: > > > > > > > Thanks for the suggestions and I am working on the process to > > > migrate > > > > > the > > > > > > > examples with the guidelines you mentioned. I will send out a > > list > > > of > > > > > > > examples and the destination modules very soon. > > > > > > > > > > > > > > > > > > > > > On Thu, Oct 27, 2016 at 1:43 PM, Thomas Weise < > > > > [email protected]> > > > > > > > wrote: > > > > > > >> > > > > > > >> Maybe a good first step would be to identify which examples to > > > bring > > > > > > over > > > > > > >> and where appropriate how to structure them in Malhar (for > > > example, > > > > I > > > > > > see > > > > > > >> multiple hdfs related apps that could go into the same Maven > > > > module). > > > > > > >> > > > > > > >> > > > > > > >> On Tue, Oct 25, 2016 at 1:00 PM, Thomas Weise <[email protected] > > > > > > wrote: > > > > > > >> > > > > > > >> > That would be great. There are a few things to consider when > > > > working > > > > > > on > > > > > > >> > it: > > > > > > >> > > > > > > > >> > * preserve attribtion > > > > > > >> > * ensure there is a test that runs the application in the CI > > > > > > >> > * check that dependencies are compatible license > > > > > > >> > * maybe extract common boilerplate code from pom.xml > > > > > > >> > > > > > > > >> > etc. > > > > > > >> > > > > > > > >> > Existing examples are under https://github.com/apache/ > > > > > > >> > apex-malhar/tree/master/demos > > > > > > >> > > > > > > > >> > Perhaps we should rename it to "examples" > > > > > > >> > > > > > > > >> > I also propose that each app has a README and we add those > for > > > > > > existing > > > > > > >> > apps as well. > > > > > > >> > > > > > > > >> > Thanks, > > > > > > >> > Thomas > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > On Tue, Oct 25, 2016 at 12:49 PM, Lakshmi Velineni < > > > > > > >> > [email protected]> wrote: > > > > > > >> > > > > > > > >> >> Can i work on this? > > > > > > >> >> > > > > > > >> >> Thanks > > > > > > >> >> Lakshmi Prasanna > > > > > > >> >> > > > > > > >> >> On Mon, Sep 12, 2016 at 9:41 PM, Ashwin Chandra Putta < > > > > > > >> >> [email protected]> wrote: > > > > > > >> >> > > > > > > >> >> > Here is the JIRA: > > > > > > >> >> > https://issues.apache.org/jira/browse/APEXMALHAR-2233 > > > > > > >> >> > > > > > > > >> >> > On Tue, Sep 6, 2016 at 10:20 PM, Amol Kekre < > > > > > [email protected]> > > > > > > >> >> wrote: > > > > > > >> >> > > > > > > > >> >> > > Good idea to consolidate them into Malhar. We should > > bring > > > in > > > > > as > > > > > > >> >> > > many > > > > > > >> >> > from > > > > > > >> >> > > this gitHub as possible. > > > > > > >> >> > > > > > > > > >> >> > > Thks > > > > > > >> >> > > Amol > > > > > > >> >> > > > > > > > > >> >> > > > > > > > > >> >> > > On Tue, Sep 6, 2016 at 6:02 PM, Thomas Weise > > > > > > >> >> > > <[email protected]> > > > > > > >> >> > > wrote: > > > > > > >> >> > > > > > > > > >> >> > > > I'm also for consolidating these different example > > > > locations. > > > > > > We > > > > > > >> >> should > > > > > > >> >> > > > also look if all of it is still relevant. > > > > > > >> >> > > > > > > > > > >> >> > > > The stuff from the DT repository needs to be brought > > into > > > > > shape > > > > > > >> >> > > > wrt > > > > > > >> >> > > > licensing, checkstyle, CI support etc. > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > On Tue, Sep 6, 2016 at 4:34 PM, Pramod Immaneni < > > > > > > >> >> > [email protected]> > > > > > > >> >> > > > wrote: > > > > > > >> >> > > > > > > > > > >> >> > > > > Sounds like a good idea. How about merging demos > with > > > > apps > > > > > as > > > > > > >> >> well? > > > > > > >> >> > > > > > > > > > > >> >> > > > > On Tue, Sep 6, 2016 at 4:30 PM, Ashwin Chandra > Putta > > < > > > > > > >> >> > > > > [email protected]> wrote: > > > > > > >> >> > > > > > > > > > > >> >> > > > > > Hi All, > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > We have a lot of examples for apex malhar > operators > > > in > > > > > the > > > > > > >> >> > following > > > > > > >> >> > > > > > repository which resides outside of malhar. > > > > > > >> >> > > > > > https://github.com/DataTorrent/examples/tree/ > > > > > > master/tutorials > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Now that it has grown quite a bit, does it make > > sense > > > > to > > > > > > >> >> > > > > > bring > > > > > > >> >> some > > > > > > >> >> > > of > > > > > > >> >> > > > > the > > > > > > >> >> > > > > > most common examples to malhar repository? > Probably > > > > under > > > > > > >> >> > > > > > apps > > > > > > >> >> > > > directory? > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > That way folks looking at malhar repository will > > have > > > > > some > > > > > > >> >> samples > > > > > > >> >> > to > > > > > > >> >> > > > > look > > > > > > >> >> > > > > > at without having to search elsewhere. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > -- > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > Regards, > > > > > > >> >> > > > > > Ashwin. > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> >> > -- > > > > > > >> >> > > > > > > > >> >> > Regards, > > > > > > >> >> > Ashwin. > > > > > > >> >> > > > > > > > >> >> > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > _______________________________________________________ > > > > > > > > Munagala V. Ramanath > > > > > > > > Software Engineer > > > > > > > > E: [email protected] | M: (408) 331-5034 | Twitter: @UnknownRam > > > > > > > > www.datatorrent.com | apex.apache.org > > > > > > > > > >
