Cool stuff Steve! Thanks!
Renato M. 2016-04-27 22:28 GMT+02:00 Steve Blackmon <sblack...@apache.org>: > All, > > I think we are pretty close to pulling all of the work on STREAMS-401 (new > web page) together into a pretty slick package. Thanks for your patience > and feedback. > > Some new additions include diagrams on the architecture page and within > many modules depicting streams components and data flow among them and with > third-party systems, a 5-step tutorial to run a useful stream, consistent > look-and-feel across all three repos, integrated breadcrumbs, and > rudimentary github and twitter integrations. > > Feel free to browse these links to see where we’re at. > > > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html > > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/index.html > > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/index.html > > All of the menu link now resolve, or will resolve as soon as everything is > promoted. If you get any Not Available errors that you CAN’T resolve by > substituting ‘latest’ for ‘0.3-incubating-SNAPSHOT’ in the url, please let > me know. > > Additionally if you identify any major issues or have any last suggestions > please let me know before I kickoff the process to merge the STREAMS-401 > branches and promote the site later this week. > > Steve Blackmon > sblack...@apache.org > > On Thu, Mar 03, 2016 at 9:48 AM Ate Douma <Ate Douma > <ate+douma+%3c...@douma.nu%3E>> wrote: > >> +1 to everything below :) >> >> Good stuff Steve! >> >> On 2016-03-01 23:50, Steve Blackmon wrote: >> > Following up: >> > >> > I took a pass at cleaning up the release markdown files and they now >> look >> > mostly the same as they did under apache cms. >> > >> > I also confirmed that site-deploy can write to a sub-directory of >> > streams.incubator.apache.org using scm:svn capability. >> > >> > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html >> >> > >> > Recall that streams-project and streams-examples each publish schema >> > artifacts to their maven sites, so maintaining prior versions is >> important. >> > streams-master doesn't currently, but since it's a parent of those >> other >> > two and prior experience has led to believe we're better off using the >> same >> > scheme and setting the streams-master in the pom to be a sibling of >> > streams-project and streams-example rather that publishing it to a >> > different URL entirely. This is why the version an artifactId are being >> > included in the above url. >> > >> > Brings up two issues - >> > resolving the latest release and/or snapshot version at any given time >> > ensuring a visitor to streams.incubator.apache.org lands on the >> > streams-master site for that version >> > >> > While symlinks are a viable way to resolving the latest release and/or >> > snapshot version while keeping previous release versions around >> > >> > i.e. forwarding >> > http://streams.incubator.apache.org/site/latest/streams-master/index.html >> >> > to >> > http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html >> >> > >> > they won't help us forward http://streams.incubator.apache.org to a >> > subdirectory of itself. >> > >> > Good news is apache cms supports .htaccess and the following snippet >> can. >> > >> > RewriteEngine *on* >> > >> > Redirect / >> > http://streams.incubator.apache.org/site/latest/streams-master/index.html >> >> > Feel free to confirm by visiting streams.staging.incubator.org (you'll >> be >> > redirected to the output of the maven site-deploy) >> > >> > *I ask that everyone on the list review and send me any concerns or >> > feedback they want incorporated before this goes live on >> > streams.incubator.apache.org <http://streams.incubator.apache.org>.* >> > >> > Procedurally, the way I expect that to happen is: >> > - I'll submit a PR to streams-master/master >> > - That PR will merge >> > - Update the streams-master jenkins job to run site-deploy, and do the >> > needful to get it the necessary credentials >> > - Going forward all builds of streams-master/master will publish to >> > site/${version}/streams-master >> > - Add a few more instructions to jenkins to maintain the symlinks and >> > .htaccess >> > - If there are any manual steps required to oversee, document that on >> the >> > site itself. >> > >> > Then, we'll move on to hooking streams-project and streams-examples up >> to >> > this new site (as children with an integrated breadcrumb) and to >> jenkins CD. >> > >> > Steve Blackmon >> > sblack...@apache.org >> > >> > On Mon, Feb 29, 2016 at 5:19 PM, Steve Blackmon <sblack...@apache.org> >> > wrote: >> > >> >> One thing which I think can 'spice up' the overview page a bit more >> would >> >>> be a >> >>> diagram providing some very high-level interaction between data >> sources >> >>> and >> >>> destinations and the streams role between them. >> >> >> >> >> >> I agree - a diagram depicting streams mediating between sources and >> >> destination on the landing page is appropriate. I'll work on this >> soon. >> >> >> >> And before replacing the current site several existing pages need to >> be >> >>> moved to >> >>> the new one. But once those are done I think this is great >> improvement. >> >> >> >> >> >> I've pushed a new version (to the source and site links from my prior >> >> message) that brings most of the existing content along, using the >> >> reporting-info plugin to generate the HTML where possible. >> >> >> >> On a few of the pages - SCM, CI, and Dependency Info - the output of >> the >> >> maven plugin simply isn't adequate for a multi-repository project such >> as >> >> this, so those are now managed within src/site/markdown >> >> >> >> Release Setup and Release Process need some cleanup but all of the >> content >> >> has been migrated. >> >> >> >> Other report-info pages - such as Dependencies, Dependency Management, >> >> Dependency Convergence, JavaDocs, Test JavaDocs, - don't make sense at >> the >> >> streams-master level but do exist and are useful within >> streams-project, >> >> streams-examples, and many of their respective sub-modules. >> >> >> >> In theory those additional report items will show up when navigating >> into >> >> the site hierarchy of the sub-projects (links to each sub-project are >> >> present along the header), which will inherit the structure, >> look-and-feel, >> >> and breadcrumb of this site revision. Naturally these changes must be >> >> deployed to streams.incubator.apache.org and sub-project pom.xml and >> >> site_en.xml will need some tweaking but I'm optimistic it will all fit >> >> together nicely. >> >> >> >> Steve Blackmon >> >> sblack...@apache.org >> >> >> >> On Sat, Feb 27, 2016 at 2:27 AM, Ate Douma <a...@douma.nu> wrote: >> >> >> >>> On 2016-02-25 23:50, Steve Blackmon wrote: >> >>> >> >>>> Hello streamers, >> >>>> >> >>>> I finally consolidated some materials I've been working on into a >> >>>> nice-looking thing that is a viable candidate to replace the current >> >>>> streams web page. Please take a look and let me know what you think. >> >>>> Sometime >> >>>> soon I expect to submit a vote to take down the previous site and >> replace >> >>>> it with something derived from this version. >> >>>> >> >>>> The proposed new site is temporarily being hosted at: >> >>>> >> >>>> >> http://people.apache.org/~sblackmon/0.3-incubating-SNAPSHOT/streams-master/index.html >> >>>> >> >>>> I placed all of the content into markdown files which are checked >> into a >> >>>> branch of streams-master. This approach allows us to author the site >> >>>> using >> >>>> markdown, package and deploy it with maven-site, and manage changes >> using >> >>>> git. >> >>>> >> >>> >> >>> This looks cool! >> >>> >> >>> I like the new overview and faq pages, definitely more concrete about >> >>> Streams. >> >>> >> >>> One thing which I think can 'spice up' the overview page a bit more >> would >> >>> be a >> >>> diagram providing some very high-level interaction between data >> sources >> >>> and >> >>> destinations and the streams role between them. >> >>> >> >>> And before replacing the current site several existing pages need to >> be >> >>> moved to >> >>> the new one. But once those are done I think this is great >> improvement. >> >>> >> >>> >> >>>> If you'd like to submit specific changes feel free to do so as >> >>>> pull-requests to: >> >>>> https://github.com/steveblackmon/incubator-streams-master.git >> >>>> >> >>>> Referring back to the original list of goals, some are addressed in >> whole >> >>>> or in part with this revision, others not just yet. >> >>>> >> >>>> ADDRESSED IN WHOLE: >> >>>> - the website isn't providing any practical guidance *why* or *how* >> to >> >>>> use >> >>>> Streams >> >>>> - there is no public javadoc or other technical documentation >> >>>> linked/hosted >> >>>> - there is a 'wiki (coming soon)' link which never has been >> activated >> >>>> >> >>>> ADDRESSED IN PART: >> >>>> - its missing concrete examples and recognizable use-cases for which >> >>>> Streams might >> >>>> be used, nor comparison against other solutions, and why Streams >> would >> >>>> be a >> >>>> good/better solution >> >>>> - the Architecture page only has some basic wording but provides no >> >>>> insight >> >>>> whatsoever about the concrete implementation and certainly not its >> >>>> architecture >> >>>> >> >>>> This is just a start. Communication on the list has been dead >> lately, >> >>>> but >> >>>> I'm hoping to change that beginning with this effort to explain what >> is >> >>>> unique and valuable about Streams, and why. Hopefully this take on >> the >> >>>> project will inspire others to get on-board and help out in the >> coming >> >>>> months. >> >>>> >> >>> >> >>> +1, good work! >> >>> >> >>> >> >>> >> >>>> Steve Blackmon >> >>>> sblack...@apache.org >> >>>> >> >>>> On Thu, Jan 7, 2016 at 10:49 AM, Ate Douma <a...@douma.nu> wrote: >> >>>> >> >>>> Hi Steve, >> >>>>> >> >>>>> Cool. I don't have much extra input or ideas right now, but looking >> >>>>> forward to the steps taken you proposed below. >> >>>>> And about getting out a next release first soon, if it is not >> >>>>> side-tracking these other steps much, sure +1 >> >>>>> >> >>>>> Ate >> >>>>> >> >>>>> On 2016-01-05 15:35, Steve Blackmon wrote: >> >>>>> >> >>>>> All, >> >>>>>> >> >>>>>> Ate gave us a reality check during our board report in December >> and >> >>>>>> enumerated some of the most prominent ways our documentation >> currently >> >>>>>> falls short - of Apache community standards and of any reasonable >> >>>>>> useful-ness test. I've responded in-line below - please kick in >> any >> >>>>>> ideas >> >>>>>> you have. >> >>>>>> >> >>>>>> >> >>>>>> - the website isn't providing any practical guidance *why* or >> *how* to >> >>>>>> use >> >>>>>> Streams >> >>>>>> >> >>>>>> >> >>>>>> I agree. This is the content that belongs on the streams website >> >>>>>> landing >> >>>>>> page (high-level) or linked from the body (the details). I will >> take a >> >>>>>> first shot at this and circulate for feedback. >> >>>>>> >> >>>>>> - its missing concrete examples and recognizable use-cases for >> which >> >>>>>> Streams might be used, nor comparison against other solutions, and >> why >> >>>>>> Streams would be a good/better solution >> >>>>>> >> >>>>>> >> >>>>>> The examples documentation site [1] are meant to serve as both >> concrete >> >>>>>> examples and recognizable use-cases, but there are a few problems >> with >> >>>>>> the >> >>>>>> current setup. >> >>>>>> >> >>>>>> 1. The examples site lacks plain-language READMEs for the root >> >>>>>> module >> >>>>>> and the first children of the root module (the runtime-specific >> >>>>>> aggregators). >> >>>>>> 2. The examples site isn't linked from the landing page of the >> >>>>>> website. >> >>>>>> 3. There is no permanent url for latest/current. >> >>>>>> 4. There hasn't been a formal release of this repo. >> >>>>>> 5. There are broken links within the markdown of some examples. >> >>>>>> 6. The plain-language descriptions of the examples themselves >> >>>>>> could be >> >>>>>> >> >>>>>> improved. >> >>>>>> >> >>>>>> All of these can be addressed (easily I think) and I'm happy to >> take >> >>>>>> them >> >>>>>> on. >> >>>>>> >> >>>>>> Additionally, a page (or several) comparing streams to frameworks >> such >> >>>>>> as >> >>>>>> Storm, Spark, Samza, Flink, etc... and describing how they are >> >>>>>> different/complementary would be valuable. These questions come up >> a >> >>>>>> lot. >> >>>>>> >> >>>>>> - the Architecture page only has some basic wording but provides >> no >> >>>>>> insight >> >>>>>> whatsoever about the concrete implementation and certainly not its >> >>>>>> architecture >> >>>>>> >> >>>>>> >> >>>>>> I think this page needs either an multi-page outline format >> heavily >> >>>>>> linked >> >>>>>> into READMEs and interfaces throughout the streams repos, or a >> system >> >>>>>> diagram depicting a simple streams deployment alongside the >> vocabulary. >> >>>>>> Maybe both as several pages. >> >>>>>> >> >>>>>> - there is no public javadoc or other technical documentation >> >>>>>> linked/hosted >> >>>>>> >> >>>>>> >> >>>>>> We do generate java-docs for releases and snapshots. We should add >> a >> >>>>>> link >> >>>>>> to the root page. We've made progress but there are still many >> classes >> >>>>>> lacking any javadoc header. We should reiterate a commitment to >> fixing >> >>>>>> that and a schedule for deprecating and deleting any class that >> doesn't >> >>>>>> still serve a distinct purpose. >> >>>>>> >> >>>>>> An index of the many resource files (json/xml schemas, conversion >> >>>>>> rules, >> >>>>>> shell scripts, etc...) published to the project and examples >> release >> >>>>>> should >> >>>>>> also placed on the main website. Ideally comprehensive, but if not >> at >> >>>>>> least enough to show a developer that they exist and can be >> >>>>>> hard-linked to >> >>>>>> within their own project resources. >> >>>>>> >> >>>>>> >> >>>>>> - there is a 'wiki (coming soon)' link which never has been >> activated >> >>>>>> >> >>>>>> >> >>>>>> This link (actually most of streams.incubator.apache.org) >> predates the >> >>>>>> regular publication of the streams-project maven site where >> READMEs and >> >>>>>> other resources are hosted. Personally I'd prefer to see >> documentation >> >>>>>> improving within the code-base rather than in a separate wiki in >> the >> >>>>>> near-term and suggest we just delete this link. >> >>>>>> >> >>>>>> - the mailing list is NOT used for any form of >> discussion/information >> >>>>>> other >> >>>>>> than JIRA and Git(hub) notifications >> >>>>>> - especially the latter doesn't give a newbie any indication what >> is >> >>>>>> going >> >>>>>> on, nor how to (start) interact >> >>>>>> >> >>>>>> This is unfortunate and must change for the community to grow. I >> know >> >>>>>> I'm >> >>>>>> guilty of having ad-hoc off-list discussions with my team that >> lead to >> >>>>>> new >> >>>>>> stories and pull requests. Mea culpa, I resolve to stop doing this >> in >> >>>>>> 2016. >> >>>>>> >> >>>>>> - there are no blog posts (that I am aware of) around using/trying >> out >> >>>>>> Streams either >> >>>>>> >> >>>>>> >> >>>>>> This is a critical missing piece of the puzzle. Static HTML via >> apache >> >>>>>> httpd is not the ideal platform for hosting a blog but some other >> >>>>>> projects >> >>>>>> have made it work. I'd like us to replace 'Latest News' with a >> >>>>>> navigable, >> >>>>>> indexable set of blog posts going forward, where release summaries >> and >> >>>>>> new >> >>>>>> examples can be published, and 'in-the-wild' mentions of the >> project >> >>>>>> can >> >>>>>> be >> >>>>>> aggregated. >> >>>>>> >> >>>>>> >> >>>>>> And I personally would prefer the dev@ list to be much more / >> >>>>>> primarily >> >>>>>> used for actual human interaction, not just these notifications >> from >> >>>>>> JIRA >> >>>>>> and Github. >> >>>>>> Maybe change the configuration to delegate (some of) these to the >> >>>>>> commit@ >> >>>>>> list instead? >> >>>>>> >> >>>>>> >> >>>>>> I agree. AFAICT though com...@streams.incubator.apache.org does >> not >> >>>>>> exist. Per http://apache.org/dev/committers.html#mail we should >> hold >> >>>>>> a >> >>>>>> vote to create it. I'll open that vote today. >> >>>>>> >> >>>>>> What I suggest is to for a while stop (or largely postpone) >> working on >> >>>>>> code >> >>>>>> but instead working on any/all of the above points first... >> >>>>>> >> >>>>>> >> >>>>>> I agree that making the website helpful is the top project >> priority. >> >>>>>> >> >>>>>> Even working on a next release IMO is far less important than >> first >> >>>>>> building up some explanation *why* a release of Stream is needed. >> >>>>>> >> >>>>>> >> >>>>>> I agree that these concerns should be addressed before the next >> >>>>>> release. >> >>>>>> The main reason for releasing soon is that per [4] there are 36 >> issues >> >>>>>> resolved since the 0.2 release, including quite a few bugs and >> >>>>>> improvements >> >>>>>> related to stability, deployment flexibility, or likely to impact >> the >> >>>>>> experience of a new developer. >> >>>>>> >> >>>>>> [1] >> >>>>>> >> >>>>>> >> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/ >> >> >>>>>> [2] >> >>>>>> >> >>>>>> >> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/apidocs/index.html >> >> >>>>>> [3] >> >>>>>> >> >>>>>> >> >>>>>> http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html >> >> >>>>>> [4] >> >>>>>> >> >>>>>> >> >>>>>> >> https://issues.apache.org/jira/issues/?jql=project%20%3D%20STREAMS%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20in%20unreleasedVersions() >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>>> >> >>> >> >> >> > >> >> >
