Cool; thanks Henry! On Mon, May 25, 2020 at 7:49 PM Henry Suryawirawan <hsuryawira...@google.com> wrote:
> Hi Austin, > > The remote-info yaml files would get updated upon updating the course on > Stepik, not entirely recreated as if from scratch. > The important IDs metadata that track the course ID, section ID, lesson > ID, and task ID are preserved. > Having these files in the repo allows multiple people to be able to update > the same courses on Stepik without resulting in new courses everytime it > gets updated. > > As an illustration, let's take the Go Katas. > The course-remote-info.yaml tracks the course ID as `id: 70387`. > This corresponds to the equivalent Stepik course page: > https://stepik.org/course/70387/promo > This course ID has also been endorsed by JetBrains to feature visibly on > the course list. > If we don't keep track of the IDs, we'll always recreate the course with > new IDs every time it gets uploaded. > I hope this helps to clarify the needs for the remote-info files to be in > the repo. > > I've explored a way to automatically update the course that can be > triggered by the CI. > What I found is that at the moment it is better to update the course from > the IDE menu, otherwise we would have to find out what actually happens > behind the menu and reverse engineer the steps. > It may be possible to do, but I am also concerned if JetBrains updates the > plugin and changes the way it handles the course upload, and that we have > to keep up with the changes at the same time. > > On the stats, we have some statistics that are publicly available and > privately available for the course instructors. > The promo page shows the publicly available stat of the number of learners > who have tried the course, any star and review, e.g. for Java Katas ( > https://stepik.org/course/54530/promo). > The private stats include number of learners per day, task pass rate, the > learners, etc. > > > Henry Suryawirawan > > Strategic Cloud Engineer > > hsuryawira...@google.com > > > > > > > On Tue, May 26, 2020 at 9:27 AM Austin Bennett < > whatwouldausti...@gmail.com> wrote: > >> Hi Henry, >> >> Cool. Most makes sense. >> >> What I am missing is the need for the '*-remote-info.yaml' in the beam >> repo (I do get what purpose it serves for Stepik). There is probably a >> good reason. >> >> To get nitpicky (am genuinely curious) --> It seems that this sort of >> metadata would get (re-)created upon (re-)uploading a course. What does >> persisting it to the repo get us? This is also in line with nascent >> thoughts of auto-deploying katas to stepik on accepted merge into >> learning/katas/ -- rather than this being done manually. Perhaps this is >> just due to my lack of having spent much time with Stepik. >> >> Also, do we have download statistics from Stepik for the Katas? What >> other bits of information can we gather from Stepik (as you share, if >> Stepik is using *-remote-info.yaml for tracking, what information are we >> able to gather from there)? >> >> Thanks, >> Austin >> >> On Mon, May 25, 2020 at 1:10 AM Henry Suryawirawan < >> hsuryawira...@google.com> wrote: >> >>> Hi Austin, >>> >>> Thanks for your help in adding a new lesson. >>> I will have a look and help to review the pull request. >>> >>> On your questions: >>> 1. Apart from the *-remote-info.yaml, the other yaml files should >>> contain Apache license header. We explicitly turn off license header check >>> for the remote-info files as can be referred in the build.gradle >>> <https://github.com/apache/beam/blob/master/build.gradle>. The reason >>> is because the remote-info files are auto generated, and they will always >>> get replaced whenever we update the course on Stepik. >>> All the YAML files are important and have to be included as part of the >>> repository. The {task, lesson,section}-info.yaml files are metadata >>> files used by the JetBrains EduTools plugin for Educational projects. The >>> *-remote-info.yaml files contain metadata information (e.g. the IDs) >>> important for Stepik to track our courses. >>> >>> 2. We do not leave out the beam.Create. Apart from the Introduction >>> lesson, you can find it in the cmd/main.go file. We explicitly create the >>> main.go file in order for the learner to be able to also run the pipeline >>> independently and observe the output, just like when they write the >>> pipeline normally. >>> >>> Hope my answers help to clarify. >>> >>> >>> Henry Suryawirawan >>> >>> Strategic Cloud Engineer >>> >>> hsuryawira...@google.com >>> >>> >>> >>> >>> >>> On Mon, May 25, 2020 at 7:13 AM Austin Bennett < >>> whatwouldausti...@gmail.com> wrote: >>> >>>> @Rion, @Henry Suryawirawan <hsuryawira...@google.com>, @Damon, I >>>> added a Flatten Kata for Go. Please have a look: >>>> https://github.com/apache/beam/pull/11806 -- tagged all of you as >>>> other authors of Katas. >>>> >>>> A few questions: >>>> >>>> 1) Across all the katas, we have files '{task, >>>> lesson,section}-remote-info.yaml'. These files do not contain the apache >>>> license, and I imagine they are generated by Steptik/other (also, to get >>>> working locally those files were not needed). Should these files be >>>> ignored (via .gitignore) and kept out of the Beam repository? Wondering >>>> why we would want those in the repo, and if yes, should they have the >>>> Apache License on them? >>>> >>>> 2) On Go Katas generally. I wrote this one following convention of >>>> other Go Katas found in the repository. For the Java and Python versions, >>>> the code that people work with includes seeing the Beam.Create. This is >>>> left out of the GoLang katas, and kept behind the scenes. Is there >>>> reasoning for breaking from the convention of the other Katas? >>>> https://godoc.org/github.com/apache/beam/sdks/go/pkg/beam#Create >>>> >>>> Thanks, >>>> Austin >>>> >>>> >>>> On Thu, May 21, 2020 at 8:00 PM Rion Williams <rionmons...@gmail.com> >>>> wrote: >>>> >>>>> Hi Henry, >>>>> >>>>> I submitted a pull request related to the Beam Katas that can be found >>>>> here (https://github.com/apache/beam/pull/11761) and included you as >>>>> a reviewer. I updated all of the related metadata, generated the course, >>>>> and tested through it to ensure it worked as expected (and the >>>>> placeholders >>>>> all worked as expected as well). >>>>> >>>>> The generated course can be found here on Stepik ( >>>>> https://stepik.org/course/72488 >>>>> <https://stepik.org/course/72488/promo>) and I’ve reached out to a >>>>> few folks to put it through its paces in the wild. >>>>> >>>>> Let me know if there’s anything else I can do or changes that need to >>>>> be made in the PR or elsewhere. >>>>> >>>>> Thanks again, >>>>> >>>>> Rion >>>>> >>>>> On May 20, 2020, at 2:12 AM, Henry Suryawirawan < >>>>> hsuryawira...@google.com> wrote: >>>>> >>>>> >>>>> Yeah there was a recent pull request merged for the md file format >>>>> change. >>>>> I checked your repo and it still contains the task.html, so need your >>>>> help to merge with the latest master. >>>>> >>>>> For the answer placeholder, you may refer to this doc >>>>> <https://www.jetbrains.com/help/education/educator-start-guide.html> first >>>>> to understand how it works. >>>>> It will auto update the placeholder position in the task-info.yaml. >>>>> >>>>> If you encounter any issue, just let me know. >>>>> Thanks Rion. >>>>> >>>>> >>>>> Regards, >>>>> Henry >>>>> >>>>> >>>>> >>>>> On Wed, May 20, 2020 at 12:43 PM Rion Williams <rionmons...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi Henry, >>>>>> >>>>>> Thanks for the quick response, I appreciate it. I believe that I >>>>>> pulled the latest from master a day or so ago, so I’ll make sure to pull >>>>>> the most recent changes in. >>>>>> >>>>>> As far as the placeholders, they aren’t currently present (as I don’t >>>>>> believe they were present in the Java ones within the learning/katas >>>>>> directory), however I can easily add those in to align with the content >>>>>> of >>>>>> the existing course. I wasn’t entirely sure based on the existing >>>>>> directories if the files should contain the placeholders or the actual >>>>>> implementations, either way, it’s a pretty trivial series of changes. >>>>>> >>>>>> I’ll try to put these together tomorrow and push up a PR. I’ll make >>>>>> sure to include you as a reviewer. >>>>>> >>>>>> Thanks for the initial feedback, >>>>>> >>>>>> Rion >>>>>> >>>>>> On May 19, 2020, at 11:15 PM, Henry Suryawirawan < >>>>>> hsuryawira...@google.com> wrote: >>>>>> >>>>>> >>>>>> Thanks Rion for adding the Kotlin version. >>>>>> This is great to show other people that Beam can be done in Kotlin >>>>>> too! >>>>>> >>>>>> I can help to review your work. >>>>>> Please help to incorporate the Java Katas latest changes from master. >>>>>> There are recent changes to the task description file format from >>>>>> html to md. >>>>>> Please also help to remove all the *-remote-info.yaml files. >>>>>> I assume that you've adjusted the answer placeholders in all tasks as >>>>>> well. >>>>>> Afterwards, you can create a pull request and assign me as reviewer. >>>>>> >>>>>> Please reach out to me if you have any questions. >>>>>> >>>>>> >>>>>> Regards, >>>>>> Henry >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, May 20, 2020 at 3:33 AM Rion Williams <rionmons...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Sure! I ran through all of the tests locally on my branch (as tests) >>>>>>> and then performed a check against all of the known tasks (via Course >>>>>>> Creator > Check All Tasks) and 35/36 tasks passed successfully with the >>>>>>> only one that didn't being a Built-in IO one that doesn't currently have >>>>>>> any implementation. Although, I'd love for someone else to try the same >>>>>>> thing since as far as I can tell it "works on my machine". >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> Rion >>>>>>> >>>>>>> On 2020/05/19 19:12:57, Pablo Estrada <pabl...@google.com> wrote: >>>>>>> > This is really cool Rion! >>>>>>> > >>>>>>> > I believe it's possible to start trying out the katas from your >>>>>>> branch? If >>>>>>> > so, I can give them a try, and use that as a review... >>>>>>> > Henry, any other ideas? >>>>>>> > >>>>>>> > On Tue, May 19, 2020 at 12:04 PM Rion Williams < >>>>>>> rionmons...@gmail.com> >>>>>>> > wrote: >>>>>>> > >>>>>>> > > Hi all, >>>>>>> > > >>>>>>> > > I was recently added as a contributor and created a JIRA ticket >>>>>>> related to >>>>>>> > > the existing Katas ( >>>>>>> https://issues.apache.org/jira/browse/BEAM-10027), >>>>>>> > > specifically creating one that targets Kotlin specific as there >>>>>>> are quite a >>>>>>> > > few existing examples out there for Kotlin, so I thought a Kata >>>>>>> course that >>>>>>> > > would parallel the existing Java, Go, and Python ones. >>>>>>> > > >>>>>>> > > I basically ported over the existing Java Katas, added the >>>>>>> appropriate >>>>>>> > > dependencies, and converted all of the Java files over to >>>>>>> Kotlin, and >>>>>>> > > ensured that all of the tests pass as expected. I'd love outside >>>>>>> of this to >>>>>>> > > see if we can shift it to a Stepik course as well if that seems >>>>>>> reasonable >>>>>>> > > similar to those mentioned in this thread. >>>>>>> > > >>>>>>> > > My current branch awaiting a PR can be found here ( >>>>>>> > > https://github.com/rionmonster/beam/tree/BEAM-10027), however >>>>>>> I'm unsure >>>>>>> > > who would be the best to review such a PR and what other steps >>>>>>> might need >>>>>>> > > to be taken before trying to get it merged in. >>>>>>> > > >>>>>>> > > Any feedback would be welcome! >>>>>>> > > >>>>>>> > > Thanks, >>>>>>> > > >>>>>>> > > Rion >>>>>>> > > >>>>>>> > > On 2020/05/14 23:40:45, Rion Williams <rionmons...@gmail.com> >>>>>>> wrote: >>>>>>> > > > +1 on the contributions front. My team and I have been working >>>>>>> with Beam >>>>>>> > > primarily with Kotlin and I recently added the appropriate >>>>>>> dependencies to >>>>>>> > > Gradle and performed a bit of conversions and have it working as >>>>>>> expected >>>>>>> > > against the existing Java course. >>>>>>> > > > >>>>>>> > > > I don’t know how many others are actively working with Kotlin >>>>>>> and Beam, >>>>>>> > > but I’d love to work on transitioning that into a proper course >>>>>>> (assuming >>>>>>> > > there’s interest in it). >>>>>>> > > > >>>>>>> > > > > On May 14, 2020, at 10:32 AM, Nathan Fisher < >>>>>>> nfis...@junctionbox.ca> >>>>>>> > > wrote: >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > Yes write IO >>>>>>> > > > > >>>>>>> > > > >> On Thu, May 14, 2020 at 05:41, Henry Suryawirawan < >>>>>>> > > hsuryawira...@google.com> wrote: >>>>>>> > > > >> Yeah certainly we can expand it further. >>>>>>> > > > >> There are more lessons that definitely can be added further. >>>>>>> > > > >> >>>>>>> > > > >> >Eg more the write side windowing interactions? >>>>>>> > > > >> Are you referring to Write IOs? >>>>>>> > > > >> >>>>>>> > > > >> >>>>>>> > > > >> >>>>>>> > > > >>> On Wed, May 13, 2020 at 11:56 PM Nathan Fisher < >>>>>>> > > nfis...@junctionbox.ca> wrote: >>>>>>> > > > >>> I went through them earlier this week! Definitely helpful. >>>>>>> > > > >>> >>>>>>> > > > >>> Is it possible to expand the katas available in the lO >>>>>>> section? Eg >>>>>>> > > more the write side windowing interactions? >>>>>>> > > > >>> >>>>>>> > > > >>>> On Wed, May 13, 2020 at 11:36, Luke Cwik < >>>>>>> lc...@google.com> wrote: >>>>>>> > > > >>>> These are an excellent learning tool. >>>>>>> > > > >>>> >>>>>>> > > > >>>>> On Tue, May 12, 2020 at 11:02 PM Pablo Estrada < >>>>>>> pabl...@google.com> >>>>>>> > > wrote: >>>>>>> > > > >>>>> Sharing Damon's email with the user@ list as well. >>>>>>> Thanks Damon! >>>>>>> > > > >>>>> >>>>>>> > > > >>>>>> On Tue, May 12, 2020 at 9:02 PM Damon Douglas < >>>>>>> > > douglas.da...@gmail.com> wrote: >>>>>>> > > > >>>>>> Hello Everyone, >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> If you don't already know, there are helpful >>>>>>> instructional tools >>>>>>> > > for learning the Apache Beam SDKs called Beam Katas hosted on >>>>>>> > > https://stepik.org. Similar to traditional Kata, they are >>>>>>> meant to be >>>>>>> > > repeated as practice. Before practicing the katas myself, I >>>>>>> found myself >>>>>>> > > copy/pasting code (Please accept my confession 😎 ). Now I find >>>>>>> myself >>>>>>> > > actually composing pipelines. Just like kata forms, you find >>>>>>> them becoming >>>>>>> > > part of you. If you are interested, below are listed the >>>>>>> current available >>>>>>> > > katas: >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> 1. Java - https://stepik.org/course/54530 >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> 2. Python - https://stepik.org/course/54532 >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> 3. Go (in development) - >>>>>>> https://stepik.org/course/70387 >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> If you are absolutely brand new to Beam and it scares >>>>>>> you like it >>>>>>> > > scared me, come talk to me. >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> Best, >>>>>>> > > > >>>>>> >>>>>>> > > > >>>>>> Damon >>>>>>> > > > >>> -- >>>>>>> > > > >>> Nathan Fisher >>>>>>> > > > >>> w: http://junctionbox.ca/ >>>>>>> > > > > -- >>>>>>> > > > > Nathan Fisher >>>>>>> > > > > w: http://junctionbox.ca/ >>>>>>> > > > >>>>>>> > > >>>>>>> > >>>>>>> >>>>>>
