Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-26 Thread Aizhamal Nurmamat kyzy
Hi, Alexey, Yes, now we can keep updating the website as before. Here is a contributor guide: https://github.com/apache/beam/blob/master/website/CONTRIBUTE.md Thanks for checking :) Aizhamal On Tue, May 26, 2020 at 8:39 AM Alexey Romanenko wrote: > Thank you to everybody who worked on this

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-26 Thread Alexey Romanenko
Thank you to everybody who worked on this migration. I just wanted to clarify - does it mean, since it’s finished now, we can update a website as before? > On 15 May 2020, at 01:33, Aizhamal Nurmamat kyzy wrote: > > Thank you Ahmet, Brian, Robert and everyone else who spent time working on

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-14 Thread Aizhamal Nurmamat kyzy
Thank you Ahmet, Brian, Robert and everyone else who spent time working on this. The pull requests are now merged and the website seems to be working as expected [1]. One minor issue that I have noticed is that the code blocks have a grey background, which makes it less accessible than before. I

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-14 Thread Pablo Estrada
Here's a zipped-up tree from a staged sample of the website: https://drive.google.com/file/d/1LKL936tBJ79jpjvlL5vC5uYYwTHsWXiJ/view?usp=sharing I'd also suggest tagging the commit, so we can find the fist commit later on for reference. I can push the tag after the PR is merged. On Thu, May 14,

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-14 Thread Ahmet Altay
On Thu, May 14, 2020 at 9:16 AM Aizhamal Nurmamat kyzy wrote: > Thank you all for reviewing and validating this pull request. I see that > all tests are passing now, should we merge it? > +1 to merging now. Before the merge, please share a link to an archive copy of the old website. After the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-14 Thread Aizhamal Nurmamat kyzy
Thank you all for reviewing and validating this pull request. I see that all tests are passing now, should we merge it? On Wed, May 13, 2020, 5:41 PM Ahmet Altay wrote: > Thank you! Let's merge it once tests are done. > > On Wed, May 13, 2020 at 5:23 PM Robert Bradshaw > wrote: > >> I took a

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-13 Thread Ahmet Altay
Thank you! Let's merge it once tests are done. On Wed, May 13, 2020 at 5:23 PM Robert Bradshaw wrote: > I took a (non-comprehensive) look at these as well, and didn't see any > issues, so am happy to sign off on this. Thanks Nam, Brian, Ahmet, and > everyone else. > > On Wed, May 13, 2020 at

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-13 Thread Robert Bradshaw
I took a (non-comprehensive) look at these as well, and didn't see any issues, so am happy to sign off on this. Thanks Nam, Brian, Ahmet, and everyone else. On Wed, May 13, 2020 at 7:58 AM Nam Bui wrote: > Hi Ahmet, > "Does this mean the internal links (e.g. contribute/team) will disappear?" >

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-13 Thread Nam Bui
Hi Ahmet, "Does this mean the internal links (e.g. contribute/team) will disappear?" Yes, I'd like to get rid of them. And to make sure it won't appear to confuse people, I replaced all of the spots using "contribute/team" with the external one. Currently, we only have 2 "redirect_to" links which

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-12 Thread Ahmet Altay
- I reviewed the diff output with Nam's explanations. The change looks minimal. Large diffs are primarily coming from index and redirect files. codeblocks have differences but the content is seemingly preserved. IIUC, the source of truth is snippet files anyway. (It would be good to get one more

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-11 Thread Nam Bui
Updates for today: - Thanks Brian & Ahmet for your reviews. I left my comments for some of the questions and also adapted new changes to the reviews [1]. - I see that the new blog post was merged yesterday, so I added it to the PR as well. I briefly tried the script from Robert with the input of

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-11 Thread Nam Bui
Hi, @Ahmet: Yeah, it's all clear to me. :) @Robert: Thanks for your ideas and also the script. It really helps me to serve my works. Best regard! On Sat, May 9, 2020 at 2:10 AM Ahmet Altay wrote: > This sounds reasonable to me. Thank you. Nam, does it make sense to you? > > On Fri, May 8,

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Ahmet Altay
This sounds reasonable to me. Thank you. Nam, does it make sense to you? On Fri, May 8, 2020 at 11:53 AM Robert Bradshaw wrote: > I'd really like to not see this work go to waste, both the original > revision, the further efforts Nam has done in making it more manageable to > review, and the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Robert Bradshaw
I'd really like to not see this work go to waste, both the original revision, the further efforts Nam has done in making it more manageable to review, and the work put into reviewing this so far, so we can get the benefits of being on Hugo. How about this for a concrete proposal: (1) We get

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Robert Bradshaw
Here's a script that we could run on the old and new sites that should quickly catch any major issues but not get caught up in formatting minutia. On Fri, May 8, 2020 at 10:23 AM Robert Bradshaw wrote: > On Fri, May 8, 2020 at 9:58 AM Aizhamal Nurmamat kyzy > wrote: > >> I understand the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Robert Bradshaw
On Fri, May 8, 2020 at 9:58 AM Aizhamal Nurmamat kyzy wrote: > I understand the difficulty, and this certainly comes with lessons learned > for future similar projects. > > To your questions Robert: > (1 and 2) I will commit to review the text in the resulting pages. I will > try and use some

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Aizhamal Nurmamat kyzy
Also an update from +Nam Bui : "This commit [1] is up-to-date. So I walked through all of markdown files. Apart from Syntax changes between Jekyll & Hugo, if there were any differences in contents regarding to removed/added/modified, I would have double check the text with the current website. I

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Aizhamal Nurmamat kyzy
I understand the difficulty, and this certainly comes with lessons learned for future similar projects. To your questions Robert: (1 and 2) I will commit to review the text in the resulting pages. I will try and use some automation to extract visible text from each page and diff it with the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Brian Hulette
I'm -0 on merging as-is. I have the same concerns as Robert and he's voiced them very well so I won't waste time re-airing them. (2) I spot checked the content, pulled out some common patterns, and > it mostly looks good, but there were also some issues (e.g. several > pages were replaced with

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-08 Thread Robert Bradshaw
This is a tough situation. It would have been much better if this transition was structured in such a way that the review was more manageable (e.g. the suggestion of scripts, not mixing in voluminous unnecessary changes like whitespace, and not updating content), and possibly even incrementally

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-07 Thread Aizhamal Nurmamat kyzy
Thank you Ahmet. Robert/Brian, what do you think? The website staging and pre commit tests have passed [1]. If nobody has objections, we could merge it soon. [1] https://github.com/apache/beam/pull/11554 On Thu, May 7, 2020 at 11:38 AM Ahmet Altay wrote: > > > On Thu, May 7, 2020 at 10:50

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-07 Thread Ahmet Altay
On Thu, May 7, 2020 at 10:50 AM Aizhamal Nurmamat kyzy wrote: > Thanks for the writeup Ahmet. > > My bias is to move forward and merge the PR. After this, we'll review the > outcome, and ensure that all the content is there. Nam will help us with > that. > The reason that I'd like to move

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-07 Thread Aizhamal Nurmamat kyzy
Thanks for the writeup Ahmet. My bias is to move forward and merge the PR. After this, we'll review the outcome, and ensure that all the content is there. Nam will help us with that. The reason that I'd like to move forward and merge what we have now - is that if we don't do that, the work done

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Ahmet Altay
On Wed, May 6, 2020 at 2:33 PM Aizhamal Nurmamat kyzy wrote: > >> > 1) Currently, the main blocker for merging is Staging Test Failures. >> >> That and finishing the review. (Is someone tracking/coordinating this?) >> > > I am coordinating the work on the failed tests, but I would need other >

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Aizhamal Nurmamat kyzy
> > > > 1) Currently, the main blocker for merging is Staging Test Failures. > > That and finishing the review. (Is someone tracking/coordinating this?) > I am coordinating the work on the failed tests, but I would need other committer's help to perform the review. @Ahmet, could you help us

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Robert Bradshaw
On Wed, May 6, 2020 at 1:01 PM Aizhamal Nurmamat kyzy wrote: > > Hi all, > > Nam and Brian have been working together on blog post names and the decision > was to keep them as they are for now, because Hugo doesn’t fully support the > feature that was mentioned earlier [1]. Also I believe this

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Pablo Estrada
I believe the troublesome tasks are these three: https://github.com/apache/beam/blob/2592c8396ae1a68dadc351135c2a79397fb70942/website/build.gradle#L95-L108 (to avoid confusion, note that these are being added as part of the PR, although they appear in a commit on apache/beam in the link). Nam,

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Brian Hulette
I added some detail about the failures in the PR [1]. It seems for some reason this PR is causing those directories (node_modules and themes) to be created with root ownership. All subsequent runs of those jobs on the same worker (for other PRs as well) try to clean up the old workspace, and they

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Aizhamal Nurmamat kyzy
Hi all, Nam and Brian have been working together on blog post names and the decision was to keep them as they are for now, because Hugo doesn’t fully support the feature that was mentioned earlier [1]. Also I believe this can be done after merging the PR. 1) Currently, the main blocker for

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-06 Thread Robert Bradshaw
On Mon, May 4, 2020 at 7:07 PM Ahmet Altay wrote: > >> On Mon, May 4, 2020 at 6:30 PM Robert Bradshaw wrote: >>> >>> I took the massive commit and split it up into: >>> >>> (1) Infrastructure changes (basically everything outside of >>> (website/www/site/content) >>> (2) Sed script changes, and

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Ahmet Altay
On Mon, May 4, 2020 at 6:58 PM Aizhamal Nurmamat kyzy wrote: > Thanks everyone for your feedback and support with the review. Please add > any other comments so we can address them soon, if not please share your > LGTMs. > > @Robert, thanks for separating the PR! > > @Thomas, regarding your

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Aizhamal Nurmamat kyzy
Thanks everyone for your feedback and support with the review. Please add any other comments so we can address them soon, if not please share your LGTMs. @Robert, thanks for separating the PR! @Thomas, regarding your question "There are some changes missing though (for example [2]), are you

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Robert Bradshaw
I took the massive commit and split it up into: (1) Infrastructure changes (basically everything outside of (website/www/site/content) (2) Sed script changes, and (3) Manual changes (everything not in (1) and (2)). It does seem that (3) has a number of unintentional changes, some stylistic (e.g.

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Robert Bradshaw
On Mon, May 4, 2020 at 6:02 PM Thomas Weise wrote: > > I took a brief look at [1] and looks good overall. > > There are some changes missing though (for example [2]), are you planning to > add more recent commits later? > > Also, there was an earlier question from Brian regarding the possibility

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Thomas Weise
I took a brief look at [1] and looks good overall. There are some changes missing though (for example [2]), are you planning to add more recent commits later? Also, there was an earlier question from Brian regarding the possibility to retain the post dates in blog file names. I would second

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Hannah Jiang
Hi Aizhamal,, yes, Wednesday sounds good to me. Thank you. On Mon, May 4, 2020 at 10:40 AM Aizhamal Nurmamat kyzy wrote: > Hannah, > > We don't have an exact date, but we are hoping to address all the comments > and merge the PR by Wednesday. Will it be possible for you to wait until > then? >

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Aizhamal Nurmamat kyzy
Hannah, We don't have an exact date, but we are hoping to address all the comments and merge the PR by Wednesday. Will it be possible for you to wait until then? On Thu, Apr 30, 2020 at 4:29 PM Hannah Jiang wrote: > Since we want to move forward with the PR, I would like to ask the >>

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Nam Bui
Hey Kenn. Thanks so much for your useful information and research. It's great to know. On Mon, May 4, 2020 at 6:33 PM Kenneth Knowles wrote: > Regarding the detection of renames, I now recall that I have encountered > this before: it is controlled by the config diff.renameLimit. The default >

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Kenneth Knowles
Regarding the detection of renames, I now recall that I have encountered this before: it is controlled by the config diff.renameLimit. The default value these days is high enough to work for this PR. I've confirmed this: git diff --shortstat $(git merge-base github/pr/11554 github/master)

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-04 Thread Nam Bui
Hey guys, How was your weekend? Thanks for some of the compliments and also recommendations. About the commits, as Brian said, we worked together on the-asf slack. It was the tough one, we even did a few experiments. And finally came up with a solution that preserved all commits and used `git

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-01 Thread Brian Hulette
Regarding move detection: I worked with Nam on this some on the-asf slack. We couldn't make squashing into a single large commit work - when I did it, `git log` still showed many dropped and added files. Breaking out a single commit with the file moves was the best we could manage. I tested a PR

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-01 Thread Robert Bradshaw
I just took a look, and added a couple of comments, but it mostly looks good. Thanks for creating a commit that preserves changes; that's a big improvement. +1 to Ahmet's suggestion about braking the huge commit up a bit more. I would suggest one that adds the mechanics (etc.), one that applies a

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-01 Thread Kenneth Knowles
I believe taking Brian and Robert's advice to help git detect moves (even more than you already have) will make this much more manageable. I just tried it out and squashing commits brings it to "631 files changed, 10363 insertions(+), 9945 deletions(-)" according to git, so that is more manageable

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Ahmet Altay
Nam, - Website looks good and looks the same as the current website. (Visually comparing a few pages, not a deep analysis.) - contribute.md looks good. (this is new content.) - website/Dockerfile and website/README.md changes look good. - I do not know what is the new version of some files, for

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Hannah Jiang
> > Since we want to move forward with the PR, I would like to ask the > community to hold off changes to the current Beam website for a week, until > we are able to review and merge the PR. Is this acceptable to everyone? Do we have an exact date when we can push changes to the website? I have

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Nam Bui
Hey guys, I tried my best to handle renamed files in Git. I have no clue why GitHub doesn't show it, but finally, I made this commit [1] (thanks for your idea @bhulette) so you guys can review changes with ease (there is no bunch of deleted markdown files anymore :D). Also, new staged version is

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Brian Hulette
Changing the URLs is fine with me as long as the old urls will work too. But do we need to change the filenames for the blog posts to accomplish that? It's nice that the blog post markdown files start with a date so they naturally sort chronologically. It looks like this hugo PR [1] made it

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Ahmet Altay
On Thu, Apr 30, 2020 at 9:55 AM Thomas Weise wrote: > For changed URLs, will previous URLs be mapped to avoid broken external > links? > I believe the answer is yes from Nam's response "For now, we keep the old URLs working in terms of redirecting them". I very much agree that this is very

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Thomas Weise
For changed URLs, will previous URLs be mapped to avoid broken external links? On Thu, Apr 30, 2020 at 9:34 AM Aizhamal Nurmamat kyzy wrote: > Hi, > > To give a little more context regarding the URLs, the date should still > appear on the blog post, but not on the URL. > For example, we'd

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Aizhamal Nurmamat kyzy
Hi, To give a little more context regarding the URLs, the date should still appear on the blog post, but not on the URL. For example, we'd have: https://beam.apache.org/beam/python/sdk/2016/02/25/python-sdk-now-public.html become https://beam.apache.org/blog/dataflow-python-sdk-is-now-public/.

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-30 Thread Nam Bui
Hi, @altay: Hey hey. Yeah, I didn't expect the baseUrl of staging version is " http://apache-beam-website-pull-requests.storage.googleapis.com/11554/; which also includes "/11554", and Hugo considers it as a path so it breaks the path of "static files" (like images). We made a fix. Now I'm

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Robert Bradshaw
On Wed, Apr 29, 2020 at 5:08 PM Ahmet Altay wrote: > Nam, this looks better. At least links are working, and the website > visually looks similar and generally in good shape. I think there are still > issues. For example, I do not see any of the images (e.g. the beam logo on > top left is

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Ahmet Altay
Nam, this looks better. At least links are working, and the website visually looks similar and generally in good shape. I think there are still issues. For example, I do not see any of the images (e.g. the beam logo on top left is missing.) On Wed, Apr 29, 2020 at 3:11 PM Brian Hulette wrote: >

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Brian Hulette
I left a comment on the PR [1]. I think the reason all of the website content is not being tracked as file renames is because there was a series of commits that created files in the new directory, and then one commit that deleted the old directory. If there were a single commit with all of the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-29 Thread Nam Bui
Hi guys, I'm Nam - from the responsible team of Apache Beam website migration. I am pleased to answer some of the questions here. @aizhamal: Thanks for informing to the community. :) @altay, @robertwb: Yes. there is a problem with the staged version at the moment. We didn't expect some

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-28 Thread Aizhamal Nurmamat kyzy
Adding +Nam Bui and +Karolina Rosół to follow up on questions. On Tue, Apr 28, 2020 at 11:34 AM Ahmet Altay wrote: > I am having trouble reviewing the staged version. What is the best way to > review this change? > > Do we expect any changes to markdown files, beyond some metadata? > > On

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-28 Thread Ahmet Altay
I am having trouble reviewing the staged version. What is the best way to review this change? Do we expect any changes to markdown files, beyond some metadata? On Tue, Apr 28, 2020 at 10:45 AM Robert Bradshaw wrote: > Thanks. It'll be great to better support more languages. > > I looked at the

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-04-28 Thread Robert Bradshaw
Thanks. It'll be great to better support more languages. I looked at the PR and there seems to be no provenance/history. E.g. all the content seems to be entirely new files rather than diffs from the old. (There also seems to be a huge amount of auto-generated js code as well.) On Tue, Apr 28,