rshamunov commented on code in PR #25507: URL: https://github.com/apache/beam/pull/25507#discussion_r1169854206
########## playground/load_your_code.md: ########## @@ -0,0 +1,594 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# How to Add an Example/Snippet/Learning Content into Apache Beam Playground + +Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process: + +1. Prepare a code snippet. +2. Add the code snippet to Apache Beam and/or Playground. +3. Create a link to view the code snippet in Playground or to embed in a website page. + + +Playground sources and output presentation formats: + + + +The guide will walk through all steps. + + +# Table of Contents + +- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet) + * [Data access limitations](#data-access-limitations) + * [Emphasizing parts of code](#emphasizing-parts-of-code) + * [Named Sections](#named-sections) +- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground) + * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog) + + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory) + + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example) + - [Kafka emulator](#kafka-emulator) + + [3. Make a PR](#3-make-a-pr) + + [4. Save the snippet ID](#4-save-the-snippet-id) + * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground) + * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit) + * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone) + * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url) +- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet) + * [Link to a snippet](#link-to-a-snippet) + + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog) + + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-) + + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit) + + [Link to a user-shared snippet](#link-to-a-user-shared-snippet) + + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet) + + [Link to an empty editor](#link-to-an-empty-editor) + * [Embedding a snippet into HTML](#embedding-a-snippet-into-html) + + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog) + + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet) + + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources) + * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website) +- [Snippet view options](#snippet-view-options) + * [Read-only sections](#read-only-sections) + * [Folding everything except named sections](#folding-everything-except-named-sections) + * [Hiding everything except a named section](#hiding-everything-except-a-named-section) + * [Linking to multiple examples](#linking-to-multiple-examples) + + +## Step 1. Prepare a code snippet + +Playground runs example code snippets using Apache Beam Direct Runner +and requires that a code snippet is a complete runnable code. + +### Data access limitations + +For security reasons, the snippets in Playground cannot access arbitrary internet resources. +Depending on your data source you should do the following to access data: + +| Source | Notes | +|----------|-----------------------------------------------------------| +| File | Put to a GCS bucket in `apache-beam-testing` project. | +| BigQuery | The table must be added to `apache-beam-testing` project. | + +Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image). +To request adding, please send an email to dev@... + +CLONING 3RD PARTY REPOS. +IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT + +### Emphasizing parts of code + +Playground provides multiple features to help focus users on certain parts of the code. + +Playground automatically applies the following to all snippets: +- Folds a comment if a snippet starts with one. +- Folds imports. + +### Named Sections + +Playground supports *Named Sections* to tag code blocks and provide the following view options: +- Fold all blocks except tagged code blocks. + This can be useful to help user focus on specific code blocks and features presented in a snippet. +- Hide all code except tagged code blocks. + This can be useful to create runnable snippets illustrating specific concepts or transforms, + and hide all non-essential code blocks. + Such snippet can be embedded on a website to make examples in documentation and tutorials runnable. +- Make certain code parts read-only. + This feature can be useful to create learning units where user modifications are desired + only in certain parts of the code. +Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used. + +If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground). + +*Named Sections* are defined with the following syntax: + +``` +// [START section_name] +void method() { +... +} +// [END section_name] +``` + +Create a named section for each part of your code that you want the above features for. +To learn more details about the syntax please see +the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses. + +## Step 2. Add the code snippet to the Apache Beam repo and/or Playground + +There are several types of code snippets in the Playground: + +1. Public Example — a code snippet displayed in the Playground Examples Catalog. + See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog). +2. Unlisted Example — the same as a public example, but it does not show in the example dropdown + and can only be accessed by direct linking. These are typically embedded on a website. + See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground). +3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit. + See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit). +4. User-shared code snippets do not require a PR and should be used for code + not displayed on Beam resources. + See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone). +5. GitHub or other HTTPS URL sources. + See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url). + +See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources. + +### Source 1. How to add an example to Playground Examples Catalog + +Playground Examples Catalog helps users discover example snippets +and is a recommended way to add examples. Playground automatically scans, +verifies and deploys example snippets from the directories listed below. + +#### 1. Add the file with example to a directory + +Playground Java, Python, and Go examples are automatically picked from these +predefined directories by the `playground_examples_ci.yml` GitHub workflow +after a PR is merged to Beam repo: Review Comment: This yml and the workflow will be removed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
