This is an automated email from the ASF dual-hosted git repository.
dianfu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new d188c2f [hotfix][docs] Update IDE Setup guide
d188c2f is described below
commit d188c2fc7617dfcca2978e3bad34cb4e3085c3f9
Author: Ingo Bürk <[email protected]>
AuthorDate: Thu Aug 19 08:38:47 2021 +0200
[hotfix][docs] Update IDE Setup guide
This closes #16892.
---
docs/content/docs/flinkDev/ide_setup.md | 293 +++++++++++++++++---------------
1 file changed, 156 insertions(+), 137 deletions(-)
diff --git a/docs/content/docs/flinkDev/ide_setup.md
b/docs/content/docs/flinkDev/ide_setup.md
index 644759b..dac1590 100644
--- a/docs/content/docs/flinkDev/ide_setup.md
+++ b/docs/content/docs/flinkDev/ide_setup.md
@@ -49,9 +49,11 @@ e.g.
git clone https://github.com/apache/flink.git
```
-## Ignoring Refactoring Commits
+### Ignoring Refactoring Commits
-We keep a list of big refactoring commits in `.git-blame-ignore-revs`. When
looking at change annotations using `git blame` it's helpful to ignore these.
You can configure git and your IDE to do so using:
+We keep a list of big refactoring commits in `.git-blame-ignore-revs`. When
looking at change
+annotations using `git blame` it's helpful to ignore these. You can configure
git and your IDE to
+do so using:
```bash
git config blame.ignoreRevsFile .git-blame-ignore-revs
@@ -59,104 +61,33 @@ git config blame.ignoreRevsFile .git-blame-ignore-revs
## IntelliJ IDEA
-A brief guide on how to set up IntelliJ IDEA IDE for development of the Flink
core.
-As Eclipse is known to have issues with mixed Scala and Java projects, more
and more contributors are migrating to IntelliJ IDEA.
-
-The following documentation describes the steps to set up IntelliJ IDEA 2020.3
-([https://www.jetbrains.com/idea/download/](https://www.jetbrains.com/idea/download/))
-with the Flink sources.
-
-### Installing the Scala plugin
-
-The IntelliJ installation setup offers to install the Scala plugin.
-If it is not installed, follow these instructions before importing Flink
-to enable support for Scala projects and files:
-
-1. Go to Settings → Plugins and select the "Marketplace" tab.
-2. Select for and install the "Scala" plugin.
-3. Restart IntelliJ if prompted.
+The following guide has been written for [IntelliJ
IDEA](https://www.jetbrains.com/idea/download/)
+2020.3. Some details might differ in other versions. Please make sure to
follow all steps
+accurately.
### Importing Flink
-1. Start IntelliJ and choose New → Project from Existing Sources.
+1. Choose "New" → "Project from Existing Sources".
2. Select the root folder of the cloned Flink repository.
3. Choose "Import project from external model" and select "Maven".
4. Leave the default options and successively click "Next" until you reach the
SDK section.
5. If there is no SDK listed, create one using the "+" sign on the top left.
Select "JDK", choose the JDK home directory and click "OK".
- Select the most suiting JDK version. NOTE: A good rule of thumb is to
select
+ Select the most suitable JDK version. NOTE: A good rule of thumb is to
select
the JDK version matching the active Maven profile.
-6. Continue by clicking "Next" until finishing the import.
-7. Right-click on the imported Flink project → Maven → Generate Sources and
Update Folders.
- Note that this will install Flink libraries in your local Maven repository,
- located by default at "/home/$USER/.m2/repository/org/apache/flink/".
- Alternatively, `mvn clean package -DskipTests` also creates the files
necessary
- for the IDE to work but without installing the libraries.
-8. Build the Project (Build → Make Project).
-
-### Code Formatting
-
-We use the [Spotless
-plugin](https://github.com/diffplug/spotless/tree/main/plugin-maven) together
-with [google-java-format](https://github.com/google/google-java-format) to
-format our Java code.
-
-You can configure your IDE to automatically apply formatting on saving with
these steps:
-
-1. Download the [google-java-format
- plugin
v1.7.0.6](https://plugins.jetbrains.com/plugin/8527-google-java-format/versions/stable/115957).
-2. Open Settings → Plugins, click on the gear icon and select "Install Plugin
from Disk". Navigate to the downloaded zip file and select it.
-3. In the plugin settings, enable the plugin and change the code style to
"AOSP" (4-space indents).
-4. Remember to never update this plugin to a later version!
-5. Install the [Save Actions
- plugin](https://plugins.jetbrains.com/plugin/7642-save-actions).
-6. Enable the plugin, along with "Optimize imports" and "Reformat file".
-7. In the "Save Actions" settings page, set up a "File Path Inclusion" for
`.*\.java`. Otherwise, you will get unintended reformatting in other files you
edit.
-
-### Checkstyle For Java
-
-IntelliJ supports checkstyle within the IDE using the Checkstyle-IDEA plugin.
-
-1. Install the "Checkstyle-IDEA" plugin from the IntelliJ plugin repository.
-2. Configure the plugin by going to Settings → Tools → Checkstyle.
-3. Set the "Scan Scope" to "Only Java sources (including tests)".
-4. Select _8.14_ in the "Checkstyle Version" dropdown and click apply. **This
step is important,
- don't skip it!**
-5. In the "Configuration File" pane, add a new configuration using the plus
icon:
- 1. Set the "Description" to "Flink".
- 2. Select "Use a local Checkstyle file", and point it to
- `"tools/maven/checkstyle.xml"` within
- your repository.
- 3. Check the box for "Store relative to project location", and click
- "Next".
- 4. Configure the "checkstyle.suppressions.file" property value to
- `"suppressions.xml"`, and click "Next", then "Finish".
-6. Select "Flink" as the only active configuration file, and click "Apply" and
- "OK".
-7. Checkstyle will now give warnings in the editor for any Checkstyle
- violations.
-
-Once the plugin is installed you can directly import
`"tools/maven/checkstyle.xml"` by going to Settings → Editor → Code Style →
Java → Gear Icon next to Scheme dropbox. This will for example automatically
adjust the imports layout.
-
-You can scan an entire module by opening the Checkstyle tools window and
-clicking the "Check Module" button. The scan should report no errors.
-
-{{< hint info >}}
-Some modules are not fully covered by checkstyle,
-which include `flink-core`, `flink-optimizer`, and `flink-runtime`.
-Nevertheless, please make sure that code you add/modify in these modules still
conforms to the checkstyle rules.
-{{< /hint >}}
-
-### Checkstyle For Scala
-
-Enable scalastyle in IntelliJ by selecting Settings → Editor → Inspections,
then searching for "Scala style inspections". Also place
`"tools/maven/scalastyle-config.xml"` in the `"<root>/.idea"` or
`"<root>/project"` directory.
+6. Continue by clicking "Next" until the import is finished.
+7. Open the "Maven" tab (or right-click on the imported project and find
"Maven") and run
+ "Generate Sources and Update Folders". Alternatively, you can run
+ `mvn clean package -DskipTests`.
+8. Build the Project ("Build" → "Build Project").
### Copyright Profile
-Each file needs to include the Apache license as a header. This can be
automated in IntelliJ by adding a Copyright profile:
+Every file needs to include the Apache license as a header. This can be
automated in IntelliJ by
+adding a Copyright profile:
-1. Open Settings → Editor → Copyright → Copyright Profiles.
-2. Add a new profile naming it `Apache`.
+1. Go to "Settings" → "Editor" → "Copyright" → "Copyright Profiles".
+2. Add a new profile and name it "Apache".
3. Add the following text as the license text:
```
@@ -176,76 +107,164 @@ Each file needs to include the Apache license as a
header. This can be automated
See the License for the specific language governing permissions and
limitations under the License.
```
-4. Go to Editor → Copyright and choose the `Apache` profile as the default
profile for this project.
-5. Apply the changes.
+4. Go to "Editor" → "Copyright" and choose the "Apache" profile as the default
profile for this
+ project.
+5. Click "Apply".
-### FAQ
+### Required Plugins
-This section lists issues that developers have run into in the past when
working with IntelliJ:
+Go to "Settings" → "Plugins" and select the "Marketplace" tab. Search for the
following plugins,
+install them, and restart the IDE if prompted:
-- Compilation fails with `invalid flag:
--add-exports=java.base/sun.net.util=ALL-UNNAMED`
+* [Scala](https://plugins.jetbrains.com/plugin/1347-scala)
+* [Python](https://plugins.jetbrains.com/plugin/631-python) – Required for
PyFlink. If you do not
+ intend to work on PyFlink, you can skip this.
+* [Save Actions](https://plugins.jetbrains.com/plugin/7642-save-actions)
+* [Checkstyle-IDEA](https://plugins.jetbrains.com/plugin/1065-checkstyle-idea)
-This means that IntelliJ activated the `java11` profile despite an older JDK
being used.
-Open the Maven tool window (View → Tool Windows → Maven), uncheck the `java11`
profile and reimport the project.
+You will also need to install the
[google-java-format](https://github.com/google/google-java-format)
+plugin. However, a specific version of this plugin is required. Download
+[google-java-format
v1.7.0.6](https://plugins.jetbrains.com/plugin/8527-google-java-format/versions/stable/115957)
+and install it as follows. Make sure to never update this plugin.
-- Compilation fails with `cannot find symbol: symbol: method defineClass(...)
location: class sun.misc.Unsafe`
+1. Go to "Settings" → "Plugins".
+2. Click the gear icon and select "Install Plugin from Disk".
+3. Navigate to the downloaded ZIP file and select it.
-This means that IntelliJ is using JDK 11 for the project, but you are working
on a Flink version which doesn't
-support Java 11.
-This commonly happens when you have set up IntelliJ to use JDK 11 and checkout
older versions of Flink (<= 1.9).
-Open the project settings window (File → Project Structure → Project Settings:
Project) and select JDK 8 as the project
-SDK.
-You may have to revert this after switching back to the new Flink version if
you want to use JDK 11.
+#### Code Formatting
-- Examples fail with a `NoClassDefFoundError` for Flink classes.
+Flink uses
[Spotless](https://github.com/diffplug/spotless/tree/main/plugin-maven)
together with
+[google-java-format](https://github.com/google/google-java-format) to format
the Java code.
-This is likely due to Flink dependencies being set to provided, resulting in
them not being put automatically on the
-classpath.
-You can either tick the "Include dependencies with 'Provided' scope" box in
the run configuration, or create a test
-that calls the `main()` method of the example (`provided` dependencies are
available on the test classpath).
+It is recommended to automatically format your code by applying the following
settings:
-## Eclipse
+1. Go to "Settings" → "Other Settings" → "google-java-format Settings".
+2. Tick the checkbox to enable the plugin.
+3. Change the code style to "Android Open Source Project (AOSP) style".
+4. Go to "Settings" → "Other Settings" → "Save Actions".
+5. Under "General", enable your preferred settings for when to format the
code, e.g.
+ "Activate save actions on save".
+6. Under "Formatting Actions", select "Optimize imports" and "Reformat file".
+7. Under "File Path Inclusions", add an entry for `.*\.java` to avoid
formatting other file types.
+
+#### Checkstyle For Java
-{{< hint warning >}}
-**NOTE:** From our experience, this setup does not work with Flink
-due to deficiencies of the old Eclipse version bundled with Scala IDE 3.0.3 or
-due to version incompatibilities with the bundled Scala version in Scala IDE
4.4.1.
+[Checkstyle](https://checkstyle.sourceforge.io/) is used to enforce static
coding guidelines.
+
+{{< hint info >}}
+Some modules are not covered by Checkstyle, e.g. flink-core, flink-optimizer,
and flink-runtime.
+Nevertheless, please make sure to conform to the checkstyle rules in these
modules if you work in
+any of these modules.
{{< /hint >}}
-**We recommend to use IntelliJ instead (see [above](#intellij-idea))**
+1. Go to "Settings" → "Tools" → "Checkstyle".
+2. Set "Scan Scope" to "Only Java sources (including tests)".
+3. For "Checkstyle Version" select "8.14".
+4. Under "Configuration File" click the "+" icon to add a new configuration.
+5. Set "Description" to "Flink".
+6. Select "Use a local Checkstyle file" and point it to
`tools/maven/checkstyle.xml` located within
+ your cloned repository.
+7. Select "Store relative to project location" and click "Next".
+8. Configure the property `checkstyle.suppressions.file` with the value
`suppressions.xml` and click
+ "Next".
+9. Click "Finish".
+10. Select "Flink" as the only active configuration file and click "Apply".
-## PyCharm
+You can now import the Checkstyle configuration for the Java code formatter.
-A brief guide on how to set up PyCharm for development of the flink-python
module.
+1. Go to "Settings" → "Editor" → "Code Style" → "Java".
+2. Click the gear icon next to "Scheme" and select "Import Scheme" →
"Checkstyle Configuration".
+3. Navigate to and select `tools/maven/checkstyle.xml` located within your
cloned repository.
-The following documentation describes the steps to setup PyCharm 2019.1.3
-([https://www.jetbrains.com/pycharm/download/](https://www.jetbrains.com/pycharm/download/))
-with the Flink Python sources.
+To verify the setup, click "View" → "Tool Windows" → "Checkstyle" and find the
"Check Module"
+button in the opened tool window. It should report no violations.
-### Importing flink-python
-If you are in the PyCharm startup interface:
+#### Checkstyle For Scala
+
+Enable [Scalastyle](http://www.scalastyle.org/) as follows:
+
+1. Go to "Settings" → "Editor" → "Inspections".
+2. Search for "Scala style inspection" and enable it.
+
+Now copy the file `tools/maven/scalastyle-config.xml` into the `.idea/` or
`project/` folder of your
+cloned repository.
+
+#### Python for PyFlink
+
+Working on the flink-python module requires both a Java SDK and a Python SDK.
However, IntelliJ IDEA
+only supports one configured SDK per module. If you intend to work actively on
PyFlink, it is
+recommended to import the flink-python module as a separate project either in
[PyCharm](#pycharm)
+or IntelliJ IDEA for working with Python.
+
+If you only occasionally need to work on flink-python and would like to get
Python to work in
+IntelliJ IDEA, e.g. to run Python tests, you can use the following guide.
-1. Start PyCharm and choose "Open".
-2. Select the flink-python folder in the cloned Flink repository-
+1. Follow [Configure a virtual
environment](https://www.jetbrains.com/help/idea/creating-virtual-environment.html)
+ to create a new Virtualenv Python SDK in your Flink project.
+2. Find the flink-python module in the Project Explorer, right-click on it and
choose
+ "Open Module Settings". Alternatively, go to "Project Structure" →
"Modules" and find the module
+ there.
+3. Change "Module SDK" to the Virtualenv Python SDK you created earlier.
+4. Open the file `flink-python/setup.py` and install the dependencies when
IntelliJ prompts you to
+ do so.
-If you have used PyCharm to open a project:
+You can verify your setup by running some of the Python tests located in
flink-python.
-1. Select File → Open.
-2. Select the flink-python folder in the cloned Flink repository.
+### Common Problems
+This section lists issues that developers have run into in the past when
working with IntelliJ.
+
+#### Compilation fails with `invalid flag:
--add-exports=java.base/sun.net.util=ALL-UNNAMED`
+
+This happens if the "java11" Maven profile is active, but an older JDK version
is used. Go to
+"View" → "Tool Windows" → "Maven" and uncheck the "java11" profile.
Afterwards, reimport the
+project.
+
+#### Compilation fails with `cannot find symbol: symbol: method
defineClass(...) location: class sun.misc.Unsafe`
+
+This happens if you are using JDK 11, but are working on a Flink version which
doesn't yet support
+Java 11 (<= 1.9). Go to "Project Structure" → "Project Settings" → "Project"
and select JDK 8 as
+the Project SDK.
+
+When switching back to newer Flink versions you may have to revert this change
again.
+
+#### Examples fail with a `NoClassDefFoundError` for Flink classes.
+
+This happens if Flink dependencies are set to "provided", resulting in them
not being available
+on the classpath. You can either check "Include dependencies with 'Provided'
scope" in your
+run configuration, or create a test that calls the `main()` method of the
example.
+
+## Eclipse
+
+Using Eclipse with Flink is currently not supported and discouraged. Please use
+[IntelliJ IDEA](#intellij-idea) instead.
+
+## PyCharm
+
+If you intend to work on PyFlink, it is recommended to use
+[PyCharm](https://www.jetbrains.com/pycharm/download/) as a separate IDE for
the flink-python
+module. The following guide has been written for 2019.1.3. Some details might
differ in other
+versions.
+
+### Importing flink-python
+
+1. Open the PyCharm IDE and choose ("File" →) "Open".
+2. Select the "flink-python" folder within your located repository.
### Checkstyle For Python
-The Python code checkstyle of Apache Flink should create a flake8 external
tool in the project.
-
-1. Install the flake8 in the used Python interpreter(refer to
([https://pypi.org/project/flake8/](https://pypi.org/project/flake8/)).
-2. Select "PyCharm → Preferences... → Tools → External Tools → + (The bottom
left corner of the page on the right)", next configure the external tool.
-3. Set the "Name" to "flake8".
-4. Set the "Description" to "code style check".
-5. Set the "Program" to the Python interpreter path(e.g. /usr/bin/python).
-6. Set the "Arguments" to "-m flake8 \-\-config=tox.ini"
-7. Set the "Working directory" to "$ProjectFileDir$"
-
-Now, you can check your Python code style by the operation:
- "Right click in any file or folder in the flink-python project → External
Tools → flake8"
-
+
+[Flake8](https://pypi.org/project/flake8/) is used to enforce some coding
guidelines.
+
+1. Install flake8 for your Python interpreter using `pip install flake8`.
+2. In PyCharm go to "Preferences" → "Tools" → "External Tools".
+3. Select the "+" button to add a new external tool.
+4. Set "Name" to "flake8".
+5. Set "Description" to "Code Style Check".
+6. Set "Program" to the path of your Python interpreter, e.g.
`/usr/bin/python`.
+7. Set "Arguments" to `-m flake8 --config=tox.ini`.
+8. Set "Working Directory" to `$ProjectFileDir$`.
+
+You can verify the setup by right-clicking on any file or folder in the
flink-python project
+and running "External Tools" → "flake8".
+
{{< top >}}
