This is an automated email from the ASF dual-hosted git repository. zhuzh pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git
commit e9bea09510e18c6143e6e14ca17a894abfaf92bf Author: JunRuiLee <[email protected]> AuthorDate: Sun Feb 4 16:47:52 2024 +0800 [FLINK-34247][doc] Update the usage of flink-conf.yaml in doc. This closes #24251. --- .../content.zh/docs/connectors/table/filesystem.md | 4 +-- .../docs/deployment/advanced/historyserver.md | 2 +- docs/content.zh/docs/deployment/cli.md | 6 ++-- docs/content.zh/docs/deployment/config.md | 2 +- .../docs/deployment/filesystems/azure.md | 6 ++-- docs/content.zh/docs/deployment/filesystems/gcs.md | 6 ++-- docs/content.zh/docs/deployment/filesystems/oss.md | 6 ++-- docs/content.zh/docs/deployment/filesystems/s3.md | 12 +++---- .../content.zh/docs/deployment/ha/kubernetes_ha.md | 2 +- docs/content.zh/docs/deployment/ha/zookeeper_ha.md | 4 +-- .../docs/deployment/memory/mem_migration.md | 8 ++--- .../content.zh/docs/deployment/metric_reporters.md | 2 +- .../resource-providers/native_kubernetes.md | 4 +-- .../resource-providers/standalone/docker.md | 14 ++++---- .../resource-providers/standalone/kubernetes.md | 32 +++++++++--------- .../docs/deployment/resource-providers/yarn.md | 2 +- .../docs/deployment/security/security-ssl.md | 2 +- docs/content.zh/docs/deployment/trace_reporters.md | 2 +- .../docs/dev/datastream/execution/parallel.md | 2 +- .../datastream/fault-tolerance/checkpointing.md | 2 +- .../datastream/fault-tolerance/state_backends.md | 2 +- .../docs/dev/python/environment_variables.md | 2 +- docs/content.zh/docs/dev/table/catalogs.md | 4 +-- .../dev/table/hive-compatibility/hiveserver2.md | 2 +- .../docs/dev/table/sql-gateway/overview.md | 4 +-- docs/content.zh/docs/dev/table/sqlClient.md | 2 +- docs/content.zh/docs/ops/debugging/flame_graphs.md | 2 +- docs/content.zh/docs/ops/metrics.md | 4 +-- docs/content.zh/docs/ops/rest_api.md | 2 +- docs/content.zh/docs/ops/state/state_backends.md | 22 ++++++------- .../docs/ops/state/task_failure_recovery.md | 8 ++--- .../docs/try-flink/flink-operations-playground.md | 4 +-- docs/content/docs/connectors/table/filesystem.md | 4 +-- .../docs/deployment/advanced/historyserver.md | 2 +- docs/content/docs/deployment/cli.md | 6 ++-- docs/content/docs/deployment/config.md | 2 +- docs/content/docs/deployment/filesystems/azure.md | 6 ++-- docs/content/docs/deployment/filesystems/gcs.md | 6 ++-- docs/content/docs/deployment/filesystems/oss.md | 6 ++-- docs/content/docs/deployment/filesystems/s3.md | 12 +++---- docs/content/docs/deployment/ha/kubernetes_ha.md | 2 +- docs/content/docs/deployment/ha/zookeeper_ha.md | 4 +-- .../docs/deployment/memory/mem_migration.md | 12 +++---- docs/content/docs/deployment/metric_reporters.md | 2 +- .../resource-providers/native_kubernetes.md | 4 +-- .../resource-providers/standalone/docker.md | 14 ++++---- .../resource-providers/standalone/kubernetes.md | 38 +++++++++++----------- .../resource-providers/standalone/overview.md | 4 +-- .../docs/deployment/resource-providers/yarn.md | 2 +- .../docs/deployment/security/security-ssl.md | 2 +- docs/content/docs/deployment/trace_reporters.md | 2 +- .../docs/dev/datastream/execution/parallel.md | 2 +- .../datastream/fault-tolerance/checkpointing.md | 2 +- .../datastream/fault-tolerance/state_backends.md | 2 +- .../docs/dev/python/environment_variables.md | 2 +- docs/content/docs/dev/table/catalogs.md | 4 +-- docs/content/docs/dev/table/config.md | 4 +-- .../docs/dev/table/sql-gateway/hiveserver2.md | 2 +- .../content/docs/dev/table/sql-gateway/overview.md | 4 +-- docs/content/docs/dev/table/sqlClient.md | 2 +- docs/content/docs/ops/debugging/flame_graphs.md | 2 +- docs/content/docs/ops/metrics.md | 4 +-- docs/content/docs/ops/rest_api.md | 2 +- docs/content/docs/ops/state/savepoints.md | 2 +- docs/content/docs/ops/state/state_backends.md | 24 +++++++------- .../docs/ops/state/task_failure_recovery.md | 8 ++--- .../docs/try-flink/flink-operations-playground.md | 2 +- docs/content/docs/try-flink/local_installation.md | 2 +- 68 files changed, 189 insertions(+), 189 deletions(-) diff --git a/docs/content.zh/docs/connectors/table/filesystem.md b/docs/content.zh/docs/connectors/table/filesystem.md index b28a41d9467..17d05045a20 100644 --- a/docs/content.zh/docs/connectors/table/filesystem.md +++ b/docs/content.zh/docs/connectors/table/filesystem.md @@ -241,8 +241,8 @@ CREATE TABLE MyUserTableWithFilepath ( **注意：** 对于 bulk formats 数据 (parquet、orc、avro)，滚动策略与 checkpoint 间隔（pending 状态的文件会在下个 checkpoint 完成）控制了 part 文件的大小和个数。 -**注意：** 对于 row formats 数据 (csv、json)，如果想使得分区文件更快在文件系统中可见，可以设置 `sink.rolling-policy.file-size` 或 `sink.rolling-policy.rollover-interval` 属性以及在 flink-conf.yaml 中的 `execution.checkpointing.interval` 属性。 -对于其他 formats (avro、orc)，可以只设置 flink-conf.yaml 中的 `execution.checkpointing.interval` 属性。 +**注意：** 对于 row formats 数据 (csv、json)，如果想使得分区文件更快在文件系统中可见，可以设置 `sink.rolling-policy.file-size` 或 `sink.rolling-policy.rollover-interval` 属性以及在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中的 `execution.checkpointing.interval` 属性。 +对于其他 formats (avro、orc)，可以只设置 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中的 `execution.checkpointing.interval` 属性。 <a name="file-compaction"></a> diff --git a/docs/content.zh/docs/deployment/advanced/historyserver.md b/docs/content.zh/docs/deployment/advanced/historyserver.md index 0d5d05e078e..16950b4a06b 100644 --- a/docs/content.zh/docs/deployment/advanced/historyserver.md +++ b/docs/content.zh/docs/deployment/advanced/historyserver.md @@ -58,7 +58,7 @@ bin/historyserver.sh (start|start-foreground|stop) **JobManager** -已完成作业的存档在 JobManager 上进行，将已存档的作业信息上传到文件系统目录中。你可以在 `flink-conf.yaml` 文件中通过 `jobmanager.archive.fs.dir` 设置一个目录存档已完成的作业。 +已完成作业的存档在 JobManager 上进行，将已存档的作业信息上传到文件系统目录中。你可以在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中通过 `jobmanager.archive.fs.dir` 设置一个目录存档已完成的作业。 ```yaml # 上传已完成作业信息的目录 diff --git a/docs/content.zh/docs/deployment/cli.md b/docs/content.zh/docs/deployment/cli.md index d71dbdd4e2c..64196be5da1 100644 --- a/docs/content.zh/docs/deployment/cli.md +++ b/docs/content.zh/docs/deployment/cli.md @@ -30,7 +30,7 @@ under the License. Flink provides a Command-Line Interface (CLI) `bin/flink` to run programs that are packaged as JAR files and to control their execution. The CLI is part of any Flink setup, available in local single node setups and in distributed setups. -It connects to the running JobManager specified in `conf/flink-conf.yaml`. +It connects to the running JobManager specified in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). @@ -366,7 +366,7 @@ Here's an overview of actions supported by Flink's CLI tool: This action can be used to create or disposing savepoints for a given job. It might be necessary to specify a savepoint directory besides the JobID, if the <a href="{{< ref "docs/deployment/config" >}}#state-savepoints-dir">state.savepoints.dir</a> - parameter was not specified in <code class="highlighter-rouge">conf/flink-conf.yaml</code>. + parameter was not specified in <code class="highlighter-rouge">Flink configuration file</code>. </td> </tr> <tr> @@ -431,7 +431,7 @@ parameter combinations: * `./bin/flink run --target remote`: Submission to an already running Flink cluster The `--target` will overwrite the [execution.target]({{< ref "docs/deployment/config" >}}#execution-target) -specified in the `conf/flink-conf.yaml`. +specified in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). For more details on the commands and the available options, please refer to the Resource Provider-specific pages of the documentation. diff --git a/docs/content.zh/docs/deployment/config.md b/docs/content.zh/docs/deployment/config.md index d3119d4cf8f..857d8ec497e 100644 --- a/docs/content.zh/docs/deployment/config.md +++ b/docs/content.zh/docs/deployment/config.md @@ -31,7 +31,7 @@ All configuration can be set in Flink configuration file in the `conf/` director The configuration is parsed and evaluated when the Flink processes are started. Changes to the configuration file require restarting the relevant processes. -The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in Flink configuration file if you want to manually override the Java runtime to use. +The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in Flink configuration file if you want to manually override the Java runtime to use. Note that the configuration key `env.java.home` must be specified in a flattened format (i.e. one-line key-value format) in the configuration file. You can specify a different configuration directory location by defining the `FLINK_CONF_DIR` environment variable. For resource providers which provide non-session deployments, you can specify per-job configurations this way. Make a copy of the `conf` directory from the Flink distribution and modify the settings on a per-job basis. Note that this is not supported in Docker or standalone Kubernetes deployments. On Docker-based deployments, you can use the `FLINK_PROPERTIES` environment v [...] diff --git a/docs/content.zh/docs/deployment/filesystems/azure.md b/docs/content.zh/docs/deployment/filesystems/azure.md index 1482a302dac..76b34fb6a3d 100644 --- a/docs/content.zh/docs/deployment/filesystems/azure.md +++ b/docs/content.zh/docs/deployment/filesystems/azure.md @@ -86,13 +86,13 @@ cp ./opt/flink-azure-fs-hadoop-{{< version >}}.jar ./plugins/azure-fs-hadoop/ ### WASB Hadoop 的 WASB Azure 文件系统支持通过 Hadoop 配置来配置凭据，如 [Hadoop Azure Blob Storage 文档](https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Configuring_Credentials) 所述。 -为方便起见，Flink 将所有的 Flink 配置添加 `fs.azure` 键前缀后转发至文件系统的 Hadoop 配置中。因此，可通过以下方法在 `flink-conf.yaml` 中配置 Azure Blob 存储密钥： +为方便起见，Flink 将所有的 Flink 配置添加 `fs.azure` 键前缀后转发至文件系统的 Hadoop 配置中。因此，可通过以下方法在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中配置 Azure Blob 存储密钥： ```yaml fs.azure.account.key.<account_name>.blob.core.windows.net: <azure_storage_key> ``` -或者通过在 `flink-conf.yaml` 中设置以下配置键，将文件系统配置为从环境变量 `AZURE_STORAGE_KEY` 读取 Azure Blob 存储密钥： +或者通过在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中设置以下配置键，将文件系统配置为从环境变量 `AZURE_STORAGE_KEY` 读取 Azure Blob 存储密钥： ```yaml fs.azure.account.keyprovider.<account_name>.blob.core.windows.net: org.apache.flink.fs.azurefs.EnvironmentVariableKeyProvider @@ -107,7 +107,7 @@ Azure 推荐使用 Azure 托管身份来使用 abfs 访问 ADLS Gen2 存储帐 {{< /hint >}} ##### 使用存储密钥访问ABFS(不鼓励) -Azure blob 存储密钥可以通过以下方式在 `flink-conf.yaml` 中配置： +Azure blob 存储密钥可以通过以下方式在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中配置： ```yaml fs.azure.account.key.<account_name>.dfs.core.windows.net: <azure_storage_key> diff --git a/docs/content.zh/docs/deployment/filesystems/gcs.md b/docs/content.zh/docs/deployment/filesystems/gcs.md index 47cd137db33..ab8cc6a83de 100644 --- a/docs/content.zh/docs/deployment/filesystems/gcs.md +++ b/docs/content.zh/docs/deployment/filesystems/gcs.md @@ -71,13 +71,13 @@ cp ./opt/flink-gs-fs-hadoop-{{< version >}}.jar ./plugins/gs-fs-hadoop/ ### Configuration -The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.18/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your `flink-conf.yaml`. +The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.18/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). -For example, `gcs-connector` has a `fs.gs.http.connect-timeout` configuration key. If you want to change it, you need to set `gs.http.connect-timeout: xyz` in `flink-conf.yaml`. Flink will internally translate this back to `fs.gs.http.connect-timeout`. +For example, `gcs-connector` has a `fs.gs.http.connect-timeout` configuration key. If you want to change it, you need to set `gs.http.connect-timeout: xyz` in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). Flink will internally translate this back to `fs.gs.http.connect-timeout`. You can also set `gcs-connector` options directly in the Hadoop `core-site.xml` configuration file, so long as the Hadoop configuration directory is made known to Flink via the `env.hadoop.conf.dir` Flink option or via the `HADOOP_CONF_DIR` environment variable. -`flink-gs-fs-hadoop` can also be configured by setting the following options in `flink-conf.yaml`: +`flink-gs-fs-hadoop` can also be configured by setting the following options in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): | Key | Description [...] |---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...] diff --git a/docs/content.zh/docs/deployment/filesystems/oss.md b/docs/content.zh/docs/deployment/filesystems/oss.md index 5a210ad9954..849859fb148 100644 --- a/docs/content.zh/docs/deployment/filesystems/oss.md +++ b/docs/content.zh/docs/deployment/filesystems/oss.md @@ -70,11 +70,11 @@ cp ./opt/flink-oss-fs-hadoop-{{< version >}}.jar ./plugins/oss-fs-hadoop/ 在设置好 OSS 文件系统包装器之后，需要添加一些配置以保证 Flink 有权限访问 OSS buckets。 -为了简单使用，可直接在 `flink-conf.yaml` 中使用与 Hadoop `core-site.xml` 相同的配置关键字。 +为了简单使用，可直接在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中使用与 Hadoop `core-site.xml` 相同的配置关键字。 可在 [Hadoop OSS 文档](http://hadoop.apache.org/docs/current/hadoop-aliyun/tools/hadoop-aliyun/index.html) 中查看配置关键字。 -一些配置必须添加至 `flink-conf.yaml` （**在 Hadoop OSS 文档中定义的其它配置为用作性能调优的高级配置**）： +一些配置必须添加至 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) （**在 Hadoop OSS 文档中定义的其它配置为用作性能调优的高级配置**）： ```yaml fs.oss.endpoint: 连接的 Aliyun OSS endpoint @@ -82,7 +82,7 @@ fs.oss.accessKeyId: Aliyun access key ID fs.oss.accessKeySecret: Aliyun access key secret ``` -备选的 `CredentialsProvider` 也可在 `flink-conf.yaml` 中配置，例如： +备选的 `CredentialsProvider` 也可在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中配置，例如： ```yaml # 从 OSS_ACCESS_KEY_ID 和 OSS_ACCESS_KEY_SECRET 读取凭据 (Credentials) fs.oss.credentials.provider: com.aliyun.oss.common.auth.EnvironmentVariableCredentialsProvider diff --git a/docs/content.zh/docs/deployment/filesystems/s3.md b/docs/content.zh/docs/deployment/filesystems/s3.md index 5f9a895c786..7605fb54338 100644 --- a/docs/content.zh/docs/deployment/filesystems/s3.md +++ b/docs/content.zh/docs/deployment/filesystems/s3.md @@ -66,12 +66,12 @@ env.configure(config); Flink 提供两种文件系统用来与 S3 交互：`flink-s3-fs-presto` 和 `flink-s3-fs-hadoop`。两种实现都是独立的且没有依赖项，因此使用时无需将 Hadoop 添加至 classpath。 - `flink-s3-fs-presto`，通过 *s3://* 和 *s3p://* 两种 scheme 使用，基于 [Presto project](https://prestodb.io/)。 - 可以使用[和 Presto 文件系统相同的配置项](https://prestodb.io/docs/0.272/connector/hive.html#amazon-s3-configuration)进行配置，方式为将配置添加到 `flink-conf.yaml` 文件中。如果要在 S3 中使用 checkpoint，推荐使用 Presto S3 文件系统。 + 可以使用[和 Presto 文件系统相同的配置项](https://prestodb.io/docs/0.272/connector/hive.html#amazon-s3-configuration)进行配置，方式为将配置添加到 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中。如果要在 S3 中使用 checkpoint，推荐使用 Presto S3 文件系统。 - `flink-s3-fs-hadoop`，通过 *s3://* 和 *s3a://* 两种 scheme 使用, 基于 [Hadoop Project](https://hadoop.apache.org/)。 - 本文件系统可以使用类似 [Hadoop S3A 的配置项](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A)进行配置，方式为将配置添加到 `flink-conf.yaml` 文件中。 + 本文件系统可以使用类似 [Hadoop S3A 的配置项](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A)进行配置，方式为将配置添加到 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中。 - 例如，Hadoop 有 `fs.s3a.connection.maximum` 的配置选项。 如果你想在 Flink 程序中改变该配置的值，你需要将配置 `s3.connection.maximum: xyz` 添加到 `flink-conf.yaml` 文件中。Flink 会内部将其转换成配置 `fs.s3a.connection.maximum`。 而无需通过 Hadoop 的 XML 配置文件来传递参数。 + 例如，Hadoop 有 `fs.s3a.connection.maximum` 的配置选项。 如果你想在 Flink 程序中改变该配置的值，你需要将配置 `s3.connection.maximum: xyz` 添加到 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中。Flink 会内部将其转换成配置 `fs.s3a.connection.maximum`。 而无需通过 Hadoop 的 XML 配置文件来传递参数。 另外，它是唯一支持 [FileSystem]({{< ref "docs/connectors/datastream/filesystem" >}}) 的 S3 文件系统。 @@ -99,7 +99,7 @@ cp ./opt/flink-s3-fs-presto-{{< version >}}.jar ./plugins/s3-fs-presto/ 可以通过**访问密钥对（access and secret key）**授予 S3 访问权限。请注意，根据 [Introduction of IAM roles](https://blogs.aws.amazon.com/security/post/Tx1XG3FX6VMU6O5/A-safer-way-to-distribute-AWS-credentials-to-EC2)，不推荐使用该方法。 - `s3.access-key` 和 `s3.secret-key` 均需要在 Flink 的 `flink-conf.yaml` 中进行配置： + `s3.access-key` 和 `s3.secret-key` 均需要在 Flink 的 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中进行配置： ```yaml s3.access-key: your-access-key @@ -108,7 +108,7 @@ s3.secret-key: your-secret-key ## 配置非 S3 访问点 -S3 文件系统还支持兼容 S3 的对象存储服务，如 [IBM's Cloud Object Storage](https://www.ibm.com/cloud/object-storage) 和 [Minio](https://min.io/)。可在 `flink-conf.yaml` 中配置使用的访问点： +S3 文件系统还支持兼容 S3 的对象存储服务，如 [IBM's Cloud Object Storage](https://www.ibm.com/cloud/object-storage) 和 [Minio](https://min.io/)。可在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中配置使用的访问点： ```yaml s3.endpoint: your-endpoint-hostname @@ -116,7 +116,7 @@ s3.endpoint: your-endpoint-hostname ## 配置路径样式的访问 -某些兼容 S3 的对象存储服务可能没有默认启用虚拟主机样式的寻址。这种情况下需要在 `flink-conf.yaml` 中添加配置以启用路径样式的访问： +某些兼容 S3 的对象存储服务可能没有默认启用虚拟主机样式的寻址。这种情况下需要在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中添加配置以启用路径样式的访问： ```yaml s3.path.style.access: true diff --git a/docs/content.zh/docs/deployment/ha/kubernetes_ha.md b/docs/content.zh/docs/deployment/ha/kubernetes_ha.md index 41ce67e287d..40da006218c 100644 --- a/docs/content.zh/docs/deployment/ha/kubernetes_ha.md +++ b/docs/content.zh/docs/deployment/ha/kubernetes_ha.md @@ -67,7 +67,7 @@ kubernetes.cluster-id: cluster1337 ### 配置示例 -在 `conf/flink-conf.yaml` 中配置高可用模式: +在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中配置高可用模式: ```yaml kubernetes.cluster-id: <cluster-id> diff --git a/docs/content.zh/docs/deployment/ha/zookeeper_ha.md b/docs/content.zh/docs/deployment/ha/zookeeper_ha.md index 2a26f5ae2f8..705da25acbd 100644 --- a/docs/content.zh/docs/deployment/ha/zookeeper_ha.md +++ b/docs/content.zh/docs/deployment/ha/zookeeper_ha.md @@ -68,7 +68,7 @@ Flink 利用 **[ZooKeeper](http://zookeeper.apache.org)** 在所有运行的 Job ### 配置示例 -在 `conf/flink-conf.yaml` 中配置高可用模式和 ZooKeeper 复制组（quorum）: +在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中配置高可用模式和 ZooKeeper 复制组（quorum）: ```bash high-availability.type: zookeeper @@ -82,7 +82,7 @@ high-availability.storageDir: hdfs:///flink/recovery ## ZooKeeper 安全配置 -如果 ZooKeeper 使用 Kerberos 以安全模式运行，必要时可以在 `flink-conf.yaml` 中覆盖以下配置: +如果 ZooKeeper 使用 Kerberos 以安全模式运行，必要时可以在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中覆盖以下配置: ```bash # 默认配置为 "zookeeper". 如果 ZooKeeper quorum 配置了不同的服务名称， diff --git a/docs/content.zh/docs/deployment/memory/mem_migration.md b/docs/content.zh/docs/deployment/memory/mem_migration.md index 9f5ee5ec06f..c1a7059475f 100644 --- a/docs/content.zh/docs/deployment/memory/mem_migration.md +++ b/docs/content.zh/docs/deployment/memory/mem_migration.md @@ -53,7 +53,7 @@ under the License. <br/> -Flink 自带的[默认 flink-conf.yaml](#default-configuration-in-flink-confyaml) 文件指定了 [`taskmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size)（*>= 1.10*）和 [`jobmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size) (*>= 1.11*)，以便与此前的行为保持一致。 +Flink 自带的 [Flink 默认配置文件](#default-configuration-in-flink-confyaml) 文件指定了 [`taskmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size)（*>= 1.10*）和 [`jobmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size) (*>= 1.11*)，以便与此前的行为保持一致。 可以使用这张[电子表格](https://docs.google.com/spreadsheets/d/1mJaMkMPfDJJ-w6nMXALYmTc4XxiV30P5U7DzgwLkSoE)来估算和比较原本的和新的内存配置下的计算结果。 @@ -288,9 +288,9 @@ Flink 通过设置上述 JVM 内存限制降低内存泄漏问题的排查难度 <a name="default-configuration-in-flink-confyaml" /> -## flink-conf.yaml 中的默认配置 +## Flink 配置文件中的默认配置 -本节描述 Flink 自带的默认 `flink-conf.yaml` 文件中的变化。 +本节描述 Flink 自带的默认 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中的变化。 原本的 TaskManager 总内存（`taskmanager.heap.size`）被新的配置项 [`taskmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size) 所取代。 默认值从 1024MB 增加到了 1728MB。 @@ -301,5 +301,5 @@ Flink 通过设置上述 JVM 内存限制降低内存泄漏问题的排查难度 请参考[如何配置总内存]({{< ref "docs/deployment/memory/mem_setup" >}}#configure-total-memory)。 {{< hint warning >}} -**注意：** 使用新的默认 `flink-conf.yaml` 可能会造成各内存部分的大小发生变化，从而产生性能变化。 +**注意：** 使用新的默认 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 可能会造成各内存部分的大小发生变化，从而产生性能变化。 {{< /hint >}} diff --git a/docs/content.zh/docs/deployment/metric_reporters.md b/docs/content.zh/docs/deployment/metric_reporters.md index cbbbcf2d14a..7413f7cc581 100644 --- a/docs/content.zh/docs/deployment/metric_reporters.md +++ b/docs/content.zh/docs/deployment/metric_reporters.md @@ -31,7 +31,7 @@ under the License. Flink 支持用户将 Flink 的各项运行时指标发送给外部系统。 了解更多指标方面信息可查看 [metric 系统相关文档]({{< ref "docs/ops/metrics" >}})。 -你可以通过 `conf/flink-conf.yaml` 文件来配置一种或多种发送器，将运行时指标暴露给外部系统。 +你可以通过 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})来配置一种或多种发送器，将运行时指标暴露给外部系统。 发送器会在 TaskManager、Flink 作业启动时进行实例化。 下面列出了所有发送器都适用的参数，可以通过配置文件中的 `metrics.reporter.<reporter_name>.<property>` 项进行配置。有些发送器有自己特有的配置，详见该发送器章节下的具体说明。 diff --git a/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md b/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md index 4f3a537cdbc..5876cbeab7c 100644 --- a/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md +++ b/docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md @@ -140,7 +140,7 @@ $ ./bin/flink list --target kubernetes-application -Dkubernetes.cluster-id=my-fi $ ./bin/flink cancel --target kubernetes-application -Dkubernetes.cluster-id=my-first-application-cluster <jobId> ``` -You can override configurations set in `conf/flink-conf.yaml` by passing key-value pairs `-Dkey=value` to `bin/flink`. +You can override configurations set in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) by passing key-value pairs `-Dkey=value` to `bin/flink`. ### Per-Job Cluster Mode @@ -166,7 +166,7 @@ $ ./bin/kubernetes-session.sh \ -Dexecution.attached=true ``` -You can override configurations set in `conf/flink-conf.yaml` by passing key-value pairs `-Dkey=value` to `bin/kubernetes-session.sh`. +You can override configurations set in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) by passing key-value pairs `-Dkey=value` to `bin/kubernetes-session.sh`. #### Stop a Running Session Cluster diff --git a/docs/content.zh/docs/deployment/resource-providers/standalone/docker.md b/docs/content.zh/docs/deployment/resource-providers/standalone/docker.md index a4c6a37819f..06adb8c0a42 100644 --- a/docs/content.zh/docs/deployment/resource-providers/standalone/docker.md +++ b/docs/content.zh/docs/deployment/resource-providers/standalone/docker.md @@ -491,7 +491,7 @@ $ docker run flink:{{< stable >}}{{< version >}}-scala{{< scala_version >}}{{< / -D blob.server.port=6124 ``` -Options set via dynamic properties overwrite the options from `flink-conf.yaml`. +Options set via dynamic properties overwrite the options from [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ### Via Environment Variables @@ -508,11 +508,11 @@ $ docker run --env FLINK_PROPERTIES=${FLINK_PROPERTIES} flink:{{< stable >}}{{< The [`jobmanager.rpc.address`]({{< ref "docs/deployment/config" >}}#jobmanager-rpc-address) option must be configured, others are optional to set. The environment variable `FLINK_PROPERTIES` should contain a list of Flink cluster configuration options separated by new line, -the same way as in the `flink-conf.yaml`. `FLINK_PROPERTIES` takes precedence over configurations in `flink-conf.yaml`. +the same way as in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). `FLINK_PROPERTIES` takes precedence over configurations in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). -### Via flink-conf.yaml +### Via Flink configuration file -The configuration files (`flink-conf.yaml`, logging, hosts etc) are located in the `/opt/flink/conf` directory in the Flink image. +The configuration files ([Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}), logging, hosts etc) are located in the `/opt/flink/conf` directory in the Flink image. To provide a custom location for the Flink configuration files, you can * **either mount a volume** with the custom configuration files to this path `/opt/flink/conf` when you run the Flink image: @@ -528,13 +528,13 @@ To provide a custom location for the Flink configuration files, you can ```dockerfile FROM flink - ADD /host/path/to/flink-conf.yaml /opt/flink/conf/flink-conf.yaml + ADD /host/path/to/config.yaml /opt/flink/conf/config.yaml ADD /host/path/to/log4j.properties /opt/flink/conf/log4j.properties ``` {{< hint info >}} The mounted volume must contain all necessary configuration files. -The `flink-conf.yaml` file must have write permission so that the Docker entry point script can modify it in certain cases. +The [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) file must have write permission so that the Docker entry point script can modify it in certain cases. {{< /hint >}} ### Using Filesystem Plugins @@ -630,7 +630,7 @@ You can customize the Flink image in several ways: RUN set -ex; apt-get update; apt-get -y install python - ADD /host/path/to/flink-conf.yaml /container/local/path/to/custom/conf/flink-conf.yaml + ADD /host/path/to/config.yaml /container/local/path/to/custom/conf/config.yaml ADD /host/path/to/log4j.properties /container/local/path/to/custom/conf/log4j.properties RUN ln -fs /opt/flink/opt/flink-sql-gateway-*.jar /opt/flink/lib/. diff --git a/docs/content.zh/docs/deployment/resource-providers/standalone/kubernetes.md b/docs/content.zh/docs/deployment/resource-providers/standalone/kubernetes.md index 95fb16d7dec..a3f5edce0b9 100644 --- a/docs/content.zh/docs/deployment/resource-providers/standalone/kubernetes.md +++ b/docs/content.zh/docs/deployment/resource-providers/standalone/kubernetes.md @@ -158,7 +158,7 @@ $ ./bin/flink run -m localhost:8081 ./examples/streaming/TopSpeedWindowing.jar ### Configuration -所有配置项都展示在[配置页面]({{< ref "docs/deployment/config" >}})上。在 config map 配置文件 `flink-configuration-configmap.yaml` 中，可以将配置添加在 `flink-conf.yaml` 部分。 +所有配置项都展示在[配置页面]({{< ref "docs/deployment/config" >}})上。在 config map 配置文件 `flink-configuration-configmap.yaml` 中，可以将配置添加在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 部分。 <a name="accessing-flink-in-kubernetes"></a> @@ -231,7 +231,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ ... kubernetes.cluster-id: <cluster-id> high-availability.type: kubernetes @@ -281,7 +281,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ jobmanager.rpc.address: flink-jobmanager taskmanager.numberOfTaskSlots: 2 blob.server.port: 6124 @@ -348,7 +348,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ jobmanager.rpc.address: flink-jobmanager taskmanager.numberOfTaskSlots: 2 blob.server.port: 6124 @@ -495,8 +495,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties ``` @@ -553,8 +553,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties ``` @@ -599,8 +599,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties ``` @@ -652,8 +652,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties - name: job-artifacts-volume @@ -712,8 +712,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties - name: job-artifacts-volume @@ -764,8 +764,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties - name: job-artifacts-volume diff --git a/docs/content.zh/docs/deployment/resource-providers/yarn.md b/docs/content.zh/docs/deployment/resource-providers/yarn.md index 474a912c7f9..94970b8fa5b 100644 --- a/docs/content.zh/docs/deployment/resource-providers/yarn.md +++ b/docs/content.zh/docs/deployment/resource-providers/yarn.md @@ -166,7 +166,7 @@ You can **re-attach to a YARN session** using the following command: ./bin/yarn-session.sh -id application_XXXX_YY ``` -Besides passing [configuration]({{< ref "docs/deployment/config" >}}) via the `conf/flink-conf.yaml` file, you can also pass any configuration at submission time to the `./bin/yarn-session.sh` client using `-Dkey=value` arguments. +Besides passing [configuration]({{< ref "docs/deployment/config" >}}) via the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) file, you can also pass any configuration at submission time to the `./bin/yarn-session.sh` client using `-Dkey=value` arguments. The YARN session client also has a few "shortcut arguments" for commonly used settings. They can be listed with `./bin/yarn-session.sh -h`. diff --git a/docs/content.zh/docs/deployment/security/security-ssl.md b/docs/content.zh/docs/deployment/security/security-ssl.md index 60ce3b2c677..e585ad66664 100644 --- a/docs/content.zh/docs/deployment/security/security-ssl.md +++ b/docs/content.zh/docs/deployment/security/security-ssl.md @@ -266,7 +266,7 @@ $ keytool -importcert -keystore rest.signed.keystore -storepass rest_keystore_pa $ keytool -importcert -keystore rest.signed.keystore -storepass rest_keystore_password -file rest.cer -alias flink.rest -noprompt ``` -Now add the following configuration to your `flink-conf.yaml`: +Now add the following configuration to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```yaml security.ssl.rest.enabled: true diff --git a/docs/content.zh/docs/deployment/trace_reporters.md b/docs/content.zh/docs/deployment/trace_reporters.md index eaab7d4ce3d..bfbb038c240 100644 --- a/docs/content.zh/docs/deployment/trace_reporters.md +++ b/docs/content.zh/docs/deployment/trace_reporters.md @@ -29,7 +29,7 @@ under the License. Flink allows reporting traces to external systems. For more information about Flink's tracing system go to the [tracing system documentation]({{< ref "docs/ops/traces" >}}). -Traces can be exposed to an external system by configuring one or several reporters in `conf/flink-conf.yaml`. These +Traces can be exposed to an external system by configuring one or several reporters in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). These reporters will be instantiated on each job and task manager when they are started. Below is a list of parameters that are generally applicable to all reporters. diff --git a/docs/content.zh/docs/dev/datastream/execution/parallel.md b/docs/content.zh/docs/dev/datastream/execution/parallel.md index 216dea0281c..6b2f120d875 100644 --- a/docs/content.zh/docs/dev/datastream/execution/parallel.md +++ b/docs/content.zh/docs/dev/datastream/execution/parallel.md @@ -206,7 +206,7 @@ Python API 中尚不支持该特性。 ### 系统层次 -可以通过设置 `./conf/flink-conf.yaml` 文件中的 `parallelism.default` 参数，在系统层次来指定所有执行环境的默认并行度。你可以通过查阅[配置文档]({{< ref "docs/deployment/config" >}})获取更多细节。 +可以通过设置 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中的 `parallelism.default` 参数，在系统层次来指定所有执行环境的默认并行度。你可以通过查阅[配置文档]({{< ref "docs/deployment/config" >}})获取更多细节。 ## 设置最大并行度 diff --git a/docs/content.zh/docs/dev/datastream/fault-tolerance/checkpointing.md b/docs/content.zh/docs/dev/datastream/fault-tolerance/checkpointing.md index 5f10720e592..4a94f9c3ab1 100644 --- a/docs/content.zh/docs/dev/datastream/fault-tolerance/checkpointing.md +++ b/docs/content.zh/docs/dev/datastream/fault-tolerance/checkpointing.md @@ -182,7 +182,7 @@ env.get_checkpoint_config().enable_unaligned_checkpoints() ### 相关的配置选项 -更多的属性与默认值能在 `conf/flink-conf.yaml` 中设置（完整教程请阅读 [配置]({{< ref "docs/deployment/config" >}})）。 +更多的属性与默认值能在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中设置（完整教程请阅读 [配置]({{< ref "docs/deployment/config" >}})）。 {{< generated/checkpointing_configuration >}} diff --git a/docs/content.zh/docs/dev/datastream/fault-tolerance/state_backends.md b/docs/content.zh/docs/dev/datastream/fault-tolerance/state_backends.md index 9234fe78a89..6a71b906f3b 100644 --- a/docs/content.zh/docs/dev/datastream/fault-tolerance/state_backends.md +++ b/docs/content.zh/docs/dev/datastream/fault-tolerance/state_backends.md @@ -30,7 +30,7 @@ Flink 提供了多种 state backends，它用于指定状态的存储方式和 状态可以位于 Java 的堆或堆外内存。取决于你的 state backend，Flink 也可以自己管理应用程序的状态。 为了让应用程序可以维护非常大的状态，Flink 可以自己管理内存（如果有必要可以溢写到磁盘）。 -默认情况下，所有 Flink Job 会使用配置文件 *flink-conf.yaml* 中指定的 state backend。 +默认情况下，所有 Flink Job 会使用配置文件 [*Flink 配置文件*]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中指定的 state backend。 但是，配置文件中指定的默认 state backend 会被 Job 中指定的 state backend 覆盖，如下所示。 diff --git a/docs/content.zh/docs/dev/python/environment_variables.md b/docs/content.zh/docs/dev/python/environment_variables.md index 0353977d613..61340e79bad 100644 --- a/docs/content.zh/docs/dev/python/environment_variables.md +++ b/docs/content.zh/docs/dev/python/environment_variables.md @@ -57,7 +57,7 @@ These environment variables will affect the behavior of PyFlink: <ol> <li>The configuration 'python.client.executable' defined in the source code; </li> <li>The environment variable PYFLINK_CLIENT_EXECUTABLE; </li> - <li>The configuration 'python.client.executable' defined in flink-conf.yaml</li> + <li>The configuration 'python.client.executable' defined in Flink configuration file</li> </ol> If none of above is set, the default Python interpreter 'python' will be used. </td> diff --git a/docs/content.zh/docs/dev/table/catalogs.md b/docs/content.zh/docs/dev/table/catalogs.md index 2933d35a93e..34ed574d158 100644 --- a/docs/content.zh/docs/dev/table/catalogs.md +++ b/docs/content.zh/docs/dev/table/catalogs.md @@ -839,7 +839,7 @@ TableEnvironment env = TableEnvironment.create( env.executeSql("CREATE TABLE ...").wait(); ``` -For sql-gateway, you can add the option `table.catalog-modification.listeners` in the `flink-conf.yaml` and start +For sql-gateway, you can add the option `table.catalog-modification.listeners` in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) and start the gateway, or you can also start sql-gateway with dynamic parameter, then you can use sql-client to perform ddl directly. ## Catalog Store @@ -879,7 +879,7 @@ final EnvironmentSettings settings = final TableEnvironment tableEnv = TableEnvironment.create(settings); ``` -在 SQL Gateway 中，推荐在 `flink-conf.yaml` 文件中进行配置，所有的 session 可以自动使用已经创建好的 Catalog 。 +在 SQL Gateway 中，推荐在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中进行配置，所有的 session 可以自动使用已经创建好的 Catalog 。 配置的格式如下，一般情况下需要配置 Catalog Store 的 kind ，以及 Catalog Store 需要的其他参数配置。 ```yaml table.catalog-store.kind: file diff --git a/docs/content.zh/docs/dev/table/hive-compatibility/hiveserver2.md b/docs/content.zh/docs/dev/table/hive-compatibility/hiveserver2.md index 3d414e7eda1..c60748d807a 100644 --- a/docs/content.zh/docs/dev/table/hive-compatibility/hiveserver2.md +++ b/docs/content.zh/docs/dev/table/hive-compatibility/hiveserver2.md @@ -38,7 +38,7 @@ The HiveServer2 Endpoint is not the default endpoint for the SQL Gateway. You ca $ ./bin/sql-gateway.sh start -Dsql-gateway.endpoint.type=hiveserver2 -Dsql-gateway.endpoint.hiveserver2.catalog.hive-conf-dir=<path to hive conf> ``` -or add the following configuration into `conf/flink-conf.yaml` (please replace the `<path to hive conf>` with your hive conf path). +or add the following configuration into [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) (please replace the `<path to hive conf>` with your hive conf path). ```yaml sql-gateway.endpoint.type: hiveserver2 diff --git a/docs/content.zh/docs/dev/table/sql-gateway/overview.md b/docs/content.zh/docs/dev/table/sql-gateway/overview.md index 4b91d6862ab..5c979168655 100644 --- a/docs/content.zh/docs/dev/table/sql-gateway/overview.md +++ b/docs/content.zh/docs/dev/table/sql-gateway/overview.md @@ -237,14 +237,14 @@ SQL Gateway 默认集成 REST Endpoint。由于架构的可扩展性，用户可 $ ./bin/sql-gateway.sh start -Dsql-gateway.endpoint.type=hiveserver2 ``` -或者在 `conf/flink-conf.yaml` 中增加如下配置： +或者在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中增加如下配置： ```yaml sql-gateway.endpoint.type: hiveserver2 ``` {{< hint info >}} -Notice: 如果 CLI 命令和 flink-conf.yaml 都有 `sql-gateway.endpoint.type`，CLI 的优先级比 flink-conf.yaml 更高。 +Notice: 如果 CLI 命令和 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 都有 `sql-gateway.endpoint.type`，CLI 的优先级比 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 更高。 {{< /hint >}} 具体的 endpoint 请参考相应页面。 diff --git a/docs/content.zh/docs/dev/table/sqlClient.md b/docs/content.zh/docs/dev/table/sqlClient.md index 15396e03207..f12bb7e6375 100644 --- a/docs/content.zh/docs/dev/table/sqlClient.md +++ b/docs/content.zh/docs/dev/table/sqlClient.md @@ -82,7 +82,7 @@ Cache-Control: no-cache EOF) ``` -By default, the SQL Client will use the truststore configured using the `security.ssl.rest.truststore` and `security.ssl.rest.truststore-password` properties in the `flink-conf.yaml` file on the SQL client side. If these properties aren't explicitly configured, the client will use the default certificate stores provided by the JDK. +By default, the SQL Client will use the truststore configured using the `security.ssl.rest.truststore` and `security.ssl.rest.truststore-password` properties in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) file on the SQL client side. If these properties aren't explicitly configured, the client will use the default certificate stores provided by the JDK. <span class="label label-danger">Note</span> SQL 客户端目前只支持和 REST API 版本大于 v1 的 [REST Endpoint]({{< ref "docs/dev/table/sql-gateway/rest" >}}#rest-api) 通信。 diff --git a/docs/content.zh/docs/ops/debugging/flame_graphs.md b/docs/content.zh/docs/ops/debugging/flame_graphs.md index fdb71fba45e..ca41d0c7c4f 100644 --- a/docs/content.zh/docs/ops/debugging/flame_graphs.md +++ b/docs/content.zh/docs/ops/debugging/flame_graphs.md @@ -48,7 +48,7 @@ Operator's On-CPU Flame Graph {{< hint warning >}} -Any measurement process in and of itself inevitably affects the subject of measurement (see the [double-split experiment](https://en.wikipedia.org/wiki/Double-slit_experiment#Relational_interpretation)). Sampling CPU stack traces is no exception. In order to prevent unintended impacts on production environments, Flame Graphs are currently available as an opt-in feature. To enable it, you'll need to set [`rest.flamegraph.enabled: true`]({{< ref "docs/deployment/config">}}#rest-flamegraph- [...] +Any measurement process in and of itself inevitably affects the subject of measurement (see the [double-split experiment](https://en.wikipedia.org/wiki/Double-slit_experiment#Relational_interpretation)). Sampling CPU stack traces is no exception. In order to prevent unintended impacts on production environments, Flame Graphs are currently available as an opt-in feature. To enable it, you'll need to set [`rest.flamegraph.enabled: true`]({{< ref "docs/deployment/config">}}#rest-flamegraph- [...] {{< /hint >}} diff --git a/docs/content.zh/docs/ops/metrics.md b/docs/content.zh/docs/ops/metrics.md index 959f0f0e700..dba14ee4fa2 100644 --- a/docs/content.zh/docs/ops/metrics.md +++ b/docs/content.zh/docs/ops/metrics.md @@ -492,7 +492,7 @@ Every metric is assigned an identifier and a set of key-value pairs under which The identifier is based on 3 components: a user-defined name when registering the metric, an optional user-defined scope and a system-provided scope. For example, if `A.B` is the system scope, `C.D` the user scope and `E` the name, then the identifier for the metric will be `A.B.C.D.E`. -You can configure which delimiter to use for the identifier (default: `.`) by setting the `metrics.scope.delimiter` key in `conf/flink-conf.yaml`. +You can configure which delimiter to use for the identifier (default: `.`) by setting the `metrics.scope.delimiter` key in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ### User Scope @@ -551,7 +551,7 @@ counter = runtime_context \ The system scope contains context information about the metric, for example in which task it was registered or what job that task belongs to. -Which context information should be included can be configured by setting the following keys in `conf/flink-conf.yaml`. +Which context information should be included can be configured by setting the following keys in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). Each of these keys expect a format string that may contain constants (e.g. "taskmanager") and variables (e.g. "<task_id>") which will be replaced at runtime. - `metrics.scope.jm` diff --git a/docs/content.zh/docs/ops/rest_api.md b/docs/content.zh/docs/ops/rest_api.md index 8ee9c2c4531..2cd6da955e3 100644 --- a/docs/content.zh/docs/ops/rest_api.md +++ b/docs/content.zh/docs/ops/rest_api.md @@ -35,7 +35,7 @@ Flink 具有监控 API ，可用于查询正在运行的作业以及最近完成 ## 概览 -该监控 API 由作为 `JobManager` 一部分运行的 web 服务器提供支持。默认情况下，该服务器监听 8081 端口，端口号可以通过修改 `flink-conf.yaml` 文件的 `rest.port` 进行配置。请注意，该监控 API 的 web 服务器和仪表盘的 web 服务器目前是相同的，因此在同一端口一起运行。不过，它们响应不同的 HTTP URL 。 +该监控 API 由作为 `JobManager` 一部分运行的 web 服务器提供支持。默认情况下，该服务器监听 8081 端口，端口号可以通过修改 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})的 `rest.port` 进行配置。请注意，该监控 API 的 web 服务器和仪表盘的 web 服务器目前是相同的，因此在同一端口一起运行。不过，它们响应不同的 HTTP URL 。 在多个 `JobManager` 的情况下（为了高可用），每个 JobManager 将运行自己的监控 API 实例，当 JobManager 被选举成为集群 leader 时，该实例将提供已完成和正在运行作业的相关信息。 diff --git a/docs/content.zh/docs/ops/state/state_backends.md b/docs/content.zh/docs/ops/state/state_backends.md index 5d7d4f92b1c..4e4b2b08926 100644 --- a/docs/content.zh/docs/ops/state/state_backends.md +++ b/docs/content.zh/docs/ops/state/state_backends.md @@ -114,7 +114,7 @@ EmbeddedRocksDBStateBackend 是目前唯一支持增量 CheckPoint 的 State Bac ## 设置 State Backend -如果没有明确指定，将使用 jobmanager 做为默认的 state backend。你能在 **flink-conf.yaml** 中为所有 Job 设置其他默认的 State Backend。 +如果没有明确指定，将使用 jobmanager 做为默认的 state backend。你能在 [**Flink 配置文件**]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中为所有 Job 设置其他默认的 State Backend。 每一个 Job 的 state backend 配置会覆盖默认的 state backend 配置，如下所示： <a name="setting-the-per-job-state-backend"></a> @@ -158,14 +158,14 @@ env = StreamExecutionEnvironment.get_execution_environment(config) ``` {{< hint info >}} - **注意:** 由于 RocksDB 是 Flink 默认分发包的一部分，所以如果你没在代码中使用 RocksDB，则不需要添加此依赖。而且可以在 `flink-conf.yaml` 文件中通过 `state.backend.type` 配置 State Backend，以及更多的 [checkpointing]({{< ref "docs/deployment/config" >}}#checkpointing) 和 [RocksDB 特定的]({{< ref "docs/deployment/config" >}}#rocksdb-state-backend) 参数。 + **注意:** 由于 RocksDB 是 Flink 默认分发包的一部分，所以如果你没在代码中使用 RocksDB，则不需要添加此依赖。而且可以在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}})中通过 `state.backend.type` 配置 State Backend，以及更多的 [checkpointing]({{< ref "docs/deployment/config" >}}#checkpointing) 和 [RocksDB 特定的]({{< ref "docs/deployment/config" >}}#rocksdb-state-backend) 参数。 {{< /hint >}} <a name="setting-default-state-backend"></a> ### 设置默认的（全局的） State Backend -在 `flink-conf.yaml` 可以通过键 `state.backend.type` 设置默认的 State Backend。 +在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 可以通过键 `state.backend.type` 设置默认的 State Backend。 可选值包括 *jobmanager* (HashMapStateBackend), *rocksdb* (EmbeddedRocksDBStateBackend)， 或使用实现了 state backend 工厂 {{< gh_link file="flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendFactory.java" name="StateBackendFactory" >}} 的类的全限定类名， @@ -204,7 +204,7 @@ RocksDB 支持*增量快照*。不同于产生一个包含所有数据的全量 和基于全量快照的恢复时间相比，如果网络带宽是瓶颈，那么基于增量快照恢复可能会消耗更多时间，因为增量快照包含的 sst 文件之间可能存在数据重叠导致需要下载的数据量变大；而当 CPU 或者 IO 是瓶颈的时候，基于增量快照恢复会更快，因为从增量快照恢复不需要解析 Flink 的统一快照格式来重建本地的 RocksDB 数据表，而是可以直接基于 sst 文件加载。 虽然状态数据量很大时我们推荐使用增量快照，但这并不是默认的快照机制，您需要通过下述配置手动开启该功能： - - 在 `flink-conf.yaml` 中设置：`state.backend.incremental: true` 或者 + - 在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中设置：`state.backend.incremental: true` 或者 - 在代码中按照右侧方式配置（来覆盖默认配置）：`EmbeddedRocksDBStateBackend backend = new EmbeddedRocksDBStateBackend(true);` 需要注意的是，一旦启用了增量快照，网页上展示的 `Checkpointed Data Size` 只代表增量上传的数据量，而不是一次快照的完整数据量。 @@ -270,12 +270,12 @@ Flink还提供了两个参数来控制*写路径*（MemTable）和*读路径*（ 使用*预定义选项*，用户可以在每个 RocksDB 列族上应用一些预定义的配置，例如配置内存使用、线程、Compaction 设置等。目前每个算子的每个状态都在 RocksDB 中有专门的一个列族存储。 有两种方法可以选择要应用的预定义选项： - - 通过 `state.backend.rocksdb.predefined-options` 配置项将选项名称设置进 `flink-conf.yaml` 。 + - 通过 `state.backend.rocksdb.predefined-options` 配置项将选项名称设置进 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 。 - 通过程序设置：`EmbeddedRocksDBStateBackend.setPredefinedOptions(PredefinedOptions.SPINNING_DISK_OPTIMIZED_HIGH_MEM)` 。 该选项的默认值是 `DEFAULT` ，对应 `PredefinedOptions.DEFAULT` 。 -#### 从 flink-conf.yaml 中读取列族选项 +#### 从 Flink 配置文件中读取列族选项 RocksDB State Backend 会将 [这里定义]({{< ref "docs/deployment/config" >}}#advanced-rocksdb-state-backends-options) 的所有配置项全部加载。 因此您可以简单的通过关闭 RocksDB 使用托管内存的功能并将需要的设置选项加入配置文件来配置底层的列族选项。 @@ -288,11 +288,11 @@ RocksDB State Backend 会将 [这里定义]({{< ref "docs/deployment/config" >}} 有两种方法可以将 `RocksDBOptionsFactory` 传递给 RocksDB State Backend： - - 通过 `state.backend.rocksdb.options-factory` 选项将工厂实现类的名称设置到`flink-conf.yaml` 。 + - 通过 `state.backend.rocksdb.options-factory` 选项将工厂实现类的名称设置到[Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 。 - 通过程序设置，例如 `EmbeddedRocksDBStateBackend.setRocksDBOptions(new MyOptionsFactory());` 。 -<span class="label label-info">注意</span> 通过程序设置的 `RocksDBOptionsFactory` 将覆盖 `flink-conf.yaml` 配置文件的设置，且 `RocksDBOptionsFactory` 设置的优先级高于预定义选项（`PredefinedOptions`）。 +<span class="label label-info">注意</span> 通过程序设置的 `RocksDBOptionsFactory` 将覆盖 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 配置文件的设置，且 `RocksDBOptionsFactory` 设置的优先级高于预定义选项（`PredefinedOptions`）。 <span class="label label-info">注意</span> RocksDB是一个本地库，它直接从进程分配内存， 而不是从JVM分配内存。分配给 RocksDB 的任何内存都必须被考虑在内，通常需要将这部分内存从任务管理器（`TaskManager`）的JVM堆中减去。 @@ -485,7 +485,7 @@ env.enable_changelog_statebackend(true) 旧版本的 `MemoryStateBackend` 等价于使用 [`HashMapStateBackend`](#the-hashmapstatebackend) 和 [`JobManagerCheckpointStorage`]({{< ref "docs/ops/state/checkpoints#the-jobmanagercheckpointstorage" >}})。 -#### `flink-conf.yaml` 配置 +#### 使用 Flink 配置文件 ```yaml state.backend: hashmap @@ -527,7 +527,7 @@ env = StreamExecutionEnvironment.get_execution_environment(config) 旧版本的 `FsStateBackend` 等价于使用 [`HashMapStateBackend`](#the-hashmapstatebackend) 和 [`FileSystemCheckpointStorage`]({{< ref "docs/ops/state/checkpoints#the-filesystemcheckpointstorage" >}})。 -#### `flink-conf.yaml` 配置 +#### 使用 Flink 配置文件 ```yaml state.backend: hashmap @@ -589,7 +589,7 @@ env.configure(config); 旧版本的 `RocksDBStateBackend` 等价于使用 [`EmbeddedRocksDBStateBackend`](#the-embeddedrocksdbstatebackend) 和 [`FileSystemCheckpointStorage`]({{< ref "docs/ops/state/checkpoints#the-filesystemcheckpointstorage" >}}). -#### `flink-conf.yaml` 配置 +#### 使用 Flink 配置文件 ```yaml state.backend: rocksdb diff --git a/docs/content.zh/docs/ops/state/task_failure_recovery.md b/docs/content.zh/docs/ops/state/task_failure_recovery.md index 31af2859fce..50a3c7d6da8 100644 --- a/docs/content.zh/docs/ops/state/task_failure_recovery.md +++ b/docs/content.zh/docs/ops/state/task_failure_recovery.md @@ -38,7 +38,7 @@ Flink 通过重启策略和故障恢复策略来控制 Task 重启：重启策 Flink 作业如果没有定义重启策略，则会遵循集群启动时加载的默认重启策略。 如果提交作业时设置了重启策略，该策略将覆盖掉集群的默认策略。 -通过 Flink 的配置文件 `flink-conf.yaml` 来设置默认的重启策略。配置参数 *restart-strategy.type* 定义了采取何种策略。 +通过 Flink 的配置文件 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 来设置默认的重启策略。配置参数 *restart-strategy.type* 定义了采取何种策略。 如果没有启用 checkpoint，就采用“不重启”策略。如果启用了 checkpoint 且没有配置重启策略，那么就采用固定延时重启策略， 此时最大尝试重启次数由 `Integer.MAX_VALUE` 参数设置。下表列出了可用的重启策略和与其对应的配置值。 @@ -93,7 +93,7 @@ env = StreamExecutionEnvironment.get_execution_environment(config) 如果尝试超过了给定的最大次数，作业将最终失败。 在连续的两次重启尝试之间，重启策略等待一段固定长度的时间。 -通过在 `flink-conf.yaml` 中设置如下配置参数，默认启用此策略。 +通过在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中设置如下配置参数，默认启用此策略。 ```yaml restart-strategy.type: fixed-delay @@ -205,7 +205,7 @@ Python API 不支持。 故障率重启策略在故障发生之后重启作业，但是当**故障率**（每个时间间隔发生故障的次数）超过设定的限制时，作业会最终失败。 在连续的两次重启尝试之间，重启策略等待一段固定长度的时间。 -通过在 `flink-conf.yaml` 中设置如下配置参数，默认启用此策略。 +通过在 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中设置如下配置参数，默认启用此策略。 ```yaml restart-strategy.type: failure-rate @@ -298,7 +298,7 @@ env = StreamExecutionEnvironment.get_execution_environment(config) ## Failover Strategies -Flink 支持多种不同的故障恢复策略，该策略需要通过 Flink 配置文件 `flink-conf.yaml` 中的 *jobmanager.execution.failover-strategy* +Flink 支持多种不同的故障恢复策略，该策略需要通过 Flink 配置文件 [Flink 配置文件]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 中的 *jobmanager.execution.failover-strategy* 配置项进行配置。 <table class="table table-bordered"> diff --git a/docs/content.zh/docs/try-flink/flink-operations-playground.md b/docs/content.zh/docs/try-flink/flink-operations-playground.md index 67c70f68acc..cdc73428f73 100644 --- a/docs/content.zh/docs/try-flink/flink-operations-playground.md +++ b/docs/content.zh/docs/try-flink/flink-operations-playground.md @@ -377,8 +377,8 @@ Suspending job "<job-id>" with a savepoint. Suspended job "<job-id>" with a savepoint. ``` -Savepoint 已保存在 `state.savepoints.dir` 指定的路径中，该配置在 *flink-conf.yaml* -中定义，*flink-conf.yaml* 挂载在本机的 */tmp/flink-savepoints-directory/* 目录下。 +Savepoint 已保存在 `state.savepoints.dir` 指定的路径中，该配置在 [*Flink 配置文件*]({{< ref "docs/deployment/config#flink-configuration-file" >}}) +中定义，[*Flink 配置文件*]({{< ref "docs/deployment/config#flink-configuration-file" >}}) 挂载在本机的 */tmp/flink-savepoints-directory/* 目录下。 在下一步操作中我们会用到这个 Savepoint 路径，如果我们是通过 REST API 操作的， 那么 Savepoint 路径会随着响应结果一起返回，我们可以直接查看文件系统来确认 Savepoint 保存情况。 diff --git a/docs/content/docs/connectors/table/filesystem.md b/docs/content/docs/connectors/table/filesystem.md index b9c6e94c285..2037f988657 100644 --- a/docs/content/docs/connectors/table/filesystem.md +++ b/docs/content/docs/connectors/table/filesystem.md @@ -249,8 +249,8 @@ a timeout that specifies the maximum duration for which a file can be open. **NOTE:** For bulk formats (parquet, orc, avro), the rolling policy in combination with the checkpoint interval(pending files become finished on the next checkpoint) control the size and number of these parts. -**NOTE:** For row formats (csv, json), you can set the parameter `sink.rolling-policy.file-size` or `sink.rolling-policy.rollover-interval` in the connector properties and parameter `execution.checkpointing.interval` in flink-conf.yaml together -if you don't want to wait a long period before observe the data exists in file system. For other formats (avro, orc), you can just set parameter `execution.checkpointing.interval` in flink-conf.yaml. +**NOTE:** For row formats (csv, json), you can set the parameter `sink.rolling-policy.file-size` or `sink.rolling-policy.rollover-interval` in the connector properties and parameter `execution.checkpointing.interval` in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) together +if you don't want to wait a long period before observe the data exists in file system. For other formats (avro, orc), you can just set parameter `execution.checkpointing.interval` in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ### File Compaction diff --git a/docs/content/docs/deployment/advanced/historyserver.md b/docs/content/docs/deployment/advanced/historyserver.md index db005c11ea8..7d0053bb230 100644 --- a/docs/content/docs/deployment/advanced/historyserver.md +++ b/docs/content/docs/deployment/advanced/historyserver.md @@ -52,7 +52,7 @@ The configuration keys `jobmanager.archive.fs.dir` and `historyserver.archive.fs **JobManager** -The archiving of completed jobs happens on the JobManager, which uploads the archived job information to a file system directory. You can configure the directory to archive completed jobs in `flink-conf.yaml` by setting a directory via `jobmanager.archive.fs.dir`. +The archiving of completed jobs happens on the JobManager, which uploads the archived job information to a file system directory. You can configure the directory to archive completed jobs in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) by setting a directory via `jobmanager.archive.fs.dir`. ```yaml # Directory to upload completed job information diff --git a/docs/content/docs/deployment/cli.md b/docs/content/docs/deployment/cli.md index a8818a4fb6b..39a35ff0480 100644 --- a/docs/content/docs/deployment/cli.md +++ b/docs/content/docs/deployment/cli.md @@ -30,7 +30,7 @@ under the License. Flink provides a Command-Line Interface (CLI) `bin/flink` to run programs that are packaged as JAR files and to control their execution. The CLI is part of any Flink setup, available in local single node setups and in distributed setups. -It connects to the running JobManager specified in `conf/flink-conf.yaml`. +It connects to the running JobManager specified in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ## Job Lifecycle Management @@ -364,7 +364,7 @@ Here's an overview of actions supported by Flink's CLI tool: This action can be used to create or disposing savepoints for a given job. It might be necessary to specify a savepoint directory besides the JobID, if the <a href="{{< ref "docs/deployment/config" >}}#state-savepoints-dir">state.savepoints.dir</a> - parameter was not specified in <code class="highlighter-rouge">conf/flink-conf.yaml</code>. + parameter was not specified in <code class="highlighter-rouge">Flink configuration file</code>. </td> </tr> <tr> @@ -429,7 +429,7 @@ parameter combinations: * `./bin/flink run --target remote`: Submission to an already running Flink cluster The `--target` will overwrite the [execution.target]({{< ref "docs/deployment/config" >}}#execution-target) -specified in the `conf/flink-conf.yaml`. +specified in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). For more details on the commands and the available options, please refer to the Resource Provider-specific pages of the documentation. diff --git a/docs/content/docs/deployment/config.md b/docs/content/docs/deployment/config.md index 126d8289019..bf1e9c67d51 100644 --- a/docs/content/docs/deployment/config.md +++ b/docs/content/docs/deployment/config.md @@ -31,7 +31,7 @@ All configuration can be set in Flink configuration file in the `conf/` director The configuration is parsed and evaluated when the Flink processes are started. Changes to the configuration file require restarting the relevant processes. -The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in Flink configuration file if you want to manually override the Java runtime to use. +The out of the box configuration will use your default Java installation. You can manually set the environment variable `JAVA_HOME` or the configuration key `env.java.home` in Flink configuration file if you want to manually override the Java runtime to use. Note that the configuration key `env.java.home` must be specified in a flattened format (i.e. one-line key-value format) in the configuration file. You can specify a different configuration directory location by defining the `FLINK_CONF_DIR` environment variable. For resource providers which provide non-session deployments, you can specify per-job configurations this way. Make a copy of the `conf` directory from the Flink distribution and modify the settings on a per-job basis. Note that this is not supported in Docker or standalone Kubernetes deployments. On Docker-based deployments, you can use the `FLINK_PROPERTIES` environment v [...] diff --git a/docs/content/docs/deployment/filesystems/azure.md b/docs/content/docs/deployment/filesystems/azure.md index 0e761606e02..ea8832e9d5d 100644 --- a/docs/content/docs/deployment/filesystems/azure.md +++ b/docs/content/docs/deployment/filesystems/azure.md @@ -92,7 +92,7 @@ Hadoop's WASB Azure Filesystem supports configuration of credentials via the Had outlined in the [Hadoop Azure Blob Storage documentation](https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Configuring_Credentials). For convenience Flink forwards all Flink configurations with a key prefix of `fs.azure` to the Hadoop configuration of the filesystem. Consequently, the azure blob storage key can be configured -in `flink-conf.yaml` via: +in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) via: ```yaml fs.azure.account.key.<account_name>.blob.core.windows.net: <azure_storage_key> @@ -100,7 +100,7 @@ fs.azure.account.key.<account_name>.blob.core.windows.net: <azure_storage_key> Alternatively, the filesystem can be configured to read the Azure Blob Storage key from an environment variable `AZURE_STORAGE_KEY` by setting the following configuration keys in -`flink-conf.yaml`. +[Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ```yaml fs.azure.account.keyprovider.<account_name>.blob.core.windows.net: org.apache.flink.fs.azurefs.EnvironmentVariableKeyProvider @@ -117,7 +117,7 @@ Please visit the [page](https://docs.microsoft.com/en-us/azure/active-directory/ {{< /hint >}} ##### Accessing ABFS using storage Keys (Discouraged) -Azure blob storage key can be configured in `flink-conf.yaml` via: +Azure blob storage key can be configured in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) via: ```yaml fs.azure.account.key.<account_name>.dfs.core.windows.net: <azure_storage_key> diff --git a/docs/content/docs/deployment/filesystems/gcs.md b/docs/content/docs/deployment/filesystems/gcs.md index e6f8520b019..58670cb3355 100644 --- a/docs/content/docs/deployment/filesystems/gcs.md +++ b/docs/content/docs/deployment/filesystems/gcs.md @@ -71,13 +71,13 @@ cp ./opt/flink-gs-fs-hadoop-{{< version >}}.jar ./plugins/gs-fs-hadoop/ ### Configuration -The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.18/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your `flink-conf.yaml`. +The underlying Hadoop file system can be configured using the [Hadoop configuration keys](https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v2.2.18/gcs/CONFIGURATION.md) for `gcs-connector` by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). -For example, `gcs-connector` has a `fs.gs.http.connect-timeout` configuration key. If you want to change it, you need to set `gs.http.connect-timeout: xyz` in `flink-conf.yaml`. Flink will internally translate this back to `fs.gs.http.connect-timeout`. +For example, `gcs-connector` has a `fs.gs.http.connect-timeout` configuration key. If you want to change it, you need to set `gs.http.connect-timeout: xyz` in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). Flink will internally translate this back to `fs.gs.http.connect-timeout`. You can also set `gcs-connector` options directly in the Hadoop `core-site.xml` configuration file, so long as the Hadoop configuration directory is made known to Flink via the `env.hadoop.conf.dir` Flink option or via the `HADOOP_CONF_DIR` environment variable. -`flink-gs-fs-hadoop` can also be configured by setting the following options in `flink-conf.yaml`: +`flink-gs-fs-hadoop` can also be configured by setting the following options in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): | Key | Description [...] |---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...] diff --git a/docs/content/docs/deployment/filesystems/oss.md b/docs/content/docs/deployment/filesystems/oss.md index 9025f7e2170..11184340b3b 100644 --- a/docs/content/docs/deployment/filesystems/oss.md +++ b/docs/content/docs/deployment/filesystems/oss.md @@ -71,11 +71,11 @@ cp ./opt/flink-oss-fs-hadoop-{{< version >}}.jar ./plugins/oss-fs-hadoop/ After setting up the OSS FileSystem wrapper, you need to add some configurations to make sure that Flink is allowed to access your OSS buckets. -To allow for easy adoption, you can use the same configuration keys in `flink-conf.yaml` as in Hadoop's `core-site.xml` +To allow for easy adoption, you can use the same configuration keys in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) as in Hadoop's `core-site.xml` You can see the configuration keys in the [Hadoop OSS documentation](http://hadoop.apache.org/docs/current/hadoop-aliyun/tools/hadoop-aliyun/index.html). -There are some required configurations that must be added to `flink-conf.yaml` (**Other configurations defined in Hadoop OSS documentation are advanced configurations which used by performance tuning**): +There are some required configurations that must be added to [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) (**Other configurations defined in Hadoop OSS documentation are advanced configurations which used by performance tuning**): ```yaml fs.oss.endpoint: Aliyun OSS endpoint to connect to @@ -83,7 +83,7 @@ fs.oss.accessKeyId: Aliyun access key ID fs.oss.accessKeySecret: Aliyun access key secret ``` -An alternative `CredentialsProvider` can also be configured in the `flink-conf.yaml`, e.g. +An alternative `CredentialsProvider` can also be configured in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}), e.g. ```yaml # Read Credentials from OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET fs.oss.credentials.provider: com.aliyun.oss.common.auth.EnvironmentVariableCredentialsProvider diff --git a/docs/content/docs/deployment/filesystems/s3.md b/docs/content/docs/deployment/filesystems/s3.md index 3488258002d..1796cbdb704 100644 --- a/docs/content/docs/deployment/filesystems/s3.md +++ b/docs/content/docs/deployment/filesystems/s3.md @@ -66,12 +66,12 @@ Flink provides two file systems to talk to Amazon S3, `flink-s3-fs-presto` and ` Both implementations are self-contained with no dependency footprint, so there is no need to add Hadoop to the classpath to use them. - `flink-s3-fs-presto`, registered under the scheme *s3://* and *s3p://*, is based on code from the [Presto project](https://prestodb.io/). - You can configure it using [the same configuration keys as the Presto file system](https://prestodb.io/docs/0.272/connector/hive.html#amazon-s3-configuration), by adding the configurations to your `flink-conf.yaml`. The Presto S3 implementation is the recommended file system for checkpointing to S3. + You can configure it using [the same configuration keys as the Presto file system](https://prestodb.io/docs/0.272/connector/hive.html#amazon-s3-configuration), by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). The Presto S3 implementation is the recommended file system for checkpointing to S3. - `flink-s3-fs-hadoop`, registered under *s3://* and *s3a://*, based on code from the [Hadoop Project](https://hadoop.apache.org/). - The file system can be [configured using Hadoop's s3a configuration keys](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A) by adding the configurations to your `flink-conf.yaml`. + The file system can be [configured using Hadoop's s3a configuration keys](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A) by adding the configurations to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). - For example, Hadoop has a `fs.s3a.connection.maximum` configuration key. If you want to change it, you need to put `s3.connection.maximum: xyz` to the `flink-conf.yaml`. Flink will internally translate this back to `fs.s3a.connection.maximum`. There is no need to pass configuration parameters using Hadoop's XML configuration files. + For example, Hadoop has a `fs.s3a.connection.maximum` configuration key. If you want to change it, you need to put `s3.connection.maximum: xyz` to the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). Flink will internally translate this back to `fs.s3a.connection.maximum`. There is no need to pass configuration parameters using Hadoop's XML configuration files. It is the only S3 file system with support for the [FileSystem]({{< ref "docs/connectors/datastream/filesystem" >}}). @@ -104,7 +104,7 @@ If you set this up correctly, you can manage access to S3 within AWS and don't n Access to S3 can be granted via your **access and secret key pair**. Please note that this is discouraged since the [introduction of IAM roles](https://blogs.aws.amazon.com/security/post/Tx1XG3FX6VMU6O5/A-safer-way-to-distribute-AWS-credentials-to-EC2). -You need to configure both `s3.access-key` and `s3.secret-key` in Flink's `flink-conf.yaml`: +You need to configure both `s3.access-key` and `s3.secret-key` in Flink's [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```yaml s3.access-key: your-access-key @@ -114,7 +114,7 @@ s3.secret-key: your-secret-key ## Configure Non-S3 Endpoint The S3 Filesystems also support using S3 compliant object stores such as [IBM's Cloud Object Storage](https://www.ibm.com/cloud/object-storage) and [MinIO](https://min.io/). -To do so, configure your endpoint in `flink-conf.yaml`. +To do so, configure your endpoint in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ```yaml s3.endpoint: your-endpoint-hostname @@ -122,7 +122,7 @@ s3.endpoint: your-endpoint-hostname ## Configure Path Style Access -Some S3 compliant object stores might not have virtual host style addressing enabled by default, for example when using Standalone MinIO for testing purpose. In such cases, you will have to provide the property to enable path style access in `flink-conf.yaml`. +Some S3 compliant object stores might not have virtual host style addressing enabled by default, for example when using Standalone MinIO for testing purpose. In such cases, you will have to provide the property to enable path style access in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ```yaml s3.path.style.access: true diff --git a/docs/content/docs/deployment/ha/kubernetes_ha.md b/docs/content/docs/deployment/ha/kubernetes_ha.md index 13f83947a43..3d8767925aa 100644 --- a/docs/content/docs/deployment/ha/kubernetes_ha.md +++ b/docs/content/docs/deployment/ha/kubernetes_ha.md @@ -69,7 +69,7 @@ kubernetes.cluster-id: cluster1337 ### Example configuration -Configure high availability mode in `conf/flink-conf.yaml`: +Configure high availability mode in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```yaml kubernetes.cluster-id: <cluster-id> diff --git a/docs/content/docs/deployment/ha/zookeeper_ha.md b/docs/content/docs/deployment/ha/zookeeper_ha.md index 38173da5e3c..6d33430cb6d 100644 --- a/docs/content/docs/deployment/ha/zookeeper_ha.md +++ b/docs/content/docs/deployment/ha/zookeeper_ha.md @@ -73,7 +73,7 @@ The *cluster-id ZooKeeper node*, under which all required coordination data for ### Example configuration -Configure high availability mode and ZooKeeper quorum in `conf/flink-conf.yaml`: +Configure high availability mode and ZooKeeper quorum in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```bash high-availability.type: zookeeper @@ -87,7 +87,7 @@ high-availability.storageDir: hdfs:///flink/recovery ## Configuring for ZooKeeper Security -If ZooKeeper is running in secure mode with Kerberos, you can override the following configurations in `flink-conf.yaml` as necessary: +If ZooKeeper is running in secure mode with Kerberos, you can override the following configurations in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) as necessary: ```bash # default is "zookeeper". If the ZooKeeper quorum is configured diff --git a/docs/content/docs/deployment/memory/mem_migration.md b/docs/content/docs/deployment/memory/mem_migration.md index 66e4d62ec47..133d544f4bc 100644 --- a/docs/content/docs/deployment/memory/mem_migration.md +++ b/docs/content/docs/deployment/memory/mem_migration.md @@ -52,7 +52,7 @@ the following options is configured explicitly, otherwise the configuration will <br/> -The [default `flink-conf.yaml`](#default-configuration-in-flink-confyaml) shipped with Flink sets +The [default Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) shipped with Flink sets [`taskmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size) (since *1.10*) and [`jobmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size) (since *1.11*) to make the default memory configuration consistent. @@ -274,18 +274,18 @@ The direct or native off-heap memory consumers can now be addressed by the follo * JVM metaspace ([`jobmanager.memory.jvm-metaspace.size`]({{< ref "docs/deployment/config" >}}#jobmanager-memory-jvm-metaspace-size)) * [JVM overhead]({{< ref "docs/deployment/memory/mem_setup_jobmanager" >}}#detailed-configuration) -## Default Configuration in flink-conf.yaml +## Default Configuration in Flink configuration file -This section describes the changes of the default `flink-conf.yaml` shipped with Flink. +This section describes the changes of the default [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) shipped with Flink. The total memory for TaskManagers (`taskmanager.heap.size`) is replaced by [`taskmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size) -in the default `flink-conf.yaml`. The value increased from 1024MB to 1728MB. +in the default [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). The value increased from 1024MB to 1728MB. The total memory for JobManagers (`jobmanager.heap.size`) is replaced by [`jobmanager.memory.process.size`]({{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size) -in the default `flink-conf.yaml`. The value increased from 1024MB to 1600MB. +in the default [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). The value increased from 1024MB to 1600MB. See also [how to configure total memory now]({{< ref "docs/deployment/memory/mem_setup" >}}#configure-total-memory). {{< hint warning >}} -**Warning:** If you use the new default `flink-conf.yaml` it can result in different sizes of memory components and can lead to performance changes. +**Warning:** If you use the new default [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) it can result in different sizes of memory components and can lead to performance changes. {{< /hint >}} diff --git a/docs/content/docs/deployment/metric_reporters.md b/docs/content/docs/deployment/metric_reporters.md index 7409630fb90..ed88a49fb97 100644 --- a/docs/content/docs/deployment/metric_reporters.md +++ b/docs/content/docs/deployment/metric_reporters.md @@ -29,7 +29,7 @@ under the License. Flink allows reporting metrics to external systems. For more information about Flink's metric system go to the [metric system documentation]({{< ref "docs/ops/metrics" >}}). -Metrics can be exposed to an external system by configuring one or several reporters in `conf/flink-conf.yaml`. These +Metrics can be exposed to an external system by configuring one or several reporters in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). These reporters will be instantiated on each job and task manager when they are started. Below is a list of parameters that are generally applicable to all reporters. All properties are configured by setting `metrics.reporter.<reporter_name>.<property>` in the configuration. Reporters may additionally offer implementation-specific parameters, which are documented in the respective reporter's section. diff --git a/docs/content/docs/deployment/resource-providers/native_kubernetes.md b/docs/content/docs/deployment/resource-providers/native_kubernetes.md index 8b676ab983b..6748afd0a95 100644 --- a/docs/content/docs/deployment/resource-providers/native_kubernetes.md +++ b/docs/content/docs/deployment/resource-providers/native_kubernetes.md @@ -148,7 +148,7 @@ $ ./bin/flink list --target kubernetes-application -Dkubernetes.cluster-id=my-fi $ ./bin/flink cancel --target kubernetes-application -Dkubernetes.cluster-id=my-first-application-cluster <jobId> ``` -You can override configurations set in `conf/flink-conf.yaml` by passing key-value pairs `-Dkey=value` to `bin/flink`. +You can override configurations set in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) by passing key-value pairs `-Dkey=value` to `bin/flink`. ### Session Mode @@ -174,7 +174,7 @@ $ ./bin/kubernetes-session.sh \ -Dexecution.attached=true ``` -You can override configurations set in `conf/flink-conf.yaml` by passing key-value pairs `-Dkey=value` to `bin/kubernetes-session.sh`. +You can override configurations set in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) by passing key-value pairs `-Dkey=value` to `bin/kubernetes-session.sh`. #### Stop a Running Session Cluster diff --git a/docs/content/docs/deployment/resource-providers/standalone/docker.md b/docs/content/docs/deployment/resource-providers/standalone/docker.md index ba355b69ced..e3eca1b1cfb 100644 --- a/docs/content/docs/deployment/resource-providers/standalone/docker.md +++ b/docs/content/docs/deployment/resource-providers/standalone/docker.md @@ -490,7 +490,7 @@ $ docker run flink:{{< stable >}}{{< version >}}-scala{{< scala_version >}}{{< / -D blob.server.port=6124 ``` -Options set via dynamic properties overwrite the options from `flink-conf.yaml`. +Options set via dynamic properties overwrite the options from [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ### Via Environment Variables @@ -507,11 +507,11 @@ $ docker run --env FLINK_PROPERTIES=${FLINK_PROPERTIES} flink:{{< stable >}}{{< The [`jobmanager.rpc.address`]({{< ref "docs/deployment/config" >}}#jobmanager-rpc-address) option must be configured, others are optional to set. The environment variable `FLINK_PROPERTIES` should contain a list of Flink cluster configuration options separated by new line, -the same way as in the `flink-conf.yaml`. `FLINK_PROPERTIES` takes precedence over configurations in `flink-conf.yaml`. +the same way as in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). `FLINK_PROPERTIES` takes precedence over configurations in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). -### Via flink-conf.yaml +### Via Flink configuration file -The configuration files (`flink-conf.yaml`, logging, hosts etc) are located in the `/opt/flink/conf` directory in the Flink image. +The configuration files ([Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}), logging, hosts etc) are located in the `/opt/flink/conf` directory in the Flink image. To provide a custom location for the Flink configuration files, you can * **either mount a volume** with the custom configuration files to this path `/opt/flink/conf` when you run the Flink image: @@ -527,13 +527,13 @@ To provide a custom location for the Flink configuration files, you can ```dockerfile FROM flink - ADD /host/path/to/flink-conf.yaml /opt/flink/conf/flink-conf.yaml + ADD /host/path/to/config.yaml /opt/flink/conf/config.yaml ADD /host/path/to/log4j.properties /opt/flink/conf/log4j.properties ``` {{< hint info >}} The mounted volume must contain all necessary configuration files. -The `flink-conf.yaml` file must have write permission so that the Docker entry point script can modify it in certain cases. +The [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) file must have write permission so that the Docker entry point script can modify it in certain cases. {{< /hint >}} ### Using Filesystem Plugins @@ -629,7 +629,7 @@ You can customize the Flink image in several ways: RUN set -ex; apt-get update; apt-get -y install python - ADD /host/path/to/flink-conf.yaml /container/local/path/to/custom/conf/flink-conf.yaml + ADD /host/path/to/config.yaml /container/local/path/to/custom/conf/config.yaml ADD /host/path/to/log4j.properties /container/local/path/to/custom/conf/log4j.properties RUN ln -fs /opt/flink/opt/flink-sql-gateway-*.jar /opt/flink/lib/. diff --git a/docs/content/docs/deployment/resource-providers/standalone/kubernetes.md b/docs/content/docs/deployment/resource-providers/standalone/kubernetes.md index f050a19e21c..d13796e3eb8 100644 --- a/docs/content/docs/deployment/resource-providers/standalone/kubernetes.md +++ b/docs/content/docs/deployment/resource-providers/standalone/kubernetes.md @@ -155,7 +155,7 @@ Deployment of a Session cluster is explained in the [Getting Started](#getting-s ### Configuration -All configuration options are listed on the [configuration page]({{< ref "docs/deployment/config" >}}). Configuration options can be added to the `flink-conf.yaml` section of the `flink-configuration-configmap.yaml` config map. +All configuration options are listed on the [configuration page]({{< ref "docs/deployment/config" >}}). Configuration options can be added to the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) section of the `flink-configuration-configmap.yaml` config map. ### Accessing Flink in Kubernetes @@ -218,7 +218,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ ... kubernetes.cluster-id: <cluster-id> high-availability.type: kubernetes @@ -274,7 +274,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ jobmanager.rpc.address: flink-jobmanager taskmanager.numberOfTaskSlots: 2 blob.server.port: 6124 @@ -341,7 +341,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ jobmanager.rpc.address: flink-jobmanager taskmanager.numberOfTaskSlots: 2 blob.server.port: 6124 @@ -483,8 +483,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties ``` @@ -541,8 +541,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties ``` @@ -587,8 +587,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties ``` @@ -638,8 +638,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties - name: job-artifacts-volume @@ -698,8 +698,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties - name: job-artifacts-volume @@ -750,8 +750,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties - name: job-artifacts-volume @@ -769,7 +769,7 @@ metadata: labels: app: flink data: - flink-conf.yaml: |+ + config.yaml: |+ jobmanager.rpc.address: flink-jobmanager taskmanager.numberOfTaskSlots: 2 blob.server.port: 6124 @@ -837,8 +837,8 @@ spec: configMap: name: flink-config items: - - key: flink-conf.yaml - path: flink-conf.yaml + - key: config.yaml + path: config.yaml - key: log4j-console.properties path: log4j-console.properties volumeClaimTemplates: diff --git a/docs/content/docs/deployment/resource-providers/standalone/overview.md b/docs/content/docs/deployment/resource-providers/standalone/overview.md index 7c37f2789e1..eda2e62fb14 100644 --- a/docs/content/docs/deployment/resource-providers/standalone/overview.md +++ b/docs/content/docs/deployment/resource-providers/standalone/overview.md @@ -148,7 +148,7 @@ Example: $ ./bin/jobmanager.sh start -D jobmanager.rpc.address=localhost -D rest.port=8081 ``` -Options set via dynamic properties overwrite the options from `flink-conf.yaml`. +Options set via dynamic properties overwrite the options from [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ### Debugging @@ -259,7 +259,7 @@ By default, the JobManager will pick a *random port* for inter process communica #### Example: Standalone HA Cluster with 2 JobManagers -1. Configure high availability mode and ZooKeeper quorum in `conf/flink-conf.yaml`: +1. Configure high availability mode and ZooKeeper quorum in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```bash high-availability.type: zookeeper diff --git a/docs/content/docs/deployment/resource-providers/yarn.md b/docs/content/docs/deployment/resource-providers/yarn.md index 8cfd9b7eb5c..f0feaf3a8bb 100644 --- a/docs/content/docs/deployment/resource-providers/yarn.md +++ b/docs/content/docs/deployment/resource-providers/yarn.md @@ -151,7 +151,7 @@ You can **re-attach to a YARN session** using the following command: ./bin/yarn-session.sh -id application_XXXX_YY ``` -Besides passing [configuration]({{< ref "docs/deployment/config" >}}) via the `conf/flink-conf.yaml` file, you can also pass any configuration at submission time to the `./bin/yarn-session.sh` client using `-Dkey=value` arguments. +Besides passing [configuration]({{< ref "docs/deployment/config" >}}) via the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) file, you can also pass any configuration at submission time to the `./bin/yarn-session.sh` client using `-Dkey=value` arguments. The YARN session client also has a few "shortcut arguments" for commonly used settings. They can be listed with `./bin/yarn-session.sh -h`. diff --git a/docs/content/docs/deployment/security/security-ssl.md b/docs/content/docs/deployment/security/security-ssl.md index 17795dfbf6f..90f4db70c3b 100644 --- a/docs/content/docs/deployment/security/security-ssl.md +++ b/docs/content/docs/deployment/security/security-ssl.md @@ -266,7 +266,7 @@ $ keytool -importcert -keystore rest.signed.keystore -storepass rest_keystore_pa $ keytool -importcert -keystore rest.signed.keystore -storepass rest_keystore_password -file rest.cer -alias flink.rest -noprompt ``` -Now add the following configuration to your `flink-conf.yaml`: +Now add the following configuration to your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```yaml security.ssl.rest.enabled: true diff --git a/docs/content/docs/deployment/trace_reporters.md b/docs/content/docs/deployment/trace_reporters.md index eaab7d4ce3d..bfbb038c240 100644 --- a/docs/content/docs/deployment/trace_reporters.md +++ b/docs/content/docs/deployment/trace_reporters.md @@ -29,7 +29,7 @@ under the License. Flink allows reporting traces to external systems. For more information about Flink's tracing system go to the [tracing system documentation]({{< ref "docs/ops/traces" >}}). -Traces can be exposed to an external system by configuring one or several reporters in `conf/flink-conf.yaml`. These +Traces can be exposed to an external system by configuring one or several reporters in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). These reporters will be instantiated on each job and task manager when they are started. Below is a list of parameters that are generally applicable to all reporters. diff --git a/docs/content/docs/dev/datastream/execution/parallel.md b/docs/content/docs/dev/datastream/execution/parallel.md index 0b0b7e67ca8..7287cc5cc8d 100644 --- a/docs/content/docs/dev/datastream/execution/parallel.md +++ b/docs/content/docs/dev/datastream/execution/parallel.md @@ -222,7 +222,7 @@ Still not supported in Python API. ### System Level A system-wide default parallelism for all execution environments can be defined by setting the -`parallelism.default` property in `./conf/flink-conf.yaml`. See the +`parallelism.default` property in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). See the [Configuration]({{< ref "docs/deployment/config" >}}) documentation for details. ## Setting the Maximum Parallelism diff --git a/docs/content/docs/dev/datastream/fault-tolerance/checkpointing.md b/docs/content/docs/dev/datastream/fault-tolerance/checkpointing.md index a17c21bee5f..5dc9e8499f6 100644 --- a/docs/content/docs/dev/datastream/fault-tolerance/checkpointing.md +++ b/docs/content/docs/dev/datastream/fault-tolerance/checkpointing.md @@ -210,7 +210,7 @@ env.get_checkpoint_config().enable_unaligned_checkpoints() ### Related Config Options -Some more parameters and/or defaults may be set via `conf/flink-conf.yaml` (see [configuration]({{< ref "docs/deployment/config" >}}) for a full guide): +Some more parameters and/or defaults may be set via Flink configuration file (see [configuration]({{< ref "docs/deployment/config" >}}))` for a full guide): {{< generated/checkpointing_configuration >}} diff --git a/docs/content/docs/dev/datastream/fault-tolerance/state_backends.md b/docs/content/docs/dev/datastream/fault-tolerance/state_backends.md index d4dc3bda82c..f379efc5ce0 100644 --- a/docs/content/docs/dev/datastream/fault-tolerance/state_backends.md +++ b/docs/content/docs/dev/datastream/fault-tolerance/state_backends.md @@ -28,7 +28,7 @@ under the License. Flink provides different state backends that specify how and where state is stored. -State can be located on Java’s heap or off-heap. Depending on your state backend, Flink can also manage the state for the application, meaning Flink deals with the memory management (possibly spilling to disk if necessary) to allow applications to hold very large state. By default, the configuration file *flink-conf.yaml* determines the state backend for all Flink jobs. +State can be located on Java’s heap or off-heap. Depending on your state backend, Flink can also manage the state for the application, meaning Flink deals with the memory management (possibly spilling to disk if necessary) to allow applications to hold very large state. By default, the configuration file [*Flink configuration file*]({{< ref "docs/deployment/config#flink-configuration-file" >}}) determines the state backend for all Flink jobs. However, the default state backend can be overridden on a per-job basis, as shown below. diff --git a/docs/content/docs/dev/python/environment_variables.md b/docs/content/docs/dev/python/environment_variables.md index 57f6f1f1c86..04327e6c2c4 100644 --- a/docs/content/docs/dev/python/environment_variables.md +++ b/docs/content/docs/dev/python/environment_variables.md @@ -57,7 +57,7 @@ These environment variables will affect the behavior of PyFlink: <ol> <li>The configuration 'python.client.executable' defined in the source code; </li> <li>The environment variable PYFLINK_CLIENT_EXECUTABLE; </li> - <li>The configuration 'python.client.executable' defined in flink-conf.yaml</li> + <li>The configuration 'python.client.executable' defined in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}})</li> </ol> If none of above is set, the default Python interpreter 'python' will be used. </td> diff --git a/docs/content/docs/dev/table/catalogs.md b/docs/content/docs/dev/table/catalogs.md index a042397ada9..534e4be4eaf 100644 --- a/docs/content/docs/dev/table/catalogs.md +++ b/docs/content/docs/dev/table/catalogs.md @@ -843,8 +843,8 @@ TableEnvironment env = TableEnvironment.create( env.executeSql("CREATE TABLE ...").wait(); ``` -For sql-gateway, you can add the option `table.catalog-modification.listeners` in the `flink-conf.yaml` and start -the gateway, or you can also start sql-gateway with dynamic parameter, then you can use sql-client to perform ddl directly. +For sql-gateway, you can add the option `table.catalog-modification.listeners` in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) +and start the gateway, or you can also start sql-gateway with dynamic parameter, then you can use sql-client to perform ddl directly. ## Catalog Store diff --git a/docs/content/docs/dev/table/config.md b/docs/content/docs/dev/table/config.md index 9d2fcc9a666..697d820db4c 100644 --- a/docs/content/docs/dev/table/config.md +++ b/docs/content/docs/dev/table/config.md @@ -125,8 +125,8 @@ Flink SQL> SET 'table.exec.mini-batch.size' = '5000'; {{< /tabs >}} {{< hint info >}} -**Note:** All of the following configuration options can also be set globally in -`conf/flink-conf.yaml` (see [configuration]({{< ref "docs/deployment/config" >}}) and can be later +**Note:** All of the following configuration options can also be set globally in +[Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) and can be later on overridden in the application, through `EnvironmentSettings`, before instantiating the `TableEnvironment`, or through the `TableConfig` of the `TableEnvironment`. {{< /hint >}} diff --git a/docs/content/docs/dev/table/sql-gateway/hiveserver2.md b/docs/content/docs/dev/table/sql-gateway/hiveserver2.md index 5ae5fe22e6b..39436a78c3e 100644 --- a/docs/content/docs/dev/table/sql-gateway/hiveserver2.md +++ b/docs/content/docs/dev/table/sql-gateway/hiveserver2.md @@ -41,7 +41,7 @@ The HiveServer2 Endpoint is not the default endpoint for the SQL Gateway. You ca $ ./bin/sql-gateway.sh start -Dsql-gateway.endpoint.type=hiveserver2 -Dsql-gateway.endpoint.hiveserver2.catalog.hive-conf-dir=<path to hive conf> ``` -or add the following configuration into `conf/flink-conf.yaml` (please replace the `<path to hive conf>` with your hive conf path). +or add the following configuration into [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) (please replace the `<path to hive conf>` with your hive conf path). ```yaml sql-gateway.endpoint.type: hiveserver2 diff --git a/docs/content/docs/dev/table/sql-gateway/overview.md b/docs/content/docs/dev/table/sql-gateway/overview.md index 83e8e1ed9e8..a46c2628c05 100644 --- a/docs/content/docs/dev/table/sql-gateway/overview.md +++ b/docs/content/docs/dev/table/sql-gateway/overview.md @@ -239,14 +239,14 @@ The SQL Gateway is bundled with the REST Endpoint by default. With the flexible $ ./bin/sql-gateway.sh start -Dsql-gateway.endpoint.type=hiveserver2 ``` -or add the following config in the `conf/flink-conf.yaml`: +or add the following config in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): ```yaml sql-gateway.endpoint.type: hiveserver2 ``` {{< hint info >}} -Notice: The CLI command has higher priority if flink-conf.yaml also contains the option `sql-gateway.endpoint.type`. +Notice: The CLI command has higher priority if [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) also contains the option `sql-gateway.endpoint.type`. {{< /hint >}} For the specific endpoint, please refer to the corresponding page. diff --git a/docs/content/docs/dev/table/sqlClient.md b/docs/content/docs/dev/table/sqlClient.md index deea6381967..95e17101144 100644 --- a/docs/content/docs/dev/table/sqlClient.md +++ b/docs/content/docs/dev/table/sqlClient.md @@ -84,7 +84,7 @@ Cache-Control: no-cache EOF) ``` -By default, the SQL Client will use the truststore configured using the `security.ssl.rest.truststore` and `security.ssl.rest.truststore-password` properties in the `flink-conf.yaml` file on the SQL client side. If these properties aren't explicitly configured, the client will use the default certificate stores provided by the JDK. +By default, the SQL Client will use the truststore configured using the `security.ssl.rest.truststore` and `security.ssl.rest.truststore-password` properties in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) file on the SQL client side. If these properties aren't explicitly configured, the client will use the default certificate stores provided by the JDK. <span class="label label-danger">Note</span> SQL Client only supports connecting to the [REST Endpoint]({{< ref "docs/dev/table/sql-gateway/rest" >}}#rest-api) since version v2. diff --git a/docs/content/docs/ops/debugging/flame_graphs.md b/docs/content/docs/ops/debugging/flame_graphs.md index 8a2a99c697e..9f9e3434ab9 100644 --- a/docs/content/docs/ops/debugging/flame_graphs.md +++ b/docs/content/docs/ops/debugging/flame_graphs.md @@ -48,7 +48,7 @@ Operator's On-CPU Flame Graph {{< hint warning >}} -Any measurement process in and of itself inevitably affects the subject of measurement (see the [double-split experiment](https://en.wikipedia.org/wiki/Double-slit_experiment#Relational_interpretation)). Sampling CPU stack traces is no exception. In order to prevent unintended impacts on production environments, Flame Graphs are currently available as an opt-in feature. To enable it, you'll need to set [`rest.flamegraph.enabled: true`]({{< ref "docs/deployment/config">}}#rest-flamegraph- [...] +Any measurement process in and of itself inevitably affects the subject of measurement (see the [double-split experiment](https://en.wikipedia.org/wiki/Double-slit_experiment#Relational_interpretation)). Sampling CPU stack traces is no exception. In order to prevent unintended impacts on production environments, Flame Graphs are currently available as an opt-in feature. To enable it, you'll need to set [`rest.flamegraph.enabled: true`]({{< ref "docs/deployment/config">}}#rest-flamegraph- [...] {{< /hint >}} diff --git a/docs/content/docs/ops/metrics.md b/docs/content/docs/ops/metrics.md index c2cddc4a6a8..2756b92171e 100644 --- a/docs/content/docs/ops/metrics.md +++ b/docs/content/docs/ops/metrics.md @@ -492,7 +492,7 @@ Every metric is assigned an identifier and a set of key-value pairs under which The identifier is based on 3 components: a user-defined name when registering the metric, an optional user-defined scope and a system-provided scope. For example, if `A.B` is the system scope, `C.D` the user scope and `E` the name, then the identifier for the metric will be `A.B.C.D.E`. -You can configure which delimiter to use for the identifier (default: `.`) by setting the `metrics.scope.delimiter` key in `conf/flink-conf.yaml`. +You can configure which delimiter to use for the identifier (default: `.`) by setting the `metrics.scope.delimiter` key in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ### User Scope @@ -551,7 +551,7 @@ counter = runtime_context \ The system scope contains context information about the metric, for example in which task it was registered or what job that task belongs to. -Which context information should be included can be configured by setting the following keys in `conf/flink-conf.yaml`. +Which context information should be included can be configured by setting the following keys in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). Each of these keys expect a format string that may contain constants (e.g. "taskmanager") and variables (e.g. "<task_id>") which will be replaced at runtime. - `metrics.scope.jm` diff --git a/docs/content/docs/ops/rest_api.md b/docs/content/docs/ops/rest_api.md index bc61beb1a2c..360e40bd937 100644 --- a/docs/content/docs/ops/rest_api.md +++ b/docs/content/docs/ops/rest_api.md @@ -35,7 +35,7 @@ The monitoring API is a REST-ful API that accepts HTTP requests and responds wit ## Overview -The monitoring API is backed by a web server that runs as part of the *JobManager*. By default, this server listens at port `8081`, which can be configured in `flink-conf.yaml` via `rest.port`. Note that the monitoring API web server and the web dashboard web server are currently the same and thus run together at the same port. They respond to different HTTP URLs, though. +The monitoring API is backed by a web server that runs as part of the *JobManager*. By default, this server listens at port `8081`, which can be configured in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) via `rest.port`. Note that the monitoring API web server and the web dashboard web server are currently the same and thus run together at the same port. They respond to different HTTP URLs, though. In the case of multiple JobManagers (for high availability), each JobManager will run its own instance of the monitoring API, which offers information about completed and running job while that JobManager was elected the cluster leader. diff --git a/docs/content/docs/ops/state/savepoints.md b/docs/content/docs/ops/state/savepoints.md index c5a4d1eee42..237eda7ba3d 100644 --- a/docs/content/docs/ops/state/savepoints.md +++ b/docs/content/docs/ops/state/savepoints.md @@ -303,7 +303,7 @@ Note that it is possible to also manually delete a savepoint via regular file sy You can configure a default savepoint target directory via the `state.savepoints.dir` key or `StreamExecutionEnvironment`. When triggering savepoints, this directory will be used to store the savepoint. You can overwrite the default by specifying a custom target directory with the trigger commands (see the [`:targetDirectory` argument](#trigger-a-savepoint)). {{< tabs "config" >}} -{{< tab "flink-conf.yaml" >}} +{{< tab "config.yaml" >}} ```yaml # Default savepoint target directory state.savepoints.dir: hdfs:///flink/savepoints diff --git a/docs/content/docs/ops/state/state_backends.md b/docs/content/docs/ops/state/state_backends.md index bd04491977f..085d10bd01f 100644 --- a/docs/content/docs/ops/state/state_backends.md +++ b/docs/content/docs/ops/state/state_backends.md @@ -110,7 +110,7 @@ take a savepoint with the new version, and only after that you can restore it wi ## Configuring a State Backend -The default state backend, if you specify nothing, is the jobmanager. If you wish to establish a different default for all jobs on your cluster, you can do so by defining a new default state backend in **flink-conf.yaml**. The default state backend can be overridden on a per-job basis, as shown below. +The default state backend, if you specify nothing, is the jobmanager. If you wish to establish a different default for all jobs on your cluster, you can do so by defining a new default state backend in [**Flink configuration file**]({{< ref "docs/deployment/config#flink-configuration-file" >}}). The default state backend can be overridden on a per-job basis, as shown below. ### Setting the Per-job State Backend @@ -151,13 +151,13 @@ If you want to use the `EmbeddedRocksDBStateBackend` in your IDE or configure it ``` {{< hint info >}} -Since RocksDB is part of the default Flink distribution, you do not need this dependency if you are not using any RocksDB code in your job and configure the state backend via `state.backend.type` and further [checkpointing]({{< ref "docs/deployment/config" >}}#checkpointing) and [RocksDB-specific]({{< ref "docs/deployment/config" >}}#rocksdb-state-backend) parameters in your `flink-conf.yaml`. +Since RocksDB is part of the default Flink distribution, you do not need this dependency if you are not using any RocksDB code in your job and configure the state backend via `state.backend.type` and further [checkpointing]({{< ref "docs/deployment/config" >}}#checkpointing) and [RocksDB-specific]({{< ref "docs/deployment/config" >}}#rocksdb-state-backend) parameters in your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). {{< /hint >}} ### Setting Default State Backend -A default state backend can be configured in the `flink-conf.yaml`, using the configuration key `state.backend.type`. +A default state backend can be configured in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}), using the configuration key `state.backend.type`. Possible values for the config entry are *hashmap* (HashMapStateBackend), *rocksdb* (EmbeddedRocksDBStateBackend), or the fully qualified class name of the class that implements the state backend factory {{< gh_link file="flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendFactory.java" name="StateBackendFactory" >}}, @@ -190,7 +190,7 @@ An incremental checkpoint builds upon (typically multiple) previous checkpoints. Recovery time of incremental checkpoints may be longer or shorter compared to full checkpoints. If your network bandwidth is the bottleneck, it may take a bit longer to restore from an incremental checkpoint, because it implies fetching more data (more deltas). Restoring from an incremental checkpoint is faster, if the bottleneck is your CPU or IOPs, because restoring from an incremental checkpoint means not re-building the local RocksDB tables from Flink's canonical key/value snapshot f [...] While we encourage the use of incremental checkpoints for large state, you need to enable this feature manually: - - Setting a default in your `flink-conf.yaml`: `state.backend.incremental: true` will enable incremental checkpoints, unless the application overrides this setting in the code. + - Setting a default in your [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}): `state.backend.incremental: true` will enable incremental checkpoints, unless the application overrides this setting in the code. - You can alternatively configure this directly in the code (overrides the config default): `EmbeddedRocksDBStateBackend backend = new EmbeddedRocksDBStateBackend(true);` Notice that once incremental checkpoont is enabled, the `Checkpointed Data Size` showed in web UI only represents the @@ -267,14 +267,14 @@ Flink offers sophisticated default [memory management for RocksDB](#memory-manag With *Predefined Options*, users can apply some predefined config profiles on each RocksDB Column Family, configuring for example memory use, thread, compaction settings, etc. There is currently one Column Family per each state in each operator. There are two ways to select predefined options to be applied: - - Set the option's name in `flink-conf.yaml` via `state.backend.rocksdb.predefined-options`. + - Set the option's name in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) via `state.backend.rocksdb.predefined-options`. - Set the predefined options programmatically: `EmbeddedRocksDBStateBackend.setPredefinedOptions(PredefinedOptions.SPINNING_DISK_OPTIMIZED_HIGH_MEM)`. The default value for this option is `DEFAULT` which translates to `PredefinedOptions.DEFAULT`. -Predefined options set programmatically would override the ones configured via `flink-conf.yaml`. +Predefined options set programmatically would override the ones configured via [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). -#### Reading Column Family Options from flink-conf.yaml +#### Reading Column Family Options from Flink configuration file RocksDB State Backend picks up all config options [defined here]({{< ref "docs/deployment/config" >}}#advanced-rocksdb-state-backends-options). Hence, you can configure low-level Column Family options simply by turning off managed memory for RocksDB and putting the relevant entries in the configuration. @@ -284,11 +284,11 @@ To manually control RocksDB's options, you need to configure an `RocksDBOptionsF There are two ways to pass a RocksDBOptionsFactory to the RocksDB State Backend: - - Configure options factory class name in the `flink-conf.yaml` via `state.backend.rocksdb.options-factory`. + - Configure options factory class name in the [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) via `state.backend.rocksdb.options-factory`. - Set the options factory programmatically, e.g. `EmbeddedRocksDBStateBackend.setRocksDBOptions(new MyOptionsFactory());` -Options factory which set programmatically would override the one configured via `flink-conf.yaml`, +Options factory which set programmatically would override the one configured via [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}), and options factory has a higher priority over the predefined options if ever configured or set. RocksDB is a native library that allocates memory directly from the process, @@ -475,7 +475,7 @@ Users can migrate existing applications to use the new API without losing any st The legacy `MemoryStateBackend` is equivalent to using [`HashMapStateBackend`](#the-hashmapstatebackend) and [`JobManagerCheckpointStorage`]({{< ref "docs/ops/state/checkpoints#the-jobmanagercheckpointstorage" >}}). -#### `flink-conf.yaml` configuration +#### Configuration using Flink configuration file ```yaml state.backend: hashmap @@ -517,7 +517,7 @@ env = StreamExecutionEnvironment.get_execution_environment(config) The legacy `FsStateBackend` is equivalent to using [`HashMapStateBackend`](#the-hashmapstatebackend) and [`FileSystemCheckpointStorage`]({{< ref "docs/ops/state/checkpoints#the-filesystemcheckpointstorage" >}}). -#### `flink-conf.yaml` configuration +#### Configuration using Flink configuration file ```yaml state.backend: hashmap @@ -579,7 +579,7 @@ env.configure(config); The legacy `RocksDBStateBackend` is equivalent to using [`EmbeddedRocksDBStateBackend`](#the-embeddedrocksdbstatebackend) and [`FileSystemCheckpointStorage`]({{< ref "docs/ops/state/checkpoints#the-filesystemcheckpointstorage" >}}). -#### `flink-conf.yaml` configuration +#### Configuration using Flink configuration file ```yaml state.backend: rocksdb diff --git a/docs/content/docs/ops/state/task_failure_recovery.md b/docs/content/docs/ops/state/task_failure_recovery.md index 6f444b15461..911796000f2 100644 --- a/docs/content/docs/ops/state/task_failure_recovery.md +++ b/docs/content/docs/ops/state/task_failure_recovery.md @@ -39,7 +39,7 @@ Failover strategies decide which tasks should be restarted to recover the job. The cluster can be started with a default restart strategy which is always used when no job specific restart strategy has been defined. In case that the job is submitted with a restart strategy, this strategy overrides the cluster's default setting. -The default restart strategy is set via Flink's configuration file `flink-conf.yaml`. +The default restart strategy is set via Flink's configuration file [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). The configuration parameter *restart-strategy.type* defines which strategy is taken. If checkpointing is not enabled, the "no restart" strategy is used. If checkpointing is activated and the restart strategy has not been configured, the fixed-delay strategy is used with @@ -96,7 +96,7 @@ The fixed delay restart strategy attempts a given number of times to restart the If the maximum number of attempts is exceeded, the job eventually fails. In-between two consecutive restart attempts, the restart strategy waits a fixed amount of time. -This strategy is enabled as default by setting the following configuration parameter in `flink-conf.yaml`. +This strategy is enabled as default by setting the following configuration parameter in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ```yaml restart-strategy.type: fixed-delay @@ -209,7 +209,7 @@ Still not supported in Python API. The failure rate restart strategy restarts job after failure, but when `failure rate` (failures per time interval) is exceeded, the job eventually fails. In-between two consecutive restart attempts, the restart strategy waits a fixed amount of time. -This strategy is enabled as default by setting the following configuration parameter in `flink-conf.yaml`. +This strategy is enabled as default by setting the following configuration parameter in [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). ```yaml restart-strategy.type: failure-rate @@ -301,7 +301,7 @@ By default, a fixed delay restart strategy is chosen if there is no other restar ## Failover Strategies Flink supports different failover strategies which can be configured via the configuration parameter -*jobmanager.execution.failover-strategy* in Flink's configuration file `flink-conf.yaml`. +*jobmanager.execution.failover-strategy* in Flink's configuration file [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}). <table class="table table-bordered"> <thead> diff --git a/docs/content/docs/try-flink/flink-operations-playground.md b/docs/content/docs/try-flink/flink-operations-playground.md index 4a2ecf0e017..cd93943c9ad 100644 --- a/docs/content/docs/try-flink/flink-operations-playground.md +++ b/docs/content/docs/try-flink/flink-operations-playground.md @@ -367,7 +367,7 @@ Suspending job "<job-id>" with a savepoint. Savepoint completed. Path: file:<savepoint-path> ``` -The Savepoint has been stored to the `state.savepoints.dir` configured in the *flink-conf.yaml*, +The Savepoint has been stored to the `state.savepoints.dir` configured in the [*Flink configuration file*]({{< ref "docs/deployment/config#flink-configuration-file" >}}), which is mounted under */tmp/flink-savepoints-directory/* on your local machine. You will need the path to this Savepoint in the next step. diff --git a/docs/content/docs/try-flink/local_installation.md b/docs/content/docs/try-flink/local_installation.md index a18c7a917dc..588d6d8e086 100644 --- a/docs/content/docs/try-flink/local_installation.md +++ b/docs/content/docs/try-flink/local_installation.md @@ -67,7 +67,7 @@ You should see something like: For now, you may want to note that: - __bin/__ directory contains the `flink` binary as well as several bash scripts that manage various jobs and tasks -- __conf/__ directory contains configuration files, including `flink-conf.yaml` +- __conf/__ directory contains configuration files, including [Flink configuration file]({{< ref "docs/deployment/config#flink-configuration-file" >}}) - __examples/__ directory contains sample applications that can be used as is with Flink
