[GitHub] flink issue #6396: [FLINK-9806][docs] Add canonical link element to docs
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/6396 The "hotfix" was a necessary modification to the dev-only helper scripts for building the docs locally. I assumed that it would just get squashed into the main commit when being merged, though I can do that too if it's required. I don't think it's really deserving of its own issue/PR. ---
[GitHub] flink pull request #6402: [FLINK-9914][docs] Update Docker docs
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/6402 [FLINK-9914][docs] Update Docker docs ## What is the purpose of the change Clarify the Docker-related docs for current releases. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-9914_update_docker_flink_docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/6402.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6402 commit be033162727950f6be384e9c5755947e37a09303 Author: Patrick Lucas Date: 2018-07-23T15:46:34Z [FLINK-9914][docs] Update Docker docs ---
[GitHub] flink pull request #6396: [FLINK-9806][docs] Add canonical link element to d...
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/6396 [FLINK-9806][docs] Add canonical link element to docs ## What is the purpose of the change Flink's documentation's SEO is a mess, with web searches often returning results from many Flink versions, usually with the latest not at the top. This change adds an HTML `link` tag with `rel="canonical"` to indicate to crawlers where to find the latest version of the page, i.e. served from `//ci.apache.org/projects/flink/flink-docs-stable/`. ## Brief change log - Update Docker image for building the docs to support the required ruby version - Add value to the docs config file to indicate the "stable" base URL - Add a `link` element with `rel="canonical"` to the base template so each page includes it with the appropriate value ## Notes Once this has been merged, we should port this change to the release branches whose docs are still available. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-9806_docs_canonical_url Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/6396.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6396 commit 2fec8ceafcb02d2024ab269c775418311661c9a3 Author: Patrick Lucas Date: 2018-07-23T14:24:08Z Use fedora for Docker docs build for newer Ruby The docs require Ruby >= 2.1.0 which isn't shipped by CentOS 7. It is available in Fedora 28, however, and I don't see any harm in switching to it. commit ff42e8a65ce0304ce194d3761aad446d16667eee Author: Patrick Lucas Date: 2018-07-23T15:17:14Z Add rel=canonical link to all pages ---
[GitHub] flink pull request #5380: [hotfix][connectors] Fix log format strings
GitHub user patricklucas opened a pull request:
https://github.com/apache/flink/pull/5380
[hotfix][connectors] Fix log format strings
Log4j uses '{}' instead of '%s' for interpolating values, meaning these
logging statements currently result in log entries with the literal string '%s'
instead of a useful value.
I ran into the one in `FlinkKafkaProducer011.java`, but did a brief search
in the `flink-connectors` module and found the others.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/patricklucas/flink
fix_connector_log_statements
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5380.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5380
commit 2e6c69f599921b970ec2ca9ac4bf7b6858d25189
Author: Patrick Lucas <me@...>
Date: 2018-01-29T12:58:09Z
[hotfix][connectors] Fix log format strings
Fixes three occurrences of using '%s' instead of '{}' in logging
statements.
---
[GitHub] flink pull request #4299: [FLINK-7155] [metrics] Add Influxdb reporter
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/4299#discussion_r126742152
--- Diff: flink-metrics/flink-metrics-influxdb/pom.xml ---
@@ -0,0 +1,97 @@
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;>
+
+ 4.0.0
+
+
+ org.apache.flink
+ flink-metrics
+ 1.4-SNAPSHOT
+ ..
+
+
+ flink-metrics-influxdb
+ flink-metrics-influxdb
+
+
+
+ org.apache.flink
+ flink-annotations
+ ${project.version}
+ provided
+
+
+
+ org.apache.flink
+ flink-metrics-core
+ ${project.version}
+ provided
+
+
+
+ org.apache.flink
+ flink-metrics-dropwizard
+ ${project.version}
+
+
+
+ io.dropwizard.metrics
+ metrics-core
+ ${metrics.version}
+
+
+
+ io.dropwizard.metrics
+ metrics-graphite
+ ${metrics.version}
+
+
+
+ com.github.davidb
+ metrics-influxdb
+ 0.9.3
+
+
+
+
+
+
+ maven-assembly-plugin
+ 2.4
+
+
+
jar-with-dependencies
--- End diff --
This is exactly how the graphite reporter does it, which is extremely
similar to this one. I'm not sure exactly what you want me to change here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4299: [FLINK-7155] [metrics] Add Influxdb reporter
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/4299#discussion_r126742051
--- Diff: flink-metrics/flink-metrics-influxdb/pom.xml ---
@@ -0,0 +1,97 @@
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;>
+
+ 4.0.0
+
+
+ org.apache.flink
+ flink-metrics
+ 1.4-SNAPSHOT
+ ..
+
+
+ flink-metrics-influxdb
+ flink-metrics-influxdb
+
+
+
+ org.apache.flink
+ flink-annotations
+ ${project.version}
+ provided
+
+
+
+ org.apache.flink
+ flink-metrics-core
+ ${project.version}
+ provided
+
+
+
+ org.apache.flink
+ flink-metrics-dropwizard
+ ${project.version}
+
+
+
+ io.dropwizard.metrics
+ metrics-core
+ ${metrics.version}
+
+
+
+ io.dropwizard.metrics
+ metrics-graphite
--- End diff --
This was errant; I'll remove it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4299: [FLINK-7155] [metrics] Add Influxdb reporter
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/4299 [FLINK-7155] [metrics] Add Influxdb reporter You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-7155_add_influxdb_metrics_reporter Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4299.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4299 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3887: [hotfix] [docs] Fix link to docker-compose.yml
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3887 [hotfix] [docs] Fix link to docker-compose.yml You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink hotfix_update_docker_compose_link Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3887.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3887 commit 1304a455095af43ca332aa315ec01910efd8088e Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-05-12T16:26:28Z [hotfix] [docs] Fix link to docker-compose.yml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3751: [FLINK-6330] [docs] Add basic Docker, K8s docs
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3751#discussion_r115753699
--- Diff: docs/page/css/syntax.css ---
@@ -77,3 +77,6 @@
.highlight .vg { color: #008080 } /* Name.Variable.Global */
.highlight .vi { color: #008080 } /* Name.Variable.Instance */
.highlight .il { color: #00 } /* Literal.Number.Integer.Long */
+
+/* Missing YAML styles */
+.highlight .l-Scalar-Plain { color: #008080 }
--- End diff --
Yeah, the highlighter we use doesn't have any built-in styling for YAML. I
had to search around quite a bit for context and came up mostly dry.
Moreover, the actual tokenizer it uses only emits two token types,
l-Scalar-Plain and p-Indicator. (You can see this in action by inspecting the
HTML
[here](https://ci.apache.org/projects/flink/flink-docs-release-1.2/monitoring/metrics.html#jmx-orgapacheflinkmetricsjmxjmxreporter))
This change applies the color I found most applicable from the existing
highlighting rules and applies it to l-Scalar-Plain to make it at least
slightly more interesting than black-on-white.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3751: [FLINK-6330] [docs] Add basic Docker, K8s docs
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3751 The link is available now! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3751: [FLINK-6330] [docs] Add basic Docker, K8s docs
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3751 @greghogan we're waiting patiently for @tianon to complete his review of docker-library/official-images#2895. :) As far as I know, we've done everything we need to do to get the images published. We have a +1 from @yosifkit, but two reviews are required. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3751: [FLINK-6330] [docs] Add basic Docker, K8s docs
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3751 It will be available before the next release, though I noticed just yesterday that it seems we build the [official 1.2 docs](https://ci.apache.org/projects/flink/flink-docs-release-1.2/) (erroneously?) from a development branch instead of the actual tagged release. I'm about ready to submit the PR to get the official images created, but it's fine if you want to wait. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3719: [FLINK-6303] Documentation support build in docker on OSX
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3719 Here's the PR: #3751 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3751: [FLINK-6330] [docs] Add basic Docker, K8s docs
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3751 [FLINK-6330] [docs] Add basic Docker, K8s docs I may do some more work on these before 1.2.2/1.3, but I'd like to at least get this basic information into master. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-6330_improve_docker_docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3751.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3751 commit 6894b3711fa1ab145f1ef871bcdcd74e7c7e9a6b Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-04-21T13:00:53Z [FLINK-6330] [docs] Add basic Docker, K8s docs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3719: [FLINK-6303] Documentation support build in docker on OSX
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3719 Thanks for the PR @mtunique! I actually ran into this today and have a fix [in a branch](https://github.com/patricklucas/flink/tree/FLINK-6330_improve_docker_docs) that I will hopefully open a PR for tomorrow. I had exactly your change, but tweaked it slightly to tolerate being called from sudo. You could either just wait for me to merge that change, or continue with this PR, but please implement the change as I did in the branch I linked above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3734: [FLINK-6300] Use 'exec' in start-foreground calls
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3734 Great, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3734: [FLINK-6300] Use 'exec' in start-foreground calls
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3734 @greghogan do you think you could merge this? (also to 1.2.1) If not I can try to rope in someone else. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3734: [FLINK-6300] Use 'exec' in start-foreground calls
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3734 cc @greghogan, fyi --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3734: [FLINK-6300] Use 'exec' in start-foreground calls
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3734 [FLINK-6300] Use 'exec' in start-foreground calls To avoid signal-handling issues in Docker, applications need to run as a single executable or use a process manager that forwards signals correctly, in either case running as PID 1. Since Flink uses a number of chained scripts before the ultimate call to `java`, we need to use `exec` so that the script executable is replaced, ultimately resulting in a single `java` process as PID 1. There's no need to run a process manager since Flink only actually requires a single process. **Note:** As there has not been a Flink release since these codepaths were introduced, I'm comfortable with merging this change for 1.2.1 since my manual testing was successful. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-6300_docker_pid_1_fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3734.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3734 commit 3be49de04c239083a5f050bf891834fc8abc12c9 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-04-18T12:48:49Z [FLINK-6300] Use 'exec' in start-foreground calls To avoid signal-handling issues in Docker, applications need to run as a single executable or use a process manager that forwards signals correctly, in either case running as PID 1. Since Flink uses a number of chained scripts before the ultimate call to `java`, we need to use `exec` so that the script executable is replaced, ultimately resulting in a single `java` process as PID 1. There's no need to run a process manager since Flink only actually requires a single process. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3686: [FLINK-6271][jdbc]Fix nullPointer when there's a single s...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3686 @fpompermaier FYI your author email address on this commit was "f.pomperma...@gmai.com" (missing 'l'). You might have a typo in your gitconfig? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3658: [hotfix] [docs] Remove empty Docker docs page
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3658 [hotfix] [docs] Remove empty Docker docs page This will be re-added when we have a coherent plan for Docker support and after the official Docker images are available. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink remove_empty_docker_docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3658.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3658 commit a0b7c4b7e9a286060909cd0c220d8a0c32d25807 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-03-31T15:00:21Z [hotfix] [docs] Remove empty Docker docs page This will be re-added when we have a coherent plan for Docker support and after the official Docker images are available. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3618: [FLINK-6194] [docs] Fix broken links
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3618 [FLINK-6194] [docs] Fix broken links Fix automatically-identified broken links in docs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-6194_fix_broken_links Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3618.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3618 commit fa3cff9342ebd8ccd2088755657e9bbfcec43815 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-03-27T09:10:25Z Fix broken links --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3494: [FLINK-5635] [docker] Improve Docker tooling
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3494#discussion_r105119473
--- Diff: flink-contrib/docker-flink/Dockerfile ---
@@ -36,22 +31,24 @@ ENV PATH $PATH:$FLINK_HOME/bin
EXPOSE 8081
EXPOSE 6123
+# flink-dist can point to a directory or a tarball on the local system
+ARG flink_dist=NOT_SET
+
# Install build dependencies and flink
+ADD $flink_dist $FLINK_INSTALL_PATH
RUN set -x && \
- mkdir -p $FLINK_INSTALL_PATH && \
- apk --update add --virtual build-dependencies curl && \
- curl -s $(curl -s
https://www.apache.org/dyn/closer.cgi\?preferred\=true)flink/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala_${SCALA_VERSION}.tgz
| \
--- End diff --
I agree. I'll comment on FLINK-3026 with a summary of this discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3494: [FLINK-5635] [docker] Improve Docker tooling
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3494#discussion_r104986831
--- Diff: flink-contrib/docker-flink/Dockerfile ---
@@ -36,22 +31,24 @@ ENV PATH $PATH:$FLINK_HOME/bin
EXPOSE 8081
EXPOSE 6123
+# flink-dist can point to a directory or a tarball on the local system
+ARG flink_dist=NOT_SET
+
# Install build dependencies and flink
+ADD $flink_dist $FLINK_INSTALL_PATH
RUN set -x && \
- mkdir -p $FLINK_INSTALL_PATH && \
- apk --update add --virtual build-dependencies curl && \
- curl -s $(curl -s
https://www.apache.org/dyn/closer.cgi\?preferred\=true)flink/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala_${SCALA_VERSION}.tgz
| \
--- End diff --
I see, just a terminology thing: image refers to the binary blob Docker Hub
stores for download, in the Flink repo is a Dockerfile for generating that
image. :)
This is mainly to follow suit with the other Apache projects I have found
with official images; they all tend to host their Dockerfiles totally
separately. Needless to say, the repo will be wholly licensed under the Apache
license, just not maintained under the ASF.
And we're definitely committed to maintaining these official images going
forward, updating them as a part of the Flink release process. If they need an
individual maintainer point of contact, that can be me and I can hand it off
later if I have to.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3494: [FLINK-5635] [docker] Improve Docker tooling
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3494#discussion_r104982158
--- Diff: flink-contrib/docker-flink/Dockerfile ---
@@ -36,22 +31,24 @@ ENV PATH $PATH:$FLINK_HOME/bin
EXPOSE 8081
EXPOSE 6123
+# flink-dist can point to a directory or a tarball on the local system
+ARG flink_dist=NOT_SET
+
# Install build dependencies and flink
+ADD $flink_dist $FLINK_INSTALL_PATH
RUN set -x && \
- mkdir -p $FLINK_INSTALL_PATH && \
- apk --update add --virtual build-dependencies curl && \
- curl -s $(curl -s
https://www.apache.org/dyn/closer.cgi\?preferred\=true)flink/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala_${SCALA_VERSION}.tgz
| \
--- End diff --
Which existing image are you referring to that we need to keep updated? I'm
not aware of a Flink project- or data Artisans-maintained image on Docker Hub.
I'm almost done with the basic functionality for the first Alpine image
(and a script to template its Dockerfile) but there will be plenty of room for
improvements. Once the new repo is live I'll ping you and you can take a look.
I don't know if I need to be the "official maintainer", but the repo will
live under the [data Artisans GitHub org](https://github.com/dataArtisans), so
if not me someone at dA will likely handle contributions to that repo.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3493: [FLINK-3026] Publish the flink docker container to the do...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 Since this PR is no longer directly related to publishing official Docker images, could you change its title? (See my reply to your comment on #3494) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3494: [FLINK-5635] [docker] Improve Docker tooling
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3494#discussion_r104979246
--- Diff: flink-contrib/docker-flink/Dockerfile ---
@@ -36,22 +31,24 @@ ENV PATH $PATH:$FLINK_HOME/bin
EXPOSE 8081
EXPOSE 6123
+# flink-dist can point to a directory or a tarball on the local system
+ARG flink_dist=NOT_SET
+
# Install build dependencies and flink
+ADD $flink_dist $FLINK_INSTALL_PATH
RUN set -x && \
- mkdir -p $FLINK_INSTALL_PATH && \
- apk --update add --virtual build-dependencies curl && \
- curl -s $(curl -s
https://www.apache.org/dyn/closer.cgi\?preferred\=true)flink/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala_${SCALA_VERSION}.tgz
| \
--- End diff --
I spoke with @jgrier and we decided to take the route of creating a
separate repo for the official images with all the Dockerfile variants we need,
like most other projects have.
In the mean time, I think we can go ahead and merge this change (and yours
in #3493) as a general improvement for Flink developers and users who just want
a nice way to create Docker images from a release or their own Flink repo.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3494: [FLINK-5635] [docker] Improve Docker tooling
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3494#discussion_r104912987
--- Diff: flink-contrib/docker-flink/Dockerfile ---
@@ -36,22 +31,24 @@ ENV PATH $PATH:$FLINK_HOME/bin
EXPOSE 8081
EXPOSE 6123
+# flink-dist can point to a directory or a tarball on the local system
+ARG flink_dist=NOT_SET
+
# Install build dependencies and flink
+ADD $flink_dist $FLINK_INSTALL_PATH
RUN set -x && \
- mkdir -p $FLINK_INSTALL_PATH && \
- apk --update add --virtual build-dependencies curl && \
- curl -s $(curl -s
https://www.apache.org/dyn/closer.cgi\?preferred\=true)flink/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-hadoop${HADOOP_VERSION}-scala_${SCALA_VERSION}.tgz
| \
--- End diff --
That makes sense, I'll take a look at other official repos and come up with
something that preserves the functionality we're trying to have here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3493: [FLINK-3026] Publish the flink docker container to the do...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 Aha, I see. I didn't follow the chain up further. :) Then yes, I'm generally on board with following this convention. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3494: [FLINK-5635] [docker] Improve Docker tooling
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3494 cc @ex00 @tonycox @iemejia --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3493: [FLINK-3026] Publish the flink docker container to the do...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 I've opened my PR, #3494. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3494: [FLINK-5635] [docker] Improve Docker tooling
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3494 [FLINK-5635] [docker] Improve Docker tooling This PR supplants and extends @jgrier's #3205, updating with master, fixing merge conflicts, making use of the new `start-foreground` param, and applying some review feedback from #3205. When merged, #3205 can be closed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-5635_improve_docker_tooling Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3494.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3494 commit 1ca6229e71f3dad7c87d9d0a7c759998af8258f5 Author: Jamie Grier <jamie.gr...@gmail.com> Date: 2017-01-21T00:12:17Z Improvements for Docker on Flink experience. Modifying Dockerfile to build from local flink-dist as well as release URLs. Logging to stdout. Adding scripts to deploy seamlessly on Docker Swarm. Updating Docker Compose scripts to work correctly. Parameterizing things so these Docker scripts are more generally useful. commit 8bad184d3a77259d84ebe4b34c8c28d61df5c8e3 Author: Jamie Grier <jamie.gr...@gmail.com> Date: 2017-01-25T05:45:23Z Adding Apache license to new files. commit afb8a71596d2e833cd905bc6669418b74da75b61 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-03-07T15:26:12Z Use start-foreground in Docker entrypoint commit 13f7826b9403ddb376716019a9142d869e559781 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-03-07T16:22:21Z docker-entrypoint.sh should error on invalid args commit 305796df1c9f5bc81ed3e320c0aba7afed5a1d61 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-03-07T16:39:35Z Improve docker build.sh cleanup commit 545226dfbbc9a2248f34421d3ef1de572b9ad5f7 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-03-08T10:51:12Z Dockerfile improvements per review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3493: [FLINK-3026] Publish the flink docker container to the do...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 I took a look through the [top official Docker](https://hub.docker.com/explore/) repos to see what variants they provide, and the first-listed Java-based project is [Elasticsearch](https://hub.docker.com/_/elasticsearch/). They have two variants for each of their supported version, alpine and non-alpine which are based on openjdk:8-jre and openjdk:8-jre-alpine respectively. Can you point me at other official repos basing on debian/ubuntu images? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3493: [FLINK-3026] Publish the flink docker container to the do...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3493 Thanks @iemejia. I have a branch building on @jgrier's commits from #3205 which I hope to push before any further changes are made. Hopefully I can get it done today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3204: [FLINK-5634] Flink should not always redirect stdout to a...
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3204 I vote to close this issue since #3492 has been merged/FLINK-4326 has been resolved. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3492: [FLINK-4326] [scripts] Flink foreground services
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3492 [FLINK-4326] [scripts] Flink foreground services This PR consists of @greghogan's commits from #3351 with the merge conflicts fixed. If merged, this PR supplants #3351, which can be closed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-4326_start_in_foreground Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3492.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3492 commit 0867c868508183e2c5b29f8f695fd13dd9f07ea3 Author: Greg Hogan <c...@greghogan.com> Date: 2016-10-07T20:06:48Z [FLINK-4326] [scripts] Flink foreground services Add a "start-foreground" option to the Flink service scripts which does not daemonize the service nor redirect output. commit 41ab9b7e905d6bf7fbf647808f54888923b3257f Author: Greg Hogan <c...@greghogan.com> Date: 2017-02-21T17:37:04Z Add logging configuration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3351: [FLINK-4326] [scripts] Flink foreground services
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3351 I fixed the merge conflicts and some indentation here: https://github.com/patricklucas/flink/tree/FLINK-4326_start_in_foreground If you update your PR with those changes, hopefully we can get this pushed soon to unblock some of the Docker-related improvements. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3355: [FLINK-5842] [docs] Fix ES5 "since" version
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3355 [FLINK-5842] [docs] Fix ES5 "since" version You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-5842_fix_es5_since_version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3355 commit a671b6eb87109ba874bd81af6ee62ffaffe06f25 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-02-19T18:56:30Z [FLINK-5842] [docs] Fix ES5 "since" version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3332: [FLINK-5751] [docs] Add link check script
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3332#discussion_r101786279
--- Diff: docs/check_links.sh ---
@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+target=${1:-"http://localhost:4000"}
+
+# Crawl the docs, ignoring robots.txt, storing nothing locally
+wget --spider -r -nd -nv -e robots=off -p -o spider.log "$target"
+
+# Abort for anything other than 0 and 4 ("Network failure")
+status=$?
+if [ $status -ne 0 ] && [ $status -ne 4 ]; then
+exit $status
+fi
+
+# Fail the build if any broken links are found
+broken_links_str=$(grep -e 'Found [[:digit:]]\+ broken link(s)' spider.log)
+if [ -n "$broken_links_str" ]; then
+ echo -e "\e[1;31m$broken_links_str\e[0m"
--- End diff --
Mixed tabs/spaces in this file if you want to do a no-review follow-up
commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3332: [FLINK-5751] [docs] Add link check script
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3332 :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3332: [FLINK-5751] [docs] Add link check script
Github user patricklucas commented on a diff in the pull request: https://github.com/apache/flink/pull/3332#discussion_r101678217 --- Diff: docs/check_links.sh --- @@ -0,0 +1,39 @@ +#!/usr/bin/env bash + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Don't abort on any non-zero exit code +#set +e --- End diff -- This is a vestige from me writing this as a Jenkins jobs; you can remove this line and the above comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3332: [FLINK-5751] [docs] Add link check script
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3332#discussion_r101678631
--- Diff: docs/check_links.sh ---
@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+# Don't abort on any non-zero exit code
+#set +e
+
+target=${1:-"http://localhost:4000"}
+
+# Crawl the docs, ignoring robots.txt, storing nothing locally
+wget --spider -r -nd -nv -e robots=off -p -o spider.log $target
+
+# Abort for anything other than 0 and 4 ("Network failure")
+status=$?
+if [ $status -ne 0 ] && [ $status -ne 4 ]; then
+exit $status
+fi
+
+# Fail the build if any broken links are found
+broken_links_str=$(grep -e 'Found [[:digit:]]\+ broken link' spider.log)
+echo -e "$broken_links_str"
--- End diff --
If you're going to get rid of my pretty bold-red formatting you can drop
the `-e` too. :)
Actually, might as well put the `echo` line inside the `if` too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3332: [FLINK-5751] [docs] Add link check script
Github user patricklucas commented on a diff in the pull request:
https://github.com/apache/flink/pull/3332#discussion_r101678241
--- Diff: docs/check_links.sh ---
@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+# Don't abort on any non-zero exit code
+#set +e
+
+target=${1:-"http://localhost:4000"}
+
+# Crawl the docs, ignoring robots.txt, storing nothing locally
+wget --spider -r -nd -nv -e robots=off -p -o spider.log $target
--- End diff --
$target in quotes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3329: [FLINK-5751] Fix some broken links in docs
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3329 [FLINK-5751] Fix some broken links in docs I used a wget-based script to identify broken links then manually searched and corrected as many as I found. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-5751_fix_doc_404s Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3329.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3329 commit 90431e9977bb616bbdb5fc0511667ee069ad1c5d Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-02-16T01:14:43Z [FLINK-5751] Fix some broken links in docs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3268: [FLINK-5153] Support YARN application tags
Github user patricklucas closed the pull request at: https://github.com/apache/flink/pull/3268 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3268: [FLINK-5153] Support YARN application tags
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3268 I added the test (which of course also required reflection) to an existing test case. I figured that was the lesser of the two evils of requiring another test application be deployed and hiding a test within a (somewhat) unrelated one. It was a bit tricky to actually run the tests, but I verified the correct behavior with both hadoop < 2.4 and >= 2.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #3268: [FLINK-5153] Support YARN application tags
Github user patricklucas commented on the issue: https://github.com/apache/flink/pull/3268 I manually tested the behavior of this new feature as well as the other YARN application options touched by this change with various Hadoop versions (2.3/2.4/2.6) on a YARN cluster. The Travis builds have tended to fail by timing out after 50 min, but everything tests fine locally. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #3268: [FLINK-5153] Support YARN application tags
GitHub user patricklucas opened a pull request: https://github.com/apache/flink/pull/3268 [FLINK-5153] Support YARN application tags Adds a new config option `yarn.tags`, a comma-separated list of strings passed to YARN as application tags. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricklucas/flink FLINK-5153_support_yarn_application_tags Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3268.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3268 commit b4599e3aa3c10f86354336c258870703e91e1415 Author: Patrick Lucas <m...@patricklucas.com> Date: 2017-02-04T00:17:55Z [FLINK-5153] Support YARN application tags Adds a new config option `yarn.tags`, a comma-separated list of strings passed to YARN as application tags. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
