gyfora commented on code in PR #698: URL: https://github.com/apache/flink-kubernetes-operator/pull/698#discussion_r1379947506
########## flink-autoscaler-standalone/README.md: ########## @@ -0,0 +1,79 @@ +# Flink Autoscaler Standalone + +## What's the autoscaler standalone? + +`Flink Autoscaler Standalone` is an implementation of `Flink Autoscaler`, it runs as +a separate java process. It computes the reasonable parallelism of all job vertices +by monitoring the metrics, such as: processing rate, busy time, etc. Please see +[FLIP-271](https://cwiki.apache.org/confluence/display/FLINK/FLIP-271%3A+Autoscaling) Review Comment: Here can we link the operator docs autoscaler page instead? Lot of things have changed in the Flip and that page will never be updated. ########## flink-autoscaler-standalone/README.md: ########## @@ -0,0 +1,79 @@ +# Flink Autoscaler Standalone + +## What's the autoscaler standalone? Review Comment: We should add a paragraph clarifying that this should only be used in non-kubernetes environments. For Kubernetes the Operator provides a full integration that is preferable. ########## flink-autoscaler-standalone/README.md: ########## @@ -0,0 +1,79 @@ +# Flink Autoscaler Standalone + +## What's the autoscaler standalone? + +`Flink Autoscaler Standalone` is an implementation of `Flink Autoscaler`, it runs as +a separate java process. It computes the reasonable parallelism of all job vertices +by monitoring the metrics, such as: processing rate, busy time, etc. Please see +[FLIP-271](https://cwiki.apache.org/confluence/display/FLINK/FLIP-271%3A+Autoscaling) +for an overview of how autoscaling works. + +`Flink Autoscaler Standalone` rescales flink job in-place by rescale api of +[FLIP-291](https://cwiki.apache.org/confluence/x/9pRbDg). Review Comment: Here again we should link the docs instead: https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/elastic_scaling/#externalized-declarative-resource-management ########## flink-autoscaler-standalone/pom.xml: ########## @@ -0,0 +1,207 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> +<project xmlns="http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.flink</groupId> + <artifactId>flink-kubernetes-operator-parent</artifactId> + <version>1.7-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <artifactId>flink-autoscaler-standalone</artifactId> + <name>Flink Autoscaler Standalone</name> + <packaging>jar</packaging> + + + <dependencies> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-autoscaler</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-runtime</artifactId> + <version>${flink.version}</version> + <exclusions> + <exclusion> + <artifactId>flink-rpc-akka-loader</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>flink-shaded-zookeeper-3</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>flink-queryable-state-client-java</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>snappy-java</artifactId> + <groupId>org.xerial.snappy</groupId> + </exclusion> + <exclusion> + <artifactId>flink-hadoop-fs</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>commons-compress</artifactId> + <groupId>org.apache.commons</groupId> + </exclusion> + <exclusion> + <artifactId>commons-text</artifactId> + <groupId>org.apache.commons</groupId> + </exclusion> + <exclusion> Review Comment: In this entire pom we have groupId / artifactId , in the reverse order. While syntactically correct this is not how it is generally written. Can you please change them? ########## flink-autoscaler-standalone/pom.xml: ########## @@ -0,0 +1,207 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> +<project xmlns="http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.flink</groupId> + <artifactId>flink-kubernetes-operator-parent</artifactId> + <version>1.7-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <artifactId>flink-autoscaler-standalone</artifactId> + <name>Flink Autoscaler Standalone</name> + <packaging>jar</packaging> + + + <dependencies> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-autoscaler</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-runtime</artifactId> + <version>${flink.version}</version> + <exclusions> + <exclusion> + <artifactId>flink-rpc-akka-loader</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>flink-shaded-zookeeper-3</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>flink-queryable-state-client-java</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>snappy-java</artifactId> + <groupId>org.xerial.snappy</groupId> + </exclusion> + <exclusion> + <artifactId>flink-hadoop-fs</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>commons-compress</artifactId> + <groupId>org.apache.commons</groupId> + </exclusion> + <exclusion> + <artifactId>commons-text</artifactId> + <groupId>org.apache.commons</groupId> + </exclusion> + <exclusion> + <artifactId>lz4-java</artifactId> + <groupId>org.lz4</groupId> + </exclusion> + </exclusions> + </dependency> + + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>${flink.version}</version> + <exclusions> + <exclusion> + <artifactId>flink-streaming-java</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + <exclusion> + <artifactId>flink-optimizer</artifactId> + <groupId>org.apache.flink</groupId> + </exclusion> + </exclusions> + </dependency> + + <!-- Logging --> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-api</artifactId> + <version>${slf4j.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.logging.log4j</groupId> + <artifactId>log4j-slf4j-impl</artifactId> + <version>${log4j.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.logging.log4j</groupId> + <artifactId>log4j-api</artifactId> + <version>${log4j.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.logging.log4j</groupId> + <artifactId>log4j-core</artifactId> + <version>${log4j.version}</version> + </dependency> + + <dependency> + <groupId>org.apache.logging.log4j</groupId> + <artifactId>log4j-1.2-api</artifactId> + <version>${log4j.version}</version> + </dependency> + + <!-- Test --> + + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-autoscaler</artifactId> + <version>${project.version}</version> + <type>test-jar</type> + <classifier>tests</classifier> + <scope>test</scope> + </dependency> + + <dependency> + <groupId>org.junit.jupiter</groupId> + <artifactId>junit-jupiter-api</artifactId> + <scope>test</scope> + </dependency> + + <dependency> + <groupId>org.junit.jupiter</groupId> + <artifactId>junit-jupiter-params</artifactId> + <scope>test</scope> + </dependency> + + <dependency> + <groupId>org.assertj</groupId> + <artifactId>assertj-core</artifactId> + <version>${assertj.version}</version> + <scope>test</scope> + </dependency> + + </dependencies> + + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-shade-plugin</artifactId> + <executions> + <!-- Run shade goal on package phase --> + <execution> Review Comment: As we are bundling dependencies into a father and we are likely going to publish that to maven (right?) . We need to add a NOTICE file to list the bundled dependencies by license -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
