hequn8128 commented on a change in pull request #10436: [FLINK-14920] [flink-end-to-end-perf-tests] Set up environment to run performance e2e tests URL: https://github.com/apache/flink/pull/10436#discussion_r355155525
########## File path: tools/jenkins/init_env.py ########## @@ -0,0 +1,95 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +# Because of end-to-end-performance-test running in the cluster, so This init_env.py only contains flink on +# standalone env, doesn't contains the init environment of maven, java and jenkins. Defaultly maven, java and jenkins +# environment are ready in the cluster +# + +import os +from logger import logger +from utils import run_command +from restapi_common import execute_get + + +def init_standalone_env(host_list, user, source_path, dest_path): + for host in host_list: + cmd = "sudo su %s -c 'ssh %s \" rm -rf %s\"; scp -r %s %s@%s:%s'" % (user, host, dest_path, source_path, + user, host, dest_path) + logger.info("init_standalone_env cmd:%s" % cmd) + run_command(cmd) + + +def get_host_list(slave_file): + hostlist = [] + with open(slave_file) as file: + data = file.readline() + if not(data == "" or data.startswith("#")): + hostlist.append(data) + return hostlist + + +def package(flink_home): + cmd = "cd %s; mvn clean install -B -U -DskipTests -Drat.skip=true -Dcheckstyle.skip=true " % flink_home + status, output = run_command(cmd) + if status and output.find("BUILD SUCCESS") > 0: + return True + else: + return False + + +def get_target(flink_home): + cmd = "ls -lt %s/flink-dist/target/flink-*-bin/ |grep -v tar.gz" + status, output = run_command(cmd) + if status: + target_file = output.split("\n")[0] + return target_file, "%s/flink-dist/target/%s-bin/%s" % (flink_home, target_file, target_file) + else: + return "", "" + + +def update_conf_slaves(dest_path, slave_file): + cmd = "cp %s %s/conf/" % (slave_file, dest_path) + run_command(cmd) + + +def init_env(): + flink_home = os.getcwd() Review comment: The returned value may not be a path of flink home. How about get the path of current file and then navigate to the flink home? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
