Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm closed pull request #1681: MINIFICPP-1797 Python bootstrap (part 1) URL: https://github.com/apache/nifi-minifi-cpp/pull/1681 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1522706091
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,325 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import glob
+import os
+import platform
+import subprocess
+import sys
+import re
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": {"llvm"}})
+return ""
Review Comment:
I checked last week, and it was working as well as without the new python
bootstrap. (Which still doesn't build on my mac, but that's not your fault.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1521748017 ## bootstrap/py_bootstrap.bat: ## @@ -0,0 +1,29 @@ +@echo off + +REM Check if Python is installed +where python > nul 2>&1 +if %errorlevel% neq 0 ( +echo Python is not installed +exit /b 1 +) + +REM Check if venv module is available +python -m venv --help > nul 2>&1 Review Comment: good idea, https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/194c8f3ab46266f43fbdf314e96759679b83b84b -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1521747343 ## Windows.md: ## @@ -120,9 +120,22 @@ A basic working CMake configuration can be inferred from the `win_build_vs.bat`. ``` mkdir build cd build -cmake -G "Visual Studio 17 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF -DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DUSE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DOPENSSL_OFF=OFF -DENABLE_COAP=OFF -DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DFORCE_WINDOWS=ON -DUSE_SYSTEM_UUID=OFF -DDISABLE_LIBARCHIVE=OFF -DENABLE_WEL=ON -DFAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. +cmake -G "Visual Studio 17 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF -DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DMINIFI_USE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DMINIFI_OPENSSL=ON -DENABLE_COAP=OFF -DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DUSE_SYSTEM_UUID=OFF -DENABLE_LIBARCHIVE=ON -DENABLE_WEL=ON -DMINIFI_FAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. msbuild /m nifi-minifi-cpp.sln /property:Configuration=Release /property:Platform=x64 copy minifi_main\Release\minifi.exe minifi_main\ cpack ctest -C Release ``` + +## Python based bootstrapping (recommended) Review Comment: https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/194c8f3ab46266f43fbdf314e96759679b83b84b ## Windows.md: ## Review Comment: Good idea, https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/194c8f3ab46266f43fbdf314e96759679b83b84b We will probably need a followup PR anyways (removing the win_build_bat maybe even bootstrap.sh?) And fix the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1521347526 ## Windows.md: ## Review Comment: I think most of the file should go under manual bootstrap and build, including Building with Visual Studio, and Building via the build script. The first section about Requirements should also be updated, to recommend the python bootstrap instead of the win_build_vs.bat script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1520163594
##
bootstrap/py_bootstrap.bat:
##
@@ -0,0 +1,29 @@
+@echo off
+
+REM Check if Python is installed
+where python > nul 2>&1
+if %errorlevel% neq 0 (
+echo Python is not installed
+exit /b 1
+)
+
+REM Check if venv module is available
+python -m venv --help > nul 2>&1
Review Comment:
You could just remove the separate check step and handle the error when
calling it normally below.
##
Windows.md:
##
@@ -120,9 +120,22 @@ A basic working CMake configuration can be inferred from
the `win_build_vs.bat`.
```
mkdir build
cd build
-cmake -G "Visual Studio 17 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF
-DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DUSE_REAL_ODBC_TEST_DRIVER=OFF
-DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32
-DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DOPENSSL_OFF=OFF -DENABLE_COAP=OFF
-DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF
-DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF
-DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF
-DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF
-DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF
-DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF
-DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF
-DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DFORCE_WINDOWS=ON
-DUSE_SYSTEM_UUID=OFF -DDISABLE_LIBARCHIVE=OFF -DENABLE_WEL=ON
-DFAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF ..
+cmake -G "Visual Studio 17 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF
-DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF
-DMINIFI_USE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release
-DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF
-DENABLE_JNI=OFF -DMINIFI_OPENSSL=ON -DENABLE_COAP=OFF -DENABLE_AWS=OFF
-DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK=
-DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF
-DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON
-DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF
-DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF
-DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF
-DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF
-DBUILD_ROCKSDB=ON -DUSE_SYSTEM_UUID=OFF -DENABLE_LIBARCHIVE=ON -DENABLE_WEL=ON
-DMINIFI_FAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF ..
msbuild /m nifi-minifi-cpp.sln /property:Configuration=Release
/property:Platform=x64
copy minifi_main\Release\minifi.exe minifi_main\
cpack
ctest -C Release
```
+
+## Python based bootstrapping (recommended)
Review Comment:
This could go on top, and the old instructions could go under another
heading, like `## Alternative: Manual bootstrapping (advanced)`
##
bootstrap/py_bootstrap.sh:
##
@@ -0,0 +1,27 @@
+#!/bin/bash
+
+# Check if Python is installed
+if ! command -v python3 &>/dev/null; then
+echo "Python is not installed"
+exit 1
+fi
+
+# Check if virtualenv is installed
+if ! command -v python3 -m venv --help &>/dev/null; then
+echo "virtualenv is not installed"
+exit 1
+fi
+
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+VENV_DIR="$SCRIPT_DIR/venv"
+
+if [ -d "$VENV_DIR" ]; then
+source "$VENV_DIR/bin/activate"
+else
+echo "Creating virtualenv"
+python3 -m venv "$VENV_DIR"
+source "$VENV_DIR/bin/activate"
+pip install -r "$SCRIPT_DIR/requirements.txt"
+fi
Review Comment:
```suggestion
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
VENV_DIR="$SCRIPT_DIR/venv"
set -e
if [ -d "$VENV_DIR" ]; then
source "$VENV_DIR/bin/activate"
else
echo "Creating virtualenv"
if ! python3 -m venv "$VENV_DIR"; then
echo "Creating virtualenv failed. Is venv installed?"
exit 1
fi
source "$VENV_DIR/bin/activate"
pip install -r "$SCRIPT_DIR/requirements.txt"
fi
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1520154175 ## .github/workflows/ci.yml: ## @@ -75,12 +131,72 @@ jobs: timeout-minutes: 240 env: LUA_DIR: D:\a\nifi-minifi-cpp\nifi-minifi-cpp\.lua + WINDOWS_MINIFI_OPTIONS: >- +-DCMAKE_BUILD_TYPE=Release +-DBUILD_ROCKSDB=ON +-DBUILD_SHARED_LIBS=OFF +-DCI_BUILD=ON +-DCUSTOM_MALLOC=OFF +-DDOCKER_BUILD_ONLY=OFF +-DDOCKER_PUSH=OFF +-DDOCKER_SKIP_TESTS=ON +-DENABLE_ALL=OFF +-DENABLE_AWS=ON +-DENABLE_AZURE=ON +-DENABLE_BUSTACHE=OFF +-DENABLE_BZIP2=ON +-DENABLE_CIVET=ON +-DENABLE_COAP=OFF +-DENABLE_CONTROLLER=OFF Review Comment: good idea, https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/48ebdcec251fc04221d0cbcf750bc261eed9409f -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1520152166
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1514638068
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
+platform_system = platform.system()
+if platform_system == "Darwin":
+_install_with_brew(dependencies_str, no_confirm)
+elif platform_system == "Linux":
+distro_id = distro.id()
+if distro_id == "ubuntu":
+_install_with_apt(dependencies_str, no_confirm)
+elif "arch" in distro_id or "manjaro" in distro_id:
+_install_with_pacman(dependencies_str, no_confirm)
+elif "rocky" in distro_id:
+_install_with_dnf(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {distro_id} exiting")
+elif platform_system == "Windows":
+_install_with_winget(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {platform_system} exiting")
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> str:
+system_dependencies = {'patch': 'patch', 'make': 'make'}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = 'bison'
+system_dependencies['flex'] = 'flex'
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = 'libarchive'
+if minifi_options.is_enabled("ENABLE_PCAP"):
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1514586297 ## bootstrap/py_bootstrap.bat: ## @@ -0,0 +1,29 @@ +@echo off + +REM Check if Python is installed +where python > nul 2>&1 +if %errorlevel% neq 0 ( +echo Python is not installed +exit /b 1 +) + +REM Check if venv module is available +python -m venv --help > nul 2>&1 +if %errorlevel% neq 0 ( +echo venv module is not available +exit /b 1 +) + +set "SCRIPT_DIR=%~dp0" +set "VENV_DIR=%SCRIPT_DIR%venv" + +if exist "%VENV_DIR%" ( +call "%VENV_DIR%\Scripts\activate" +) else ( +echo Creating virtualenv +python -m venv "%VENV_DIR%" +call "%VENV_DIR%\Scripts\activate" Review Comment: I was especially confused, because there is an activate file in the same location without an extension, but it's a bash script. Thx for changing it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1514584816
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1980997954 > This looks interesting 樂 , does the qouted path look correct? Is it the same that vswhere returns? It doesn't exist. It seems to have stemmed from an earlier build directory, and I probably updated VS since generating it. My bad. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1979056150 > In the same Windows environment I had issues with `vswhere`, I also had issue with the compiler command line after installing vswhere with scoop and refusing the offered choco install. > > ``` > -- The C compiler identification is unknown > -- The CXX compiler identification is unknown > CMake Error at CMakeLists.txt:25 (project): > The CMAKE_C_COMPILER: > > C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe > > is not a full path to an existing compiler tool. > > Tell CMake where to find the compiler by setting either the environment > variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to > the compiler, or to the compiler name if it is in the PATH. > > > CMake Error at CMakeLists.txt:25 (project): > The CMAKE_CXX_COMPILER: > > C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe > > is not a full path to an existing compiler tool. > > Tell CMake where to find the compiler by setting either the environment > variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path > to the compiler, or to the compiler name if it is in the PATH. > > > -- Configuring incomplete, errors occurred! > ``` This looks interesting 樂 , does the qouted path look correct? Is it the same that vswhere returns? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1513040448
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1977041853 In the same Windows environment I had issues with `vswhere`, I also had issue with the compiler command line after installing vswhere with scoop and refusing the offered choco install. ``` -- The C compiler identification is unknown -- The CXX compiler identification is unknown CMake Error at CMakeLists.txt:25 (project): The CMAKE_C_COMPILER: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe is not a full path to an existing compiler tool. Tell CMake where to find the compiler by setting either the environment variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH. CMake Error at CMakeLists.txt:25 (project): The CMAKE_CXX_COMPILER: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe is not a full path to an existing compiler tool. Tell CMake where to find the compiler by setting either the environment variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH. -- Configuring incomplete, errors occurred! ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511474443
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511277613
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question), end=' ', flush=True)
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": set()})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511242466 ## .github/workflows/ci.yml: ## @@ -75,12 +131,72 @@ jobs: timeout-minutes: 240 env: LUA_DIR: D:\a\nifi-minifi-cpp\nifi-minifi-cpp\.lua + WINDOWS_MINIFI_OPTIONS: >- +-DCMAKE_BUILD_TYPE=Release +-DBUILD_ROCKSDB=ON +-DBUILD_SHARED_LIBS=OFF +-DCI_BUILD=ON +-DCUSTOM_MALLOC=OFF +-DDOCKER_BUILD_ONLY=OFF +-DDOCKER_PUSH=OFF +-DDOCKER_SKIP_TESTS=ON +-DENABLE_ALL=OFF +-DENABLE_AWS=ON +-DENABLE_AZURE=ON +-DENABLE_BUSTACHE=OFF +-DENABLE_BZIP2=ON +-DENABLE_CIVET=ON +-DENABLE_COAP=OFF +-DENABLE_CONTROLLER=OFF Review Comment: Now that you fixed its build, I think we should enable the controller even on Windows. ## bootstrap/py_bootstrap.bat: ## @@ -0,0 +1,29 @@ +@echo off + +REM Check if Python is installed +where python > nul 2>&1 +if %errorlevel% neq 0 ( +echo Python is not installed +exit /b 1 +) + +REM Check if venv module is available +python -m venv --help > nul 2>&1 +if %errorlevel% neq 0 ( +echo venv module is not available +exit /b 1 +) + +set "SCRIPT_DIR=%~dp0" +set "VENV_DIR=%SCRIPT_DIR%venv" + +if exist "%VENV_DIR%" ( +call "%VENV_DIR%\Scripts\activate" +) else ( +echo Creating virtualenv +python -m venv "%VENV_DIR%" +call "%VENV_DIR%\Scripts\activate" Review Comment: Aren't we supposed to call activate.bat here? Why does this even work? ## Windows.md: ## @@ -120,7 +120,7 @@ A basic working CMake configuration can be inferred from the `win_build_vs.bat`. ``` mkdir build cd build -cmake -G "Visual Studio 17 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF -DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DUSE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DOPENSSL_OFF=OFF -DENABLE_COAP=OFF -DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DFORCE_WINDOWS=ON -DUSE_SYSTEM_UUID=OFF -DDISABLE_LIBARCHIVE=OFF -DENABLE_WEL=ON -DFAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. +cmake -G "Visual Studio 16 2022" -A x64 -DINSTALLER_MERGE_MODULES=OFF -DTEST_CUSTOM_WEL_PROVIDER=OFF -DENABLE_SQL=OFF -DMINIFI_USE_REAL_ODBC_TEST_DRIVER=OFF -DCMAKE_BUILD_TYPE_INIT=Release -DCMAKE_BUILD_TYPE=Release -DWIN32=WIN32 -DENABLE_LIBRDKAFKA=OFF -DENABLE_JNI=OFF -DMINIFI_OPENSSL=ON -DENABLE_COAP=OFF -DENABLE_AWS=OFF -DENABLE_PDH= -DENABLE_AZURE=OFF -DENABLE_SFTP=OFF -DENABLE_SPLUNK= -DENABLE_GCP= -DENABLE_NANOFI=OFF -DENABLE_OPENCV=OFF -DENABLE_PROMETHEUS=OFF -DENABLE_ELASTICSEARCH= -DUSE_SHARED_LIBS=OFF -DENABLE_CONTROLLER=ON -DENABLE_BUSTACHE=OFF -DENABLE_COAP=OFF -DENABLE_ENCRYPT_CONFIG=OFF -DENABLE_GPS=OFF -DENABLE_LUA_SCRIPTING=OFF -DENABLE_MQTT=OFF -DENABLE_OPC=OFF -DENABLE_OPENWSMAN=OFF -DENABLE_OPS=OFF -DENABLE_PCAP=OFF -DENABLE_PYTHON_SCRIPTING= -DENABLE_SENSORS=OFF -DENABLE_USB_CAMERA=OFF -DBUILD_ROCKSDB=ON -DUSE_SYSTEM_UUID=OFF -DENABLE_LIBARCHIVE=ON -DENABLE_WEL=ON -DMINIFI_FAIL_ON_WARNINGS=OFF -DSKIP_TESTS=OFF .. Review Comment: On my machine, there's `Visual Studio 16 2019` and `Visual Studio 17 2022`, but no `Visual Studio 16`. Was this change made by accident? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1511239034 ## extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp: ## Review Comment: thx for fixing this, I also ran into this issue a few weeks ago. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1973123420 > An other issue I had with the minificontroller build was this on Windows: > > ``` > FAILED: bin/minificontroller.exe > cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_exe --intdir=controller\CMakeFiles\minificontroller.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests -- C:\PROGRA~1\MICROS~1\2022\COMMUN~1\VC\Tools\MSVC\1439~1.335\bin\Hostx64\x64\link.exe /nologo controller\CMakeFiles\minificontroller.dir\MiNiFiController.cpp.obj controller\CMakeFiles\minificontroller.dir\Controller.cpp.obj controller\CMakeFiles\minificontroller.dir\__\minifi_main\MainHelper.cpp.obj controller\CMakeFiles\minificontroller.dir\versioninfo.rc.res /out:bin\minificontroller.exe /implib:controller\minificontroller.lib /pdb:bin\minificontroller.pdb /version:0.0 /machine:x64 /INCREMENTAL:NO /subsystem:console bin\core-minifi.lib thirdparty\yaml-cpp-install\lib\yaml-cpp.lib thirdparty\zlib-install\lib\zlibstatic.lib _deps\spdlog-build\spdlog.lib _deps\fmt-build\fmt.lib thirdparty\libsodium-install\lib\sodium.lib date -tz.lib thirdparty\openssl-install\lib\libssl.lib thirdparty\openssl-install\lib\libcrypto.lib crypt32.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cmd.exe /C "cd /D D:\Projects\nifi-minifi-cpp\build\controller && cat D:/Projects/nifi-minifi-cpp/build/all.log"" > 'cat' is not recognized as an internal or external command, > operable program or batch file. > ``` I think this was caused by an old post build step, removed it in https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/9f39511a1a991454311992cbd5dbf4e128a27051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1972985944 > I created a clean new Windows VM and tried out the new python bootstrap to set up the environment. When it installed the dependencies and got into the build menu I exited the bootstrap, and tried if the build can be still run using the win_build_vs.bat script. The configuration was successful and the build started, but failed with the following issues: > > ``` > Performing configure step for 'openssl-external' > 'perl' is not recognized as an internal or external command, > operable program or batch file. > ``` > > ``` > Failure! build file wasn't produced. > Please read INSTALL.md and associated NOTES-* files. You may also have to > look over your available compiler tool chain or change your configuration. > > NASM not found - make sure it's installed and available on %PATH% > ``` > > Are the perl and NASM dependencies installed in build time using the bootstrap build command, or are they missing somehow? After I installed these 2 dependencies manually the build continued. They are installed but the choco installer doesnt add them to path. I didnt want to modify the path because strawberry ssl and nasm can mess up the path, so they are only added temporaraly to path during the bootstrap process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
lordgamez commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1966498436 I created a clean new Windows VM and tried out the new python bootstrap to set up the environment. When it installed the dependencies and got into the build menu I exited the bootstrap, and tried if the build can be still run using the win_build_vs.bat script. The configuration was successful and the build started, but failed with the following issues: ``` Performing configure step for 'openssl-external' 'perl' is not recognized as an internal or external command, operable program or batch file. ``` ``` Failure! build file wasn't produced. Please read INSTALL.md and associated NOTES-* files. You may also have to look over your available compiler tool chain or change your configuration. NASM not found - make sure it's installed and available on %PATH% ``` Are the perl and NASM dependencies installed in build time using the bootstrap build command, or are they missing somehow? After I installed these 2 dependencies manually the build continued. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1954026766 > Do we want to remove `/bootstrap.sh`? Or are we keeping it for a while until possible kinks are ironed out with the new bootstrap? I think the latter would be better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
lordgamez commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1954025505 An other issue I had with the minificontroller build was this on Windows: ``` FAILED: bin/minificontroller.exe cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_exe --intdir=controller\CMakeFiles\minificontroller.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests -- C:\PROGRA~1\MICROS~1\2022\COMMUN~1\VC\Tools\MSVC\1439~1.335\bin\Hostx64\x64\link.exe /nologo controller\CMakeFiles\minificontroller.dir\MiNiFiController.cpp.obj controller\CMakeFiles\minificontroller.dir\Controller.cpp.obj controller\CMakeFiles\minificontroller.dir\__\minifi_main\MainHelper.cpp.obj controller\CMakeFiles\minificontroller.dir\versioninfo.rc.res /out:bin\minificontroller.exe /implib:controller\minificontroller.lib /pdb:bin\minificontroller.pdb /version:0.0 /machine:x64 /INCREMENTAL:NO /subsystem:console bin\core-minifi.lib thirdparty\yaml-cpp-install\lib\yaml-cpp.lib thirdparty\zlib-install\lib\zlibstatic.lib _deps\spdlog-build\spdlog.lib _deps\fmt-build\fmt.lib thirdparty\libsodium-install\lib\sodium.lib date-t z.lib thirdparty\openssl-install\lib\libssl.lib thirdparty\openssl-install\lib\libcrypto.lib crypt32.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cmd.exe /C "cd /D D:\Projects\nifi-minifi-cpp\build\controller && cat D:/Projects/nifi-minifi-cpp/build/all.log"" 'cat' is not recognized as an internal or external command, operable program or batch file. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
lordgamez commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1953878314 > > I tried running this on windows 10, in a VS2022 developer command prompt. It prompted me to install the vs2022 build tools, which shouldn't be necessary with a full VS installation, and failed to run when I answered with no. ... > > Im curious does vswhere find this installation, and do you have the required packages installed? In theory if vswhere finds your installation and it satisfies the requirements (C++ with VC.ATL support) it shouldnt prompt for an install, but if it doesnt find it, it wont work. Because it needs a valid installation. I had a similar experience at first, then after I cancelled the vs buildtools install, uninstalled the build tools from the VS installer, and I saw my VS installation was out of date, so updated it just in case, then after retrying the bootstrapper it didn't want to try installing the build tools. Not sure if the VS update solved the issue or it was something else. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1946252397 > I tried running this on windows 10, in a VS2022 developer command prompt. It prompted me to install the vs2022 build tools, which shouldn't be necessary with a full VS installation, and failed to run when I answered with no.  Im curious does vswhere find this installation, and do you have the required packages installed? In theory if vswhere finds your installation and it satisfies the requirements (C++ with VC.ATL support) it shouldnt prompt for an install, but if it doesnt find it, it wont work. Because it needs a valid installation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
fgerlits commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1487979767
##
bootstrap/py_bootstrap.sh:
##
Review Comment:
To accomodate users who don't read instructions, we should first check if
`python` and `python-venv` are installed, and tell the user to install them if
they aren't.
Also, if `python` is installed but `python-venv` isn't, the script creates a
broken `venv` directory, so it won't work even after `python-venv` is installed
later. The `venv` directory should be removed (or not created) in this case.
##
bootstrap/cli.py:
##
Review Comment:
After a step has finished in `Step by step` mode, is it possible to return
to the `Step by step` menu instead of the main menu?
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
Review Comment:
prompts requiring a response should be printed without a newline:
```suggestion
print("{} [y/n]".format(question), end=' ', flush=True)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1929456214 I tried running this on windows 10, in a VS2022 developer command prompt. It prompted me to install the vs2022 build tools, which shouldn't be necessary with a full VS installation, and failed to run when I answered with no.  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1461704026
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
Review Comment:
Changed the default,
https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/0e43424c2a2f2567876534b06fa68ef005104874
But im not sure where (if at all) we should include this in the readme
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
lordgamez commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1445881214
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
+"One click build": do_one_click_build,
Review Comment:
I'm okay with this now, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1444692735
##
bootstrap/main.py:
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import tempfile
+
+import argparse
+import pathlib
+
+from cli import main_menu, do_one_click_build
+from minifi_option import parse_minifi_options
+from package_manager import get_package_manager
+
+if __name__ == '__main__':
+with tempfile.TemporaryDirectory() as cmake_cache_dir:
+parser = argparse.ArgumentParser()
+parser.add_argument('--noconfirm', action="store_true", default=False,
+help="Bypass any and all “Are you sure?”
messages.")
+parser.add_argument('--minifi_options', default="", help="Overrides
the default minifi options during the "
+ "initial
parsing")
+parser.add_argument('--cmake_options', default="", help="Appends this
to the final cmake command")
+parser.add_argument('--skip_compiler_install', action="store_true",
default=False,
+help="Skips the installation of the default
compiler")
Review Comment:
sry misunderstood the question, I see your point I changed that in
https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/faabed2da474a73ae6f3761eeccf128cd09426e3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1444528599
##
bootstrap/main.py:
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import tempfile
+
+import argparse
+import pathlib
+
+from cli import main_menu, do_one_click_build
+from minifi_option import parse_minifi_options
+from package_manager import get_package_manager
+
+if __name__ == '__main__':
+with tempfile.TemporaryDirectory() as cmake_cache_dir:
+parser = argparse.ArgumentParser()
+parser.add_argument('--noconfirm', action="store_true", default=False,
+help="Bypass any and all “Are you sure?”
messages.")
+parser.add_argument('--minifi_options', default="", help="Overrides
the default minifi options during the "
+ "initial
parsing")
+parser.add_argument('--cmake_options', default="", help="Appends this
to the final cmake command")
+parser.add_argument('--skip_compiler_install', action="store_true",
default=False,
+help="Skips the installation of the default
compiler")
Review Comment:
I don't really understand the answer, so I'll try to rephrase my question:
We should use `--minifi-options` instead of `--minifi_options`,
`--cmake-options` instead of `--cmake_options`, and `--skip-compiler-install`
instead of `--skip_compiler_install`, because in command line long options,
dashes are used as the word separator in most cases.
I'm not sure how cmake fits here? These long option names are parsed by
python, aren't they?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1440609170
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,325 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import glob
+import os
+import platform
+import subprocess
+import sys
+import re
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": {"llvm"}})
+return ""
Review Comment:
Sure, I was confused but as it turns out xcode should be installed already
if the user has access to brew. So we dont need to install anything here. It
should use the default compiler (what a hello world project would pick with
cmake). Can you check?
https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/6b7d1af1293bf49e121d32f90e6cd9617c78a68e#diff-0ebcdc25be3be8a77e8b404eb6962891695836a8ca6bfa65607a2622d114f35dR91
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1440609170
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,325 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import glob
+import os
+import platform
+import subprocess
+import sys
+import re
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": {"llvm"}})
+return ""
Review Comment:
Sure, I was confused but as it turns out xcode should be installed already
if the user has access to brew. So we dont need to install anything here. It
should use the default compiler (what a hello world project would pick with
cmake). Can you check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1440607564
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,325 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import glob
+import os
+import platform
+import subprocess
+import sys
+import re
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+class PackageManager(object):
+def __init__(self, no_confirm):
+self.no_confirm = no_confirm
+pass
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+raise Exception("NotImplementedException")
+
+def install_compiler(self) -> str:
+raise Exception("NotImplementedException")
+
+def _install(self, dependencies: Dict[str, Set[str]], replace_dict:
Dict[str, Set[str]], install_cmd: str) -> bool:
+dependencies.update({k: v for k, v in replace_dict.items() if k in
dependencies})
+dependencies = self._filter_out_installed_packages(dependencies)
+dependencies_str = " ".join(str(value) for value_set in
dependencies.values() for value in value_set)
+if not dependencies_str or dependencies_str.isspace():
+return True
+return _run_command_with_confirm(f"{install_cmd} {dependencies_str}",
self.no_confirm)
+
+def _get_installed_packages(self) -> Set[str]:
+raise Exception("NotImplementedException")
+
+def _filter_out_installed_packages(self, dependencies: Dict[str,
Set[str]]):
+installed_packages = self._get_installed_packages()
+filtered_packages = {k: (v - installed_packages) for k, v in
dependencies.items()}
+for installed_package in installed_packages:
+filtered_packages.pop(installed_package, None)
+return filtered_packages
+
+def run_cmd(self, cmd: str) -> bool:
+result = subprocess.run(f"{cmd}", shell=True, text=True)
+return result.returncode == 0
+
+
+class BrewPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="brew install",
+ replace_dict={"patch": set(),
+ "jni": {"maven"}})
+
+def install_compiler(self) -> str:
+self.install({"compiler": {"llvm"}})
+return ""
+
+def _get_installed_packages(self) -> Set[str]:
+result = subprocess.run(['brew', 'list'], text=True,
capture_output=True, check=True)
+lines = result.stdout.splitlines()
+lines = [line.split('@', 1)[0] for line in lines]
+return set(lines)
+
+
+class AptPackageManager(PackageManager):
+def __init__(self, no_confirm):
+PackageManager.__init__(self, no_confirm)
+
+def install(self, dependencies: Dict[str, Set[str]]) -> bool:
+return self._install(dependencies=dependencies,
+ install_cmd="sudo apt install -y",
+ replace_dict={"libarchive": {"liblzma-dev"},
+ "lua": {"liblua5.1-0-dev"},
+ "python": {"libpython3-dev"},
+ "libusb": {"libusb-1.0-0-dev",
"libusb-dev"},
+ "libpng": {"libpng-dev"},
+ "libpcap": {"libpcap-dev"},
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1440605842
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import Dict, Set
+
+from minifi_option import MinifiOptions
+from package_manager import PackageManager
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> Dict[str,
Set[str]]:
+system_dependencies = {'patch': {'patch'}, 'make': {'make'}}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = {'bison'}
+system_dependencies['flex'] = {'flex'}
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = {'libarchive'}
+if minifi_options.is_enabled("ENABLE_PCAP"):
+system_dependencies['libpcap'] = {'libpcap'}
+if minifi_options.is_enabled("ENABLE_USB_CAMERA"):
+system_dependencies['libusb'] = {'libusb'}
+system_dependencies['libpng'] = {'libpng'}
+if minifi_options.is_enabled("ENABLE_GPS"):
+system_dependencies['gpsd'] = {'gpsd'}
+if minifi_options.is_enabled("ENABLE_COAP"):
+system_dependencies['automake'] = {'automake'}
+system_dependencies['autoconf'] = {'autoconf'}
+system_dependencies['libtool'] = {'libtool'}
+if minifi_options.is_enabled("ENABLE_LUA_SCRIPTING"):
+system_dependencies['lua'] = {'lua'}
+if minifi_options.is_enabled("ENABLE_PYTHON_SCRIPTING"):
+system_dependencies['python'] = {'python'}
+if minifi_options.is_enabled("MINIFI_OPENSSL"):
+system_dependencies['openssl'] = {'perl'}
Review Comment:
I replaced this approach with only temporarly changing the path and not
deleting anything.
(I've also added NASM as a separate dependency)
https://github.com/apache/nifi-minifi-cpp/pull/1681/commits/e611b1e44e82a01e082c3b8a1099d2d7591d7401#diff-0ebcdc25be3be8a77e8b404eb6962891695836a8ca6bfa65607a2622d114f35dR235-R237
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1424686652
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import Dict, Set
+
+from minifi_option import MinifiOptions
+from package_manager import PackageManager
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> Dict[str,
Set[str]]:
+system_dependencies = {'patch': {'patch'}, 'make': {'make'}}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = {'bison'}
+system_dependencies['flex'] = {'flex'}
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = {'libarchive'}
+if minifi_options.is_enabled("ENABLE_PCAP"):
+system_dependencies['libpcap'] = {'libpcap'}
+if minifi_options.is_enabled("ENABLE_USB_CAMERA"):
+system_dependencies['libusb'] = {'libusb'}
+system_dependencies['libpng'] = {'libpng'}
+if minifi_options.is_enabled("ENABLE_GPS"):
+system_dependencies['gpsd'] = {'gpsd'}
+if minifi_options.is_enabled("ENABLE_COAP"):
+system_dependencies['automake'] = {'automake'}
+system_dependencies['autoconf'] = {'autoconf'}
+system_dependencies['libtool'] = {'libtool'}
+if minifi_options.is_enabled("ENABLE_LUA_SCRIPTING"):
+system_dependencies['lua'] = {'lua'}
+if minifi_options.is_enabled("ENABLE_PYTHON_SCRIPTING"):
+system_dependencies['python'] = {'python'}
+if minifi_options.is_enabled("MINIFI_OPENSSL"):
+system_dependencies['openssl'] = {'perl'}
Review Comment:
Wiping the Strawberry Perl bin directory is a bit too extreme IMO, possibly
ruining the user's existing installation. Would it be possible to only fix the
PATH envvar instead? Like maybe moving the Strawberry Perl bin directory to the
end of it, so the normal patch command is found first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1424034289
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import Dict, Set
+
+from minifi_option import MinifiOptions
+from package_manager import PackageManager
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> Dict[str,
Set[str]]:
+system_dependencies = {'patch': {'patch'}, 'make': {'make'}}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = {'bison'}
+system_dependencies['flex'] = {'flex'}
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = {'libarchive'}
+if minifi_options.is_enabled("ENABLE_PCAP"):
+system_dependencies['libpcap'] = {'libpcap'}
+if minifi_options.is_enabled("ENABLE_USB_CAMERA"):
+system_dependencies['libusb'] = {'libusb'}
+system_dependencies['libpng'] = {'libpng'}
+if minifi_options.is_enabled("ENABLE_GPS"):
+system_dependencies['gpsd'] = {'gpsd'}
+if minifi_options.is_enabled("ENABLE_COAP"):
+system_dependencies['automake'] = {'automake'}
+system_dependencies['autoconf'] = {'autoconf'}
+system_dependencies['libtool'] = {'libtool'}
+if minifi_options.is_enabled("ENABLE_LUA_SCRIPTING"):
+system_dependencies['lua'] = {'lua'}
+if minifi_options.is_enabled("ENABLE_PYTHON_SCRIPTING"):
+system_dependencies['python'] = {'python'}
+if minifi_options.is_enabled("MINIFI_OPENSSL"):
+system_dependencies['openssl'] = {'perl'}
Review Comment:
Thats packaged with strawberryperl, (with the whole toolchain but that
causes other problems). Usually we wipe the c:\Strawberry\c\bin dir to fix
those problems, but then we also wipe the NASM which is required, so we had to
install that aswell, instead Im deleting the installed toolchain but leave NASM
alone.
https://github.com/martinzink/nifi-minifi-cpp/blob/python_bootstrap/bootstrap/package_manager.py#L178
(installing NASM separatly came with its own issues, since neither choco nor
winget sets the PATH for NASM)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1424035886
##
bootstrap/main.py:
##
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import tempfile
+
+import argparse
+import pathlib
+
+from cli import main_menu, do_one_click_build
+from minifi_option import parse_minifi_options
+from package_manager import get_package_manager
+
+if __name__ == '__main__':
+with tempfile.TemporaryDirectory() as cmake_cache_dir:
+parser = argparse.ArgumentParser()
+parser.add_argument('--noconfirm', action="store_true", default=False,
+help="Bypass any and all “Are you sure?”
messages.")
+parser.add_argument('--minifi_options', default="", help="Overrides
the default minifi options during the "
+ "initial
parsing")
+parser.add_argument('--cmake_options', default="", help="Appends this
to the final cmake command")
+parser.add_argument('--skip_compiler_install', action="store_true",
default=False,
+help="Skips the installation of the default
compiler")
Review Comment:
We pass that directly to the first cmake command that parses the
minifioptions cmake file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1422556409
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import Dict, Set
+
+from minifi_option import MinifiOptions
+from package_manager import PackageManager
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> Dict[str,
Set[str]]:
+system_dependencies = {'patch': {'patch'}, 'make': {'make'}}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = {'bison'}
+system_dependencies['flex'] = {'flex'}
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = {'libarchive'}
+if minifi_options.is_enabled("ENABLE_PCAP"):
+system_dependencies['libpcap'] = {'libpcap'}
+if minifi_options.is_enabled("ENABLE_USB_CAMERA"):
+system_dependencies['libusb'] = {'libusb'}
+system_dependencies['libpng'] = {'libpng'}
+if minifi_options.is_enabled("ENABLE_GPS"):
+system_dependencies['gpsd'] = {'gpsd'}
+if minifi_options.is_enabled("ENABLE_COAP"):
+system_dependencies['automake'] = {'automake'}
+system_dependencies['autoconf'] = {'autoconf'}
+system_dependencies['libtool'] = {'libtool'}
+if minifi_options.is_enabled("ENABLE_LUA_SCRIPTING"):
+system_dependencies['lua'] = {'lua'}
+if minifi_options.is_enabled("ENABLE_PYTHON_SCRIPTING"):
+system_dependencies['python'] = {'python'}
+if minifi_options.is_enabled("MINIFI_OPENSSL"):
+system_dependencies['openssl'] = {'perl'}
Review Comment:
OpenSSL also needs NASM, at least on Windows. Is it handled elsewhere?
##
bootstrap/py_bootstrap.bat:
##
@@ -0,0 +1,15 @@
+@echo off
+
+set SCRIPT_DIR=%~dp0
+set VENV_DIR=%SCRIPT_DIR%venv
+
+if exist %VENV_DIR% (
+call %VENV_DIR%\Scripts\activate
+) else (
+echo Creating virtualenv
+python -m venv %VENV_DIR%
+call %VENV_DIR%\Scripts\activate
+pip install -r %SCRIPT_DIR%requirements.txt
+)
+
+python %SCRIPT_DIR%main.py
Review Comment:
I like to use quotes, even when they're not strictly required. Windows cmd
scripts have a strange syntax, so you usually need to quote the whole argument,
not just the part containing the variable reference, otherwise the quotes just
become part of the argument.
```suggestion
set "SCRIPT_DIR=%~dp0"
set "VENV_DIR=%SCRIPT_DIR%venv"
if exist "%VENV_DIR%" (
call "%VENV_DIR%\Scripts\activate"
) else (
echo Creating virtualenv
python -m venv "%VENV_DIR%"
call "%VENV_DIR%\Scripts\activate"
pip install -r "%SCRIPT_DIR%requirements.txt"
)
python "%SCRIPT_DIR%main.py"
```
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,325 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import glob
+import os
+import platform
+import subprocess
+import sys
+import re
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
lordgamez commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1422546555 ## bootstrap/system_dependency.py: ## @@ -0,0 +1,168 @@ +from __future__ import annotations + +import os +import platform +import sys +from typing import Dict + +import distro + +from minifi_option import MinifiOptions + + +def _query_yes_no(question: str, no_confirm: bool) -> bool: Review Comment: If it's minimal now I think that's okay -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1422537500
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
Review Comment:
We could do that. Automations could still explicitly set it to off if
needed. We should mention this in the readme if we change the default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1839101077 I've refactored pretty much the whole thing. Now it only installs the packages if they are not installed. Rewrote the CI to use the new bootstrapping process. On windows it only supports chocolatey, due to various problems with winget -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414218470
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
Review Comment:
Should we simply chnage the default in cmake/MiNiFiOptions.cmake or override
it in the bootstrap script regardless of its value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414216533 ## bootstrap/system_dependency.py: ## @@ -0,0 +1,168 @@ +from __future__ import annotations + +import os +import platform +import sys +from typing import Dict + +import distro + +from minifi_option import MinifiOptions + + +def _query_yes_no(question: str, no_confirm: bool) -> bool: Review Comment: Now that we dont reinstall the packages, one would only encounter this a limited amount of times, so this might be not neccessary. What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414213464
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
+platform_system = platform.system()
+if platform_system == "Darwin":
+_install_with_brew(dependencies_str, no_confirm)
+elif platform_system == "Linux":
+distro_id = distro.id()
+if distro_id == "ubuntu":
+_install_with_apt(dependencies_str, no_confirm)
+elif "arch" in distro_id or "manjaro" in distro_id:
+_install_with_pacman(dependencies_str, no_confirm)
+elif "rocky" in distro_id:
+_install_with_dnf(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {distro_id} exiting")
+elif platform_system == "Windows":
+_install_with_winget(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {platform_system} exiting")
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> str:
+system_dependencies = {'patch': 'patch', 'make': 'make'}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = 'bison'
+system_dependencies['flex'] = 'flex'
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = 'libarchive'
+if minifi_options.is_enabled("ENABLE_PCAP"):
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414212713
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
Review Comment:
Good idea, I've refactored the whole thing around this idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414212288
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
+platform_system = platform.system()
+if platform_system == "Darwin":
+_install_with_brew(dependencies_str, no_confirm)
+elif platform_system == "Linux":
+distro_id = distro.id()
+if distro_id == "ubuntu":
+_install_with_apt(dependencies_str, no_confirm)
+elif "arch" in distro_id or "manjaro" in distro_id:
+_install_with_pacman(dependencies_str, no_confirm)
+elif "rocky" in distro_id:
+_install_with_dnf(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {distro_id} exiting")
+elif platform_system == "Windows":
+_install_with_winget(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {platform_system} exiting")
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> str:
+system_dependencies = {'patch': 'patch', 'make': 'make'}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = 'bison'
+system_dependencies['flex'] = 'flex'
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = 'libarchive'
+if minifi_options.is_enabled("ENABLE_PCAP"):
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414211453 ## README.md: ## @@ -304,8 +304,30 @@ sudo brew install libpcap ### Bootstrapping -- MiNiFi C++ offers a bootstrap script in the root of our github repo that will bootstrap the cmake and build process for you without the need to install dependencies yourself. To use this - process, please run the command `bootstrap.sh` from the root of the MiNiFi C++ source tree. +MiNiFi C++ offers bootstrap scripts that will bootstrap the cmake and build process for you without the need to install dependencies yourself. + + Python based bootstrapping + +Set up a virtual environment in the bootstrap folder. +``` +python -m venv venv +source venv/bin/activate +pip install -r requirements.txt +``` + +After that you can run the bootstrap script that will guide you through the configuration, dependency installation and build processes. Review Comment: I've added wrapper scripts, so I've rephrased this section. https://github.com/apache/nifi-minifi-cpp/pull/1681/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R309 ## README.md: ## @@ -304,8 +304,30 @@ sudo brew install libpcap ### Bootstrapping -- MiNiFi C++ offers a bootstrap script in the root of our github repo that will bootstrap the cmake and build process for you without the need to install dependencies yourself. To use this - process, please run the command `bootstrap.sh` from the root of the MiNiFi C++ source tree. +MiNiFi C++ offers bootstrap scripts that will bootstrap the cmake and build process for you without the need to install dependencies yourself. + + Python based bootstrapping + +Set up a virtual environment in the bootstrap folder. +``` +python -m venv venv Review Comment: I've added wrapper scripts, so I've rephrased this section. https://github.com/apache/nifi-minifi-cpp/pull/1681/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414209680
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
+platform_system = platform.system()
+if platform_system == "Darwin":
+_install_with_brew(dependencies_str, no_confirm)
+elif platform_system == "Linux":
+distro_id = distro.id()
+if distro_id == "ubuntu":
+_install_with_apt(dependencies_str, no_confirm)
+elif "arch" in distro_id or "manjaro" in distro_id:
+_install_with_pacman(dependencies_str, no_confirm)
+elif "rocky" in distro_id:
+_install_with_dnf(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {distro_id} exiting")
+elif platform_system == "Windows":
+_install_with_winget(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {platform_system} exiting")
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> str:
+system_dependencies = {'patch': 'patch', 'make': 'make'}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = 'bison'
+system_dependencies['flex'] = 'flex'
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = 'libarchive'
+if minifi_options.is_enabled("ENABLE_PCAP"):
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414208743
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
+"One click build": do_one_click_build,
+"Step by step build": step_by_step_menu,
+"Exit": lambda options: True,
+}
+
+questions = [
+inquirer.List(
+"sub_menu",
+message="Main Menu",
+choices=[menu_option_name for menu_option_name in
main_menu_options],
+),
+]
+
+main_menu_prompt = inquirer.prompt(questions)
+done = main_menu_options[main_menu_prompt["sub_menu"]](minifi_options)
+
+
+def build_type_menu(minifi_options: MinifiOptions) -> bool:
+questions = [
+inquirer.List(
+"build_type",
+message="Build type",
+choices=minifi_options.build_type.possible_values,
+),
+]
+
+answers = inquirer.prompt(questions)
+minifi_options.build_type.value = answers["build_type"]
+return False
+
+
+def build_dir_menu(minifi_options: MinifiOptions) -> bool:
+questions = [
+inquirer.Path('build_dir',
+ message="Build directory",
+ default=minifi_options.build_dir
+ ),
+]
+minifi_options.build_dir = inquirer.prompt(questions)["build_dir"]
+return False
+
+
+def bool_menu(minifi_options: MinifiOptions) -> bool:
+possible_values = [option_name for option_name in
minifi_options.bool_options]
+selected_values = [option.name for option in
minifi_options.bool_options.values() if option.value == "ON"]
+questions = [
+inquirer.Checkbox(
+"options",
+message="MiNiFi C++ Options",
+choices=possible_values,
+default=selected_values
+),
+]
+
+answers = inquirer.prompt(questions)
+for bool_option in minifi_options.bool_options.values():
+if bool_option.name in answers["options"]:
+bool_option.value = "ON"
+else:
+bool_option.value = "OFF"
+
+return False
Review Comment:
I've added some clarifications to the title of this menu.
https://github.com/apache/nifi-minifi-cpp/pull/1681/files#diff-540b6c7fb2ac6e2c1e8c04096674ab473e4e16eaf2f06cff971b097ae65bc2cdR104
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414206664
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
+"One click build": do_one_click_build,
Review Comment:
I've added a feature so it only installs if there is some dependency
missing, so it wont be annoying, would that suffice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414205047
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
+"One click build": do_one_click_build,
+"Step by step build": step_by_step_menu,
+"Exit": lambda options: True,
+}
+
+questions = [
+inquirer.List(
+"sub_menu",
+message="Main Menu",
+choices=[menu_option_name for menu_option_name in
main_menu_options],
+),
+]
+
+main_menu_prompt = inquirer.prompt(questions)
+done = main_menu_options[main_menu_prompt["sub_menu"]](minifi_options)
+
+
+def build_type_menu(minifi_options: MinifiOptions) -> bool:
+questions = [
+inquirer.List(
+"build_type",
+message="Build type",
+choices=minifi_options.build_type.possible_values,
+),
+]
+
+answers = inquirer.prompt(questions)
+minifi_options.build_type.value = answers["build_type"]
+return False
+
+
+def build_dir_menu(minifi_options: MinifiOptions) -> bool:
+questions = [
+inquirer.Path('build_dir',
+ message="Build directory",
+ default=minifi_options.build_dir
+ ),
+]
+minifi_options.build_dir = inquirer.prompt(questions)["build_dir"]
+return False
+
+
+def bool_menu(minifi_options: MinifiOptions) -> bool:
Review Comment:
I think that would be possible, but since its not trivial I think I'd rather
do that in part 2 aswell.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414204309
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
Review Comment:
You are right, I've removed the assertions from these places
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414201337 ## README.md: ## @@ -304,8 +304,30 @@ sudo brew install libpcap ### Bootstrapping -- MiNiFi C++ offers a bootstrap script in the root of our github repo that will bootstrap the cmake and build process for you without the need to install dependencies yourself. To use this - process, please run the command `bootstrap.sh` from the root of the MiNiFi C++ source tree. +MiNiFi C++ offers bootstrap scripts that will bootstrap the cmake and build process for you without the need to install dependencies yourself. + + Python based bootstrapping + +Set up a virtual environment in the bootstrap folder. Review Comment: Good idea, I've created a wrapper https://github.com/apache/nifi-minifi-cpp/blob/02abe24dd97f8a160fd1fe0e2982e27cb7e5936e/bootstrap/py_bootstrap.bat https://github.com/apache/nifi-minifi-cpp/blob/02abe24dd97f8a160fd1fe0e2982e27cb7e5936e/bootstrap/py_bootstrap.sh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1414202235
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
Review Comment:
Good idea, that would be a good way to improve it in a followup PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1385294428
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
+platform_system = platform.system()
+if platform_system == "Darwin":
+_install_with_brew(dependencies_str, no_confirm)
+elif platform_system == "Linux":
+distro_id = distro.id()
+if distro_id == "ubuntu":
+_install_with_apt(dependencies_str, no_confirm)
+elif "arch" in distro_id or "manjaro" in distro_id:
+_install_with_pacman(dependencies_str, no_confirm)
+elif "rocky" in distro_id:
+_install_with_dnf(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {distro_id} exiting")
+elif platform_system == "Windows":
+_install_with_winget(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {platform_system} exiting")
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> str:
+system_dependencies = {'patch': 'patch', 'make': 'make'}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = 'bison'
+system_dependencies['flex'] = 'flex'
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = 'libarchive'
+if minifi_options.is_enabled("ENABLE_PCAP"):
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
lordgamez commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1389526709
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
+"One click build": do_one_click_build,
Review Comment:
I would love to see a once click build without checking for dependencies,
just cmake + build :)
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
Review Comment:
Could we change this to a `[Y/n]` functionality to have the default answer
yes when the Enter is pressed?
##
README.md:
##
@@ -304,8 +304,30 @@ sudo brew install libpcap
### Bootstrapping
-- MiNiFi C++ offers a bootstrap script in the root of our github repo that
will bootstrap the cmake and build process for you without the need to install
dependencies yourself. To use this
- process, please run the command `bootstrap.sh` from the root of the MiNiFi
C++ source tree.
+MiNiFi C++ offers bootstrap scripts that will bootstrap the cmake and build
process for you without the need to install dependencies yourself.
+
+ Python based bootstrapping
+
+Set up a virtual environment in the bootstrap folder.
Review Comment:
A wrapper script would be useful that creates the venv and starts the
bootstrap without manually taking care of that
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict =
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#issuecomment-1806163996
Got this on Windows, after I managed to install winget and accept the source
terms by running it once manually:
```
(venv) C:\a\m\bootstrap>python main.py
For CMake to work we need a working compiler
Running winget install --disable-interactivity --accept-package-agreements
msvc [y/n]
y
No package found matching input criteria.
Traceback (most recent call last):
File "C:\a\m\bootstrap\main.py", line 18, in
compiler_override = install_compiler(no_confirm)
File "C:\a\m\bootstrap\system_dependency.py", line 159, in install_compiler
_install("msvc", no_confirm)
File "C:\a\m\bootstrap\system_dependency.py", line 115, in _install
_install_with_winget(dependencies_str, no_confirm)
File "C:\a\m\bootstrap\system_dependency.py", line 97, in
_install_with_winget
assert _run_command_with_confirm(command, no_confirm)
AssertionError
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1386024240
##
bootstrap/system_dependency.py:
##
@@ -0,0 +1,168 @@
+from __future__ import annotations
+
+import os
+import platform
+import sys
+from typing import Dict
+
+import distro
+
+from minifi_option import MinifiOptions
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+return True
+while True:
+print("{} [y/n]".format(question))
+choice = input().lower()
+if choice in valid:
+return valid[choice]
+else:
+print("Please respond with 'yes' or 'no' " "(or 'y' or 'n').")
+
+
+def _get_system_identifier():
+platform_system = platform.system()
+if platform_system == "Linux":
+return distro.id()
+return platform_system
+
+
+def _replace_wholewords(target: str, replace_dict: Dict[str]):
+words = target.split()
+output_string = ' '.join(replace_dict.get(word, word) for word in words)
+return output_string
+
+
+def _run_command_with_confirm(command: str, no_confirm: bool) -> bool:
+if _query_yes_no("Running {}".format(command), no_confirm):
+return os.system(command) == 0
+
+
+def _install_with_brew(dependencies_str: str, no_confirm: bool):
+replace_dict = {"patch": "",
+"jni": "maven"}
+command = "brew install {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_apt(dependencies_str: str, no_confirm: bool):
+replace_dict = {"libarchive": "liblzma-dev",
+"lua": "liblua5.1-0-dev",
+"python": "libpython3-dev",
+"libusb": "libusb-1.0-0-dev libusb-dev",
+"libpng": "libpng-dev",
+"libpcap": "libpcap-dev",
+"jni": "openjdk-8-jdk openjdk-8-source maven"}
+
+command = "sudo apt install -y {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_dnf(dependencies_str: str, no_confirm: bool):
+replace_dict = {"gpsd": "gpsd-devel",
+"libpcap": "libpcap-devel",
+"lua": "lua-devel",
+"python": "python-devel",
+"jni": "java-1.8.0-openjdk java-1.8.0-openjdk-devel maven",
+"libpng": "libpng-devel",
+"libusb": "libusb-devel"}
+
+command = "sudo dnf --enablerepo=crb install -y epel-release {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_pacman(dependencies_str: str, no_confirm: bool):
+replace_dict = {"g++": "gcc",
+"jni": "jdk8-openjdk maven"}
+command = "sudo pacman --noconfirm -S {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install_with_winget(dependencies_str: str, no_confirm: bool):
+replace_dict = {"lua": "DEVCOM.Lua",
+"python": "python",
+"patch": "",
+"bison": "",
+"flex": ""}
+command = "winget install --disable-interactivity
--accept-package-agreements {}".format(
+_replace_wholewords(dependencies_str, replace_dict))
+assert _run_command_with_confirm(command, no_confirm)
+
+
+def _install(dependencies_str: str, no_confirm: bool):
+platform_system = platform.system()
+if platform_system == "Darwin":
+_install_with_brew(dependencies_str, no_confirm)
+elif platform_system == "Linux":
+distro_id = distro.id()
+if distro_id == "ubuntu":
+_install_with_apt(dependencies_str, no_confirm)
+elif "arch" in distro_id or "manjaro" in distro_id:
+_install_with_pacman(dependencies_str, no_confirm)
+elif "rocky" in distro_id:
+_install_with_dnf(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {distro_id} exiting")
+elif platform_system == "Windows":
+_install_with_winget(dependencies_str, no_confirm)
+else:
+sys.exit(f"Unsupported platform {platform_system} exiting")
+
+
+def _create_system_dependencies(minifi_options: MinifiOptions) -> str:
+system_dependencies = {'patch': 'patch', 'make': 'make'}
+if minifi_options.is_enabled("ENABLE_EXPRESSION_LANGUAGE"):
+system_dependencies['bison'] = 'bison'
+system_dependencies['flex'] = 'flex'
+if minifi_options.is_enabled("ENABLE_LIBARCHIVE"):
+system_dependencies['libarchive'] = 'libarchive'
+if minifi_options.is_enabled("ENABLE_PCAP"):
+
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1385283642
##
README.md:
##
@@ -304,8 +304,30 @@ sudo brew install libpcap
### Bootstrapping
-- MiNiFi C++ offers a bootstrap script in the root of our github repo that
will bootstrap the cmake and build process for you without the need to install
dependencies yourself. To use this
- process, please run the command `bootstrap.sh` from the root of the MiNiFi
C++ source tree.
+MiNiFi C++ offers bootstrap scripts that will bootstrap the cmake and build
process for you without the need to install dependencies yourself.
+
+ Python based bootstrapping
+
+Set up a virtual environment in the bootstrap folder.
+```
+python -m venv venv
Review Comment:
```suggestion
cd bootstrap
python -m venv venv
```
##
bootstrap/cli.py:
##
@@ -0,0 +1,141 @@
+import os
+
+import inquirer
+
+from minifi_option import MinifiOptions
+from system_dependency import install_required
+
+
+def install_dependencies(minifi_options: MinifiOptions):
+install_required(minifi_options)
+
+
+def run_cmake(minifi_options: MinifiOptions):
+if not os.path.exists(minifi_options.build_dir):
+os.mkdir(minifi_options.build_dir)
+os.chdir(minifi_options.build_dir)
+cmake_cmd = f"cmake -G Ninja {minifi_options.create_cmake_options_str()}
{minifi_options.source_dir}"
+print(f"Running {cmake_cmd}")
+os.system(cmake_cmd)
+
+
+def do_build(minifi_options: MinifiOptions):
+os.chdir(minifi_options.build_dir)
+os.system("cmake --build .")
+
+
+def do_one_click_build(minifi_options: MinifiOptions) -> bool:
+install_dependencies(minifi_options)
+run_cmake(minifi_options)
+do_build(minifi_options)
+return True
+
+
+def modify_bool_options(minifi_options: MinifiOptions):
+options = [inquirer.Checkbox(
+"MiNiFi C++ options",
+message="Select MiNiFi C++ components",
+choices=[name for name, obj in minifi_options.bool_options.items()],
+default=[name for name, obj in minifi_options.bool_options.items() if
obj.value == "ON"]
+)]
+
+selection_result = inquirer.prompt(options)
+for minifi_option in minifi_options.bool_options.values():
+if minifi_option.name in selection_result:
+minifi_option.value = "ON"
+else:
+minifi_option.value = "OFF"
+
+
+def main_menu(minifi_options: MinifiOptions):
+done = False
+while not done:
+main_menu_options = {
+f"Build dir: {minifi_options.build_dir}": build_dir_menu,
+f"Build type: {minifi_options.build_type.value}": build_type_menu,
+"Build options": bool_menu,
+"One click build": do_one_click_build,
+"Step by step build": step_by_step_menu,
+"Exit": lambda options: True,
+}
+
+questions = [
+inquirer.List(
+"sub_menu",
+message="Main Menu",
+choices=[menu_option_name for menu_option_name in
main_menu_options],
+),
+]
+
+main_menu_prompt = inquirer.prompt(questions)
+done = main_menu_options[main_menu_prompt["sub_menu"]](minifi_options)
+
+
+def build_type_menu(minifi_options: MinifiOptions) -> bool:
+questions = [
+inquirer.List(
+"build_type",
+message="Build type",
+choices=minifi_options.build_type.possible_values,
+),
+]
+
+answers = inquirer.prompt(questions)
+minifi_options.build_type.value = answers["build_type"]
+return False
+
+
+def build_dir_menu(minifi_options: MinifiOptions) -> bool:
+questions = [
+inquirer.Path('build_dir',
+ message="Build directory",
+ default=minifi_options.build_dir
+ ),
+]
+minifi_options.build_dir = inquirer.prompt(questions)["build_dir"]
+return False
+
+
+def bool_menu(minifi_options: MinifiOptions) -> bool:
+possible_values = [option_name for option_name in
minifi_options.bool_options]
+selected_values = [option.name for option in
minifi_options.bool_options.values() if option.value == "ON"]
+questions = [
+inquirer.Checkbox(
+"options",
+message="MiNiFi C++ Options",
+choices=possible_values,
+default=selected_values
+),
+]
+
+answers = inquirer.prompt(questions)
+for bool_option in minifi_options.bool_options.values():
+if bool_option.name in answers["options"]:
+bool_option.value = "ON"
+else:
+bool_option.value = "OFF"
+
+return False
Review Comment:
This menu needs guidance on how to exit from it after selecting the options.
##
README.md:
##
@@ -304,8 +304,30
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1363864275
##
CMakeLists.txt:
##
@@ -269,13 +260,12 @@ add_library(ut INTERFACE)
target_include_directories(ut SYSTEM INTERFACE
"${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/ut")
# cURL
-if(NOT DISABLE_CURL)
+if(ENABLE_CURL)
Review Comment:
Good idea, i've created a ticket for this.
https://issues.apache.org/jira/browse/MINIFICPP-2251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
martinzink commented on code in PR #1681: URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1363851877 ## .gitignore: ## @@ -48,6 +48,7 @@ cmake-build-debug *flowfile_checkpoint build /*build* +!win_build_vs.bat Review Comment: Yes, the one above it. `/*build*` matches `win_build_vs.bat` since its in the root directory -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
szaszm commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1362202851
##
CMakeLists.txt:
##
@@ -269,13 +260,12 @@ add_library(ut INTERFACE)
target_include_directories(ut SYSTEM INTERFACE
"${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/ut")
# cURL
-if(NOT DISABLE_CURL)
+if(ENABLE_CURL)
Review Comment:
It doesn't strictly belong here, but I wouldn't be against hardcoding these
to true. The usability of minifi is very limited without curl and openssl.
##
.gitignore:
##
@@ -48,6 +48,7 @@ cmake-build-debug
*flowfile_checkpoint
build
/*build*
+!win_build_vs.bat
Review Comment:
Is there another rule that matches `win_build_vs.bat` that makes this
exclusion necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
