[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364800#comment-17364800
]
acecile5555555 edited comment on MESOS-10219 at 6/17/21, 9:20 AM:
------------------------------------------------------------------
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include <stout/fs.hpp>
#include <stout/os.hpp>
#include <stout/path.hpp>
+#include <stout/internal/windows/longpath.hpp>
#include <stout/os/realpath.hpp>
@@ -148,6 +149,7 @@ Future<Nothing> PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future<Nothing> PosixFilesystemIsolatorProcess::update(
continue;
}
- string link = path::join(info->directory, containerPath);
+ string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -182,8 +184,9 @@ Future<Nothing> PosixFilesystemIsolatorProcess::update(
}
// Get user and group info for this task based on the task's sandbox.
- struct stat s;
- if (::stat(info->directory.c_str(), &s) < 0) {
+ struct _stat64i32 s;
+ const std::wstring longDirectoryPath =
::internal::windows::longpath(directoryPath);
+ if (::_wstat(longDirectoryPath.c_str(), &s) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
Came up with this workaround, still have not much idea of what I'm doing but I
ended up with the following code by matching types according to compiler errors
:D
Still working fine with short path, does not crash anymore with long path, but
sadly, the app is still not starting:
{noformat}
Failed to chdir into current working directory
'C:\mesos\work\slaves\4e393619-8979-47af-8a29-0c6729b5b41a-S1873\frameworks\346d7333-a980-43a8-93ab-343ea12d77d7-0000\executors\tests_windows-azure-vm-with-a-very-very-very-very-very-very-very-very-very-very-very-very-very-long-apth.instance-135dbf9e-c872-11eb-970f-02422c91a06b._app.62\runs\7dfcc8c1-876e-46b5-b79a-9bd24b5c38ce':
The filename or extension is too long.
{noformat}
I can see the following in app stderr (so I moved one step ahead, mesos is
actually trying to start the app now).
The code crashing is here:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/mesos/launch.cpp#L1062
So if I understand correctly it should end up here and crashs (if someone can
confirm I read the code correctly and chdir is actually that call):
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/os/windows/chdir.hpp#L31
But it seems I'm not the only one complaining about SetCurrentDirectoryW
function:
https://social.msdn.microsoft.com/Forums/vstudio/en-US/fb485167-2093-4465-a916-334882690e19/setcurrentdirectoryw-does-not-work-on-windows-7-for-long-paths?forum=vcgeneral
>From my understanding I should monify a manifest to enable longpathaware:
https://stackoverflow.com/a/44515362
Still trying to figure what this madness is about...
was (Author: acecile5555555):
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include <stout/fs.hpp>
#include <stout/os.hpp>
#include <stout/path.hpp>
+#include <stout/internal/windows/longpath.hpp>
#include <stout/os/realpath.hpp>
@@ -148,6 +149,7 @@ Future<Nothing> PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future<Nothing> PosixFilesystemIsolatorProcess::update(
continue;
}
- string link = path::join(info->directory, containerPath);
+ string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -182,8 +184,9 @@ Future<Nothing> PosixFilesystemIsolatorProcess::update(
}
// Get user and group info for this task based on the task's sandbox.
- struct stat s;
- if (::stat(info->directory.c_str(), &s) < 0) {
+ struct _stat64i32 s;
+ const std::wstring longDirectoryPath =
::internal::windows::longpath(directoryPath);
+ if (::_wstat(longDirectoryPath.c_str(), &s) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
Came up with this workaround, still have not much idea of what I'm doing but I
ended up with the following code by matching types according to compiler errors
:D
Still working fine with short path, does not crash anymore with long path, but
sadly, the app is still not starting:
{noformat}
Failed to chdir into current working directory
'C:\mesos\work\slaves\4e393619-8979-47af-8a29-0c6729b5b41a-S1873\frameworks\346d7333-a980-43a8-93ab-343ea12d77d7-0000\executors\tests_windows-azure-vm-with-a-very-very-very-very-very-very-very-very-very-very-very-very-very-long-apth.instance-135dbf9e-c872-11eb-970f-02422c91a06b._app.62\runs\7dfcc8c1-876e-46b5-b79a-9bd24b5c38ce':
The filename or extension is too long.
{noformat}
I can see the following in app stderr (so I moved one step ahead, mesos is
actually trying to start the app now).
The code crashing is here:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/mesos/launch.cpp#L1062
So if I understand correctly it should end up here and crashs (if someone can
confirm I read the code correctly and chdir is actually that call):
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/os/windows/chdir.hpp#L31
But it seems I'm not the only one complaining about SetCurrentDirectoryW
function:
https://social.msdn.microsoft.com/Forums/vstudio/en-US/fb485167-2093-4465-a916-334882690e19/setcurrentdirectoryw-does-not-work-on-windows-7-for-long-paths?forum=vcgeneral
> 1.11.0 does not build on Windows
> --------------------------------
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
> Affects Versions: 1.11.0
> Reporter: acecile5555555
> Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
> # This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
> # Enable level 4 C4061: The enumerate has no associated handler in a switch
> # statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> # Enable level 4 C4254: A larger bit field was assigned to a smaller bit
> # field.
> SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option<T> os::spawn(const std::string &,const
> std::vector<std::string,std::allocator<std::string>> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option<T> os::spawn(const std::string &,const
> std::vector<std::string,std::allocator<std::string>> &,const
> Option<std::map<std::string,std::string,std::less<std::string>,std::allocator<std::pair<const
> std::string,std::string>>>> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
> message : while trying to match the argument list '(const char [3],
> initializer list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option<T> os::spawn(const std::string &,const
> std::vector<std::string,std::allocator<std::string>> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option<T> os::spawn(const std::string &,const
> std::vector<std::string,std::allocator<std::string>> &,const
> Option<std::map<std::string,std::string,std::less<std::string>,std::allocator<std::pair<const
> std::string,std::string>>>> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> message : while trying to match the argument list '(const std::string,
> std::vector<std::string,std::allocator<std::string>>)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher_tracker.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launcher.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> mount.cpp
> switchboard.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launcher_tracker.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> environment_secret.cpp
> posix.cpp
> backend.cpp
> provisioner.cpp
> store.cpp
> authenticatee.cpp
> authenticator.cpp
> auxprop.cpp
> basic_authenticatee.cpp
> basic_authenticator_factory.cpp
> combined_authenticator.cpp
> acls.cpp
> build.cpp
> command_utils.cpp
> protobuf_utils.cpp
> resource_quantities.cpp
> resources_utils.cpp
> roles.cpp
> type_utils.cpp
> exec.cpp
> v0_v1executor.cpp
> files.cpp
> hdfs.cpp
> checker.cpp
> checker_process.cpp
> health_checker.cpp
> devolve.cpp
> evolve.cpp
> logging.cpp
> framework.cpp
> maintenance.cpp
> master.cpp
> quota.cpp
> quota_handler.cpp
> readonly_handler.cpp
> registry_operations.cpp
> weights.cpp
> weights_handler.cpp
> allocator.cpp
> hierarchical.cpp
> offer_constraints_filter.cpp
> C:\Users\earthlab\mesos\src\master\allocator\mesos\offer_constraints_filter.cpp(26,10):
> fatal error C1083: Cannot open include file: 're2/re2.h': No such file or
> directory [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> messages.cpp
> daemon.cpp
> driver.cpp
> disk_profile_adaptor.cpp
> sched.cpp
> scheduler.cpp
> resolver.cpp
> in_memory.cpp
> operation.cpp
> curl.cpp
> hadoop.cpp
> usage.cpp
> mesos.cpp
> version.cpp
> whitelist_watcher.cpp
> authentication.cpp
> group.cpp
> cpu.cpp
> mem.cpp
> windows.cpp{noformat}
> If you're interested in getting it built, I'll keep my Windows VM for a
> moment.
> I'm restarting with older version to see if it goes better.
>
> Best regards, Adam.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
