[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884910#comment-16884910
]
Felipe Alfaro Solana commented on MESOS-9800:
-
Did you release a new version with this revert? If so, which version?
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868084#comment-16868084
]
James Peach commented on MESOS-9800:
Sorry it took so long to get back to you [~falfaro]. We are carrying a revert
of 2198b961d24b788564d36490cf52f78d7ec07655
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860974#comment-16860974
]
Felipe Alfaro Solana commented on MESOS-9800:
-
I tried building Mesos 1.8.1 from sources using {{--with-libarchive=/usr}} but
it still cannot extract the archive: it fails with exactly the same error. May
you share the patch you used to refer to {{/usr/bin/tar}}?
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855625#comment-16855625
]
James Peach commented on MESOS-9800:
We hit the same problem internally a while ago, and carried a patch to refer to
using {{/usr/bin/tar}}. If you are building your own Mesos, try passing the
{{\-\-with-llibarchive}} flag to use the system library, which is likely to
have been built with {{iconv}} support.
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854609#comment-16854609
]
Felipe Alfaro Solana commented on MESOS-9800:
-
I built Mesos 1.9 from sources including the patch attached and it does not
work. I still get the same error:
{{Jun 03 13:45:22 t01m01.node.t01.dns.teralytics.net mesos-slave[4977]: E0603
13:45:22.032101 5446 fetcher.cpp:615] EXIT with status 1: Failed to fetch
'/tmp/certificates.tar.gz': Failed to extract archive
'/var/mesos/slaves/226df5dd-cf6e-4eb6-85dc-9377220581f0-S0/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.d51544d5-8605-11e9-b011-0242c1d5dfd7/runs/e83576bf-83f9-44d4-83f8-f54739fdf903/certificates.tar.gz'
to
'/var/mesos/slaves/226df5dd-cf6e-4eb6-85dc-9377220581f0-S0/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.d51544d5-8605-11e9-b011-0242c1d5dfd7/runs/e83576bf-83f9-44d4-83f8-f54739fdf903':
Failed to read archive header: Linkname can't be converted from UTF-8 to
current locale.}}
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854317#comment-16854317
]
Felipe Alfaro Solana commented on MESOS-9800:
-
The output of the {{locale}} command:
{{LANG=en_US.UTF-8}}
{{LANGUAGE=}}
{{LC_CTYPE="en_US.UTF-8"}}
{{LC_NUMERIC="en_US.UTF-8"}}
{{LC_TIME="en_US.UTF-8"}}
{{LC_COLLATE="en_US.UTF-8"}}
{{LC_MONETARY="en_US.UTF-8"}}
{{LC_MESSAGES="en_US.UTF-8"}}
{{LC_PAPER="en_US.UTF-8"}}
{{LC_NAME="en_US.UTF-8"}}
{{LC_ADDRESS="en_US.UTF-8"}}
{{LC_TELEPHONE="en_US.UTF-8"}}
{{LC_MEASUREMENT="en_US.UTF-8"}}
{{LC_IDENTIFICATION="en_US.UTF-8"}}
{{LC_ALL=}}
And the environment variables used by the {{mesos-slave}} process:
{{MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins}}
{{TERM=linux}}
{{MESOS_OPTION_PREFIX=MESOS_}}
{{MESOS_PORT=5051}}
{{LC_ALL=en_US.UTF-8}}
{{LOGS=}}
{{MESOS_CONTAINERIZERS=docker,mesos}}
{{MESOS_WORK_DIR=/var/mesos}}
{{PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin}}
{{RUNLEVEL=2}}
{{PWD=/}}
{{JOB=rc}}
{{RESULT=ok}}
{{ZK=zk://10.101.0.10:2181,10.101.0.11:2181,10.101.0.12:2181/mesos}}
{{PREVLEVEL=N}}
{{MESOS_RESOURCES=ports(*):[23-9091,9093-9996,9998-3];ports(marathon_role):[9092,9997];}}
{{MESOS_ISOLATION=cgroups/cpu,cgroups/mem}}
{{MESOS_HOSTNAME=t01m01.node.t01.dns.teralytics.net}}
{{SHLVL=0}}
{{MESOS_MASTER=zk://10.101.0.10:2181,10.101.0.11:2181,10.101.0.12:2181/mesos}}
{{UPSTART_INSTANCE=}}
{{UPSTART_EVENTS=stopped}}
{{CLUSTER=mesos_cluster}}
{{MESOS_HADOOP_HOME=/usr/local/tl/hadoop}}
{{UPSTART_JOB=mesos-slave}}
{{INSTANCE=}}
{{IP=10.101.0.10}}
{{MESOS_GC_DISK_HEADROOM=0.1}}
{{MESOS_GC_DELAY=4weeks}}
{{ULIMIT=-n 20}}
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850626#comment-16850626
]
Felipe Alfaro Solana commented on MESOS-9800:
-
[^certificates2.tar.gz]
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850625#comment-16850625
]
Felipe Alfaro Solana commented on MESOS-9800:
-
I'm attaching the tar file that we the Marathon application depends on.
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850170#comment-16850170
]
Joseph Wu commented on MESOS-9800:
--
I believe this error appears when the locale of the program (Mesos agent) is
set to some non-UTF locale, such as the default POSIX locale. The addition of
the following line _might_ be enough to fix it.
{code}
diff --git a/3rdparty/stout/include/stout/archiver.hpp
b/3rdparty/stout/include/stout/archiver.hpp
index 551e644a3..706ba5282 100644
--- a/3rdparty/stout/include/stout/archiver.hpp
+++ b/3rdparty/stout/include/stout/archiver.hpp
@@ -54,6 +54,11 @@ inline Try extract(
archive_read_support_format_all(reader.get());
archive_read_support_filter_all(reader.get());
+ // Prevent Libarchive from trying to convert filenames to the locale-default
+ // character encoding. This conversion sometimes fails, for example when
+ // reading UTF-8 characters in a standard "C" locale (POSIX default).
+ archive_read_set_options(reader.get(), "hdrcharset=BINARY");
+
std::unique_ptr> writer(
archive_write_disk_new(),
[](struct archive* p) {
{code}
Could you attach an example archive that fails to extract and the locale of the
machine running your agent (i.e. the output of the {{locale}} command)? That
should give me a better idea of what case is failing.
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16849924#comment-16849924
]
Andrei Budnik commented on MESOS-9800:
--
Thanks for filing a detailed ticket!
Hope [~kaysoky] might help you with this issue.
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
