Also would be good to see the env variables (LD_LIBRARY_PATH) being seen by the container when you start it, just to make sure the env are getting set correctly.
On Mon, Oct 17, 2016 at 8:42 AM, Avinash Sridharan <[email protected]> wrote: > Does look like the symlink exists: > /usr/lib/libtiff.so.5 > > I am assuming you have checked the realpath exists as well (that's why was > asking for `ls -al`) ? > > Don't see that you have volume mounts that might obfuscate the path. Could > you create a JIRA and if possible point give access to your docker image > for us to try? Do mention the exact version of Mesos and the Distro you are > trying to run this on. > > -Avinash > > On Mon, Oct 17, 2016 at 8:35 AM, Mark Hammons < > [email protected]> wrote: > >> No, it's a regular linux log. I've reattached it. >> >> On Monday, October 17, 2016 8:32:07 AM CEST Avinash Sridharan wrote: >> > can't seem to open the attached logs, is it gzip? >> > >> > On Mon, Oct 17, 2016 at 8:14 AM, Mark Hammons < >> [email protected] >> > > wrote: >> > > >> > > Adding LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu doesn't fix the >> error. >> > > >> > > On Monday, October 17, 2016 5:06:34 PM CEST Mark Hammons wrote: >> > > > As I've shown in the original logs, the symbolic link libtiff.so.5 >> is in >> > > > /usr/ lib. I used LD_PRELOAD just to try to force >> /usr/lib/libtiff.so.5 >> > > >> > > to >> > > >> > > > be loaded since it wasn't for some reason. >> > > > >> > > > On Monday, October 17, 2016 8:03:34 AM CEST Avinash Sridharan wrote: >> > > > > Can you prepend your command with `ls -al /usr/lib` and check in >> > > >> > > stdout if >> > > >> > > > > you are seeing the shared object? By the way why are you using >> > > >> > > LD_PRELOAD >> > > >> > > > > instead of LD_LIBRARY_PATH. LD_PRELOAD will force the linker to >> load >> > > >> > > your >> > > >> > > > > library before any other libraries. It's usually used to override >> > > >> > > system >> > > >> > > > > libraries so found a bit odd that you are using it here? >> > > > > >> > > > > On Mon, Oct 17, 2016 at 7:54 AM, Mark Hammons >> > > > > <[email protected]> >> > > > > >> > > > > > wrote: >> > > > > > >> > > > > > No, but even when I set LD_PRELOAD=/usr/lib/libtiff.so.5 for >> the >> > > >> > > process >> > > >> > > > > > environment it says it can't load /usr/lib/libtiff.so.5. >> > > > > > >> > > > > > On Monday, October 17, 2016 7:52:16 AM CEST Avinash Sridharan >> wrote: >> > > > > > > Are you setting the env in the dockerfile? >> > > > > > > >> > > > > > > On Mon, Oct 17, 2016 at 6:58 AM, Mark Hammons < >> > > > > > >> > > > > > [email protected] >> > > > > > >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > Here's the code I define my executor to mesos with: >> > > > > > > > val iuwtURI = CommandInfo.URI.newBuilder(). >> > > >> > > setValue("http:// >> > > >> > > > > > ***/ >> > > > > > >> > > > > > > > IUWT.tar.gz").setExtract(true).setCache(false).build() >> > > > > > > > >> > > > > > > > val iuwtjURI = >> > > > > > > > CommandInfo.URI.newBuilder().setValue("http:// >> > > > > > >> > > > > > ***/ >> > > > > > >> > > > > > > > iuwtExecutor-assembly-0.1- >> > > > > > > > SNAPSHOT.jar").setExecutable(false).setCache(false).build() >> > > > > > > > >> > > > > > > > val iuwtExec = "java -jar >> > > > > > > > iuwtExecutor-assembly-0.1-SNAPSHOT.jar >> > > > > > >> > > > > > - >> > > > > > >> > > > > > > > Xmx1024M -Xmx128M" >> > > > > > > > >> > > > > > > > val iuwtCommand = >> > > > > > > > >> > > > > > > > CommandInfo.newBuilder.setValue(iuwtExec).addAllUris( >> > > >> > > List(iuwtjURI, >> > > >> > > > > > > > iuwtURI).asJava).setShell(true).build() >> > > > > > > > >> > > > > > > > val iuwtImageInfo = >> > > > > > > > >> > > > > > > > Image.newBuilder().setType(Image.Type.DOCKER).setDocker( >> > > > > > > > Image.Docker.newBuilder.setName("ubuntu- >> > > > > > > > mesos:0.11-17102016-IUWT").build()).build() >> > > > > > > > >> > > > > > > > val iuwtContInfo = >> > > > > > > > >> > > > > > > > ContainerInfo.MesosInfo.newBuilder().setImage( >> > > >> > > iuwtImageInfo).build() >> > > >> > > > > > > > val iuwtContainer = ContainerInfo.newBuilder() >> > > > > > > > >> > > > > > > > .setMesos(iuwtContInfo) >> > > > > > > > >> > > > > > > > .setType(ContainerInfo.Type.MESOS) >> > > > > > > > .build() >> > > > > > > > >> > > > > > > > val iuwtExecutor = ExecutorInfo.newBuilder() >> > > > > > > > >> > > > > > > > .setCommand(iuwtCommand) >> > > > > > > > .setContainer(iuwtContainer) >> > > > > > > > .setExecutorId(ExecutorID. >> > > >> > > newBuilder().setValue("iuw >> > > >> > > > > > > > t- >> > > > > > > > >> > > > > > > > executor")) >> > > > > > > > >> > > > > > > > .setName("iuwt-executor").build() >> > > > > > > > >> > > > > > > > The ubuntu-mesos:0.11-17102016-IUWT is a locally hosted >> docker >> > > > > > > > image. >> > > > > > > > IUWT is >> > > > > > > > the program I'm trying to run, and it runs perfectly fine >> in the >> > > > > > > > aforementioned docker container when running on docker. The >> > > > > > >> > > > > > libtiff.so.5 >> > > > > > >> > > > > > > > problem only manifests when I'm using mesos' containerizer. >> > > > > > > > >> > > > > > > > On Monday, October 17, 2016 6:52:17 AM CEST Avinash >> Sridharan >> > > >> > > wrote: >> > > > > > > > > You are running a container with its own image right? So >> is >> > > > > > > > >> > > > > > > > /usr/lib/x86_64 >> > > > > > > > >> > > > > > > > > in the container's file system or the host file system? >> > > > > > > > > >> > > > > > > > > On Mon, Oct 17, 2016 at 6:47 AM, Mark Hammons < >> > > > > > > > >> > > > > > > > [email protected] >> > > > > > > > >> > > > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > Yes, it's installed under /usr/lib/x86_64 or whatever >> the >> > > > > > > > > > multilib >> > > > > > > > >> > > > > > > > path is >> > > > > > > > >> > > > > > > > > > in >> > > > > > > > > > debian. It seems files under this path are not >> accessible. >> > > > > > > > > > >> > > > > > > > > > I added LD_PRELOAD=/usr/lib/libtiff.so.5 to try to >> force the >> > > > > > >> > > > > > symlink >> > > > > > >> > > > > > > > to >> > > > > > > > >> > > > > > > > > > load >> > > > > > > > > > but it refused to load it. I think the mesos >> containerizer >> > > > > > > > > > is >> > > > > > > > >> > > > > > > > preventing >> > > > > > > > >> > > > > > > > > > the >> > > > > > > > > > program from accessing anything in a directory under >> > > >> > > /usr/lib/ >> > > >> > > > > > > > > > for >> > > > > > > > > > some >> > > > > > > > > > reason, as the same program runs fine in the same >> container >> > > > > > > > > > running >> > > > > > > > >> > > > > > > > under >> > > > > > > > >> > > > > > > > > > docker. >> > > > > > > > > > >> > > > > > > > > > On Monday, October 17, 2016 6:40:49 AM CEST Avinash >> > > > > > > > > > Sridharan >> > > > > > >> > > > > > wrote: >> > > > > > > > > > > Is the library part of the image that you are running? >> > > > > > > > > > > Also >> > > > > > > > > > > you >> > > > > > > > > > > might >> > > > > > > > > > >> > > > > > > > > > need >> > > > > > > > > > >> > > > > > > > > > > to setup LD_LIBRARY_PATH in your env while launching >> the >> > > >> > > image >> > > >> > > > > > > > > > > so >> > > > > > > > >> > > > > > > > that >> > > > > > > > >> > > > > > > > > > the >> > > > > > > > > > >> > > > > > > > > > > container process knows where to look for the shared >> > > >> > > object. >> > > >> > > > > > > > > > > On Mon, Oct 17, 2016 at 5:21 AM, Mark Hammons < >> > > > > > > > > > >> > > > > > > > > > [email protected] >> > > > > > > > > > >> > > > > > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > > Hi all, >> > > > > > > > > > > > >> > > > > > > > > > > > I've been working with the mesos containerizer to >> handle >> > > >> > > my >> > > >> > > > > > docker >> > > > > > >> > > > > > > > > > > > containers >> > > > > > > > > > > > recently. I created a docker container that requires >> > > > > > >> > > > > > libtiff.so.5, >> > > > > > >> > > > > > > > and >> > > > > > > > >> > > > > > > > > > the >> > > > > > > > > > >> > > > > > > > > > > > application within it runs well. But when I try to >> run >> > > > > > > > > > > > it >> > > > > > >> > > > > > within >> > > > > > >> > > > > > > > the >> > > > > > > > >> > > > > > > > > > mesos >> > > > > > > > > > >> > > > > > > > > > > > containerizer I get an error saying libtiff.so.5 >> doesn't >> > > > > > > > > > > > exist. >> > > > > > > > > > > > >> > > > > > > > > > > > The application is being launched via java's process >> > > > > > > > > > > > mechanism >> > > > > > > > > > > > from >> > > > > > > > > > >> > > > > > > > > > inside >> > > > > > > > > > >> > > > > > > > > > > > a >> > > > > > > > > > > > java thread in a custom java executor if that makes >> a >> > > > > > >> > > > > > difference. >> > > > > > >> > > > > > > > > > > > Any idea what could be causing this change in >> behavior? >> > > >> > > As >> > > >> > > > > > > > > > > > you >> > > > > > >> > > > > > can >> > > > > > >> > > > > > > > see >> > > > > > > > >> > > > > > > > > > in >> > > > > > > > > > >> > > > > > > > > > > > the >> > > > > > > > > > > > attached log file, I check /usr/lib, and a symbolic >> link >> > > >> > > to >> > > >> > > > > > > > > > > > /usr/lib/x86..../ >> > > > > > > > > > > > libtiff.so.5 exists in /usr/lib so the program >> should be >> > > > > > > > > > > > able >> > > > > > >> > > > > > to >> > > > > > >> > > > > > > > find >> > > > > > > > >> > > > > > > > > > and >> > > > > > > > > > >> > > > > > > > > > > > load >> > > > > > > > > > > > that.... >> > > > > > > > > > > > ---- >> > > > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56 >> > > > > > > > > > >> > > > > > > > > > -- >> > > > > > > > > > ---- >> > > > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56 >> > > > > > > > >> > > > > > > > -- >> > > > > > > > ---- >> > > > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56 >> > > > > > >> > > > > > -- >> > > > > > ---- >> > > > > > Mark Edgar Hammons II | +33 06 03 69 56 56 >> > > >> > > -- >> > > ---- >> > > Mark Edgar Hammons II | +33 06 03 69 56 56 >> >> >> -- >> ---- >> Mark Edgar Hammons II | +33 06 03 69 56 56 >> > > > > -- > Avinash Sridharan, Mesosphere > +1 (323) 702 5245 > -- Avinash Sridharan, Mesosphere +1 (323) 702 5245
