On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin <[email protected]> wrote:
> May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 > address. > The issue is the actual Kudu servers, not the KDC. The KDC is on 127.0.0.1, but we have service principals like kudu/127.2.3.4@TESTREALM that are causing the issue. -Todd > > > Best regards, > > Alexey > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon <[email protected]> wrote: > > > FWIW it looks like there's already some code out there that can do the > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html > > > > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon <[email protected]> wrote: > > > > > Hey folks > > > > > > I've been looking into why our kerberos-dependent tests are failing on > > el6 > > > and it looks like it will be unfortunately difficult to fix. > > > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically > create > > > the directory for a DIR:<path> type ticket cache. That one was easy to > > fix > > > and got minikdc-test passing. > > > > > > The more insidious issue, though, is the following: > > > In miniclusters we start daemons on weird local IP addresses > (127.x.y.z) > > > which don't have any corresponding domain names. So, we have configured > > the > > > MiniKDC to disable reverse DNS, and are creating service principals > kudu/ > > > [email protected]. This works great on krb5-1.12. > > > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ > > > Ticket/Display.html?id=7603) which is preventing this from working on > > > el6. When 'rdns = false', Kerberos is unable to figure out which realm > > the > > > server is in (and doesn't fall back to the default realm). So, our > > > connections are failing with: "Cannot determine realm for numeric host > > > address". > > > > > > If we change to 'rdns = true', then it will first try to reverse lookup > > > the weird loopback, and then when it fails, still give the same error > > about > > > numeric hosts. > > > > > > It's possible there's some way to explicitly set the target realm, but > if > > > so I can't seem to find it. So, the only workarounds I can think of > are: > > > > > > 1) Use a local build of krb5 1.12 or later on el6 for test purposes. I > > > verified that if I built krb5 1.14 and put its libraries in > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, > the > > > tests passed fine. > > > > > > The downside here is that we would lose test coverage of the library > > > version actually installed on a lot of end-user systems. So another > > option > > > might be to test against a patched version of 1.10 (with only the fix > for > > > this numeric hostname issue). > > > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries > for > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that > nature. > > > We'd have to do this via LD_PRELOAD probably, which is a bummer, but I > > > can't see any other way to override the resolver on a per-user basis > > > ($HOSTALIASES only does forward DNS). > > > > > > Any other ideas? > > > > > > -Todd > > > -- > > > Todd Lipcon > > > Software Engineer, Cloudera > > > > > > > > > > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
