So, apparently I spoke too soon, as my image-from-an-image in OpenShift
actually *didn't* start up successfully.
I'll take Ted's and Charles' comments in the current "[DISCUSS]" thread
regarding attracting users who may never contribute a line of code but
nevertheless report on their use of Drill and problems they may encounter as
license to ask a follow up question... Besides, with the growing usage of
OpenShift, I'm sure I won't be the last one trying to do this!
You guys were talking about attracting enthusiastic users who want to spread
the word to their friends. Well, as a bit of a backstory, it was an
enthusiastic analyst here who floated the idea of replacing our MongoDB + JSON
with Drill + Parquet. Eventually our tech lead signed on to the idea and I got
tasked to help out. If this all works out, I'm sure word will spread to other
analysts. So here I am... (I don't mind being the "canary in the coalmine",
so to speak. I've often had to take on that role. And I get to learn new
things...)
Ok, enough blah-blah-blah. The problem I'm having, as far as I can tell, stems
from the fact that OpenShift doesn't set a user name when running a container.
In particular, the Java System.getProperty("user.name") and "user.home" calls
return "?" as reported in the Zookeeper (I'm in embedded mode but the Zookeeper
config is still being filled in) section of the sqlline.log:
[main] INFO o.apache.drill.exec.server.Drillbit - Drillbit environment:
user.name=?
[main] INFO o.apache.drill.exec.server.Drillbit - Drillbit environment:
user.home=?
As a quick point of comparison, when I run a Drill Docker on my desktop (not in
OpenShift), with zero config changes, everything of course works fine, and
"user.name" is "root" and "user.home" is "/root". (Probably because I
installed Docker with a "sudo yum install".) Similarly, when I run an
embedded-Drill on my desktop installed from the source *tar.gz - i.e. not in a
Docker, and not in OpenShift - "user.name" is my $USER and "user.home" is my
$HOME, and everything again runs fine.
I'll include the big stack trace at the bottom, but really the only question I
have at the moment is:
Given that I don't think I can get OpenShift to set or pass in a "user.name"
property for the JDK to get, is there an environment variable (OpenShift *can*
pass those in) or a magical setting in drill-override.conf that will let me get
past this "invalid null input: name" problem I'm running into during the
"login" phase of the startup?
And if so, could the "user.name" be set to anything or would it have to be
root? (I don't quite understand the relationship between the "user.name" and
the Hadoop login, etc.)
Thank you so much for any help!
Ron
--------------------------------------------------------------------------------
Error: Failure in starting embedded Drillbit:
org.apache.drill.exec.exception.DrillbitStartupException: Failed to login.
(state=,code=0)
java.sql.SQLException: Failure in starting embedded Drillbit:
org.apache.drill.exec.exception.DrillbitStartupException: Failed to login.
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:143)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
at
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
at sqlline.Commands.connect(Commands.java:1364)
at sqlline.Commands.connect(Commands.java:1244)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:730)
at sqlline.SqlLine.initArgs(SqlLine.java:410)
at sqlline.SqlLine.begin(SqlLine.java:515)
at sqlline.SqlLine.start(SqlLine.java:267)
at sqlline.SqlLine.main(SqlLine.java:206)
Caused by: org.apache.drill.exec.exception.DrillbitStartupException: Failed to
login.
at
org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:161)
at
org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:82)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:171)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:135)
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:133)
... 18 more
Caused by: org.apache.hadoop.security.KerberosAuthException: failure to login:
javax.security.auth.login.LoginException: java.lang.NullPointerException:
invalid null input: name
at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
at
com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at
org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
at
org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
at
org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:156)
at
org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:82)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:171)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:135)
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:133)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
at
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
at sqlline.Commands.connect(Commands.java:1364)
at sqlline.Commands.connect(Commands.java:1244)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:730)
at sqlline.SqlLine.initArgs(SqlLine.java:410)
at sqlline.SqlLine.begin(SqlLine.java:515)
at sqlline.SqlLine.start(SqlLine.java:267)
at sqlline.SqlLine.main(SqlLine.java:206)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
at
org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
at
org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:156)
... 22 more
Caused by: javax.security.auth.login.LoginException:
java.lang.NullPointerException: invalid null input: name
at com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
at
com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at
org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
at
org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
at
org.apache.drill.exec.server.BootStrapContext.login(BootStrapContext.java:156)
at
org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:82)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:171)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:135)
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:133)
at
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:67)
at
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:67)
at
org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
at org.apache.drill.jdbc.Driver.connect(Driver.java:75)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
at sqlline.Commands.connect(Commands.java:1364)
at sqlline.Commands.connect(Commands.java:1244)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:730)
at sqlline.SqlLine.initArgs(SqlLine.java:410)
at sqlline.SqlLine.begin(SqlLine.java:515)
at sqlline.SqlLine.start(SqlLine.java:267)
at sqlline.SqlLine.main(SqlLine.java:206)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:856)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at
org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
... 25 more
Apache Drill 1.17.0
"You told me to, Drill Sergeant!"
> On January 29, 2020 at 5:47 PM Ron Cecchini <[email protected]> wrote:
>
> Sorry for the spam, but I think I figured it out.
>
> Thank you so much for your suggestions to build an image from an image. I
> finally put 2 & 2 together and realized what you were saying and created the
> following Dockerfile. I then built and pushed the image into OpenShift - and
> it started up nicely.
>
> I haven't had a chance to test it yet, but I'm optimistic.
>
> Thank you again.
>
> ---
>
> Dockerfile:
>
> # Use the latest official release of Apache Drill
> FROM apache/drill:1.17.0
>
> # Make /opt/drill readable by everyone due to OpenShift's security
> RUN chgrp -R 0 /opt/drill && chmod -R g=u /opt/drill
>
> # Start Drill in embedded mode and connects to Sqlline
> ENTRYPOINT /opt/drill/bin/drill-embedded
