It is a Secure 6.0 MapR Cluster. It's running drill-1.12.0. I initially used the RPM directory, but have now moved to the drill.tar.gz archine inside the drill-yarn package from the same version of Drill. I am running this in Marathon on Mesos.
I am using the same config that worked just fine with Drill 1.10. I have made changed based on reviewing the scripts in the MapR install to insure I pass the right flags to Drill to start it in secure mode. Well this is interesting, I started it, and stepped away... and this is what I came back to: a running drill bit with (it appears it's not failed web server, it's slow to start web server... is there a good way to trouble shoot this aspect? 2018-02-09 12:33:18,575 [main] INFO o.apache.drill.exec.server.Drillbit - Startup completed (600534 ms). On Fri, Feb 9, 2018 at 12:44 PM, Sorabh Hamirwasia <shamirwa...@mapr.com> wrote: > Hi John, > > Can you please share all your config files (*.conf and *.env) ? If I > understand correctly below is your environment setting: > > 1) Secure MapR cluster. > > 2) Running drill-1.12.0 installed from MapR released RPM. > > 3) Re-using the configuration from drill-1.10.0. > > > Thanks, > Sorabh > > ________________________________ > From: John Omernik <j...@omernik.com> > Sent: Friday, February 9, 2018 6:40:02 AM > To: user > Subject: Re: MapR Drill 1.12 Mismatch between Native and Library Versions > > I've made progress. I've pulled out the drill-yarn rpm, and realized there > was a drill.tar.gz file in there. I had a hunch with the work that Paul > Rogers was doing on Drill on Yarn, that this was a executor package and > thus had the MapR stuff, but also the complete MapR stuff so it could be > downloaded to a node and ran. Unpacking this, I think I am correct in that > assumption. (I am running Drill on Mesos, so this is pretty good for me). > (Suggestion to MapR: Make an RPM that is drill-executor that is only the > prepackaged drill directory with all the things, and have that be a > separate rpm, and have drill-yarn be all the additional stuff needed to run > drill on yarn, that way, when I get around to contributing a drill on mesos > rpm, I can just use your executor package!) > > That said, it's starting, but now I am getting no errors, not in the > drillbit.log file, not in std out or std err. The RPC ports are listening, > but the web port is not. I am wondering if something changed in the configs > that I need to account for (I am using my configs from 1.10) > > I am running on a secure cluster in mapr, and using authentication and SSL > in Drill. The only "MAYBE" related message I saw was below. However, my > only concern is "Encryption disabled" it still doesn't explain why the web > server is not starting without an error. Any ideas on where to look to > trouble shoot? > > Thanks! > > > > > 2018-02-09 08:34:03,915 [main] INFO o.a.d.c.s.persistence.ScanResult - > loading 2 classes for > org.apache.drill.exec.rpc.user.security.UserAuthenticator took 2ms > > 2018-02-09 08:34:03,925 [main] INFO o.a.d.e.r.s.AuthenticatorProviderImpl > - Configured authentication mechanisms: [plain] > > 2018-02-09 08:34:04,352 [main] INFO o.a.d.e.s.s.PersistentStoreRegistry - > Using the configured PStoreProvider class: > 'org.apache.drill.exec.store.sys.store.provider. > ZookeeperPersistentStoreProvider'. > > 2018-02-09 08:34:05,788 [main] INFO o.a.d.e.r.user.UserConnectionConfig - > Configured all user connections to require authentication with encryption: > Encryption: disabled , MaxWrappedSize: 65536 , WrapSizeLimit: 0 using: > [plain] > > 2018-02-09 08:34:05,896 [main] INFO o.apache.drill.exec.server.Drillbit - > Construction completed (3461 ms). > > > On Fri, Feb 9, 2018 at 8:10 AM, John Omernik <j...@omernik.com> wrote: > > > So already, you have given me some things to work with. Knowing that > there > > may links to jars/3rdparty was very helpful. In opening that folder on > > drill-1.12.0 I found there were no links/files related to mapr. In my > > drill-1.10.0 version (both of them from MapR) there were three files, > > maprfs, maprdb, and mapr-hbase. > > > > So I added only those three files from /opt/mapr/lib to $DRILL_CLASSPATH > > in my drill_env.sh. This allowed drill to start without that same > error. > > > > Now, I am being a little different. Instead of "installing" drill via > > RPMs, I download the RPMs (and I did this for both 1.10 and 1.12 from > MapR) > > The difference I think is in 1.10 there was a "drill" package and now in > > 1.12, there is a drill-internal package. Perhaps the drill in 1.10 moved > > some things around better. For what ever reason that changed (I think it > > changed in 1.11, but I was having cluster issues during that). > > > > That said, I have a drill bit running, but it's not starting it's web > > service. I am going to go reverse engineer what the RPM actually does. In > > my case, for 1.10 and every version previous, I just took the mapr RPM, > > unpacked it, and grabbed the drill directory and it worked great. This is > > obviously no longer the case, and I will have to dig deeper. > > > > > > > > On Thu, Feb 8, 2018 at 4:01 PM, Abhishek Girish <agir...@apache.org> > > wrote: > > > >> Can you also share the contents of (1) MapR build version on the cluster > >> nodes (cat /opt/mapr/MapRBuildVersion) (2) Drill RPM version installed > >> (rpm > >> -qa |grep -i mapr-drill) > >> > >> And also verify if the maprfs and maprdb jars inside > >> $DRILL_HOME/jars/3rdparty are links to the corresponding jars in > >> /opt/mapr/lib? > >> > >> On Thu, Feb 8, 2018 at 1:50 PM, Kunal Khatua <kkha...@mapr.com> wrote: > >> > >> > It might be to do with the way you've installed Drill. > >> > > >> > If you've built and deployed Drill, odds are that the client will be > >> > different. With the RPM installation, however, the installer has > >> symlinks > >> > to make the mapr-client libraries required by Drill be pointing to the > >> > libraries available in /opt/mapr/lib/ > >> > > >> > I don't know the exact details of what all gets symlinked, but this > step > >> > should have ensured that you don't see mismatch between the versions. > >> > > >> > That said... Support would be better equipped to help you with this. > >> > > >> > -----Original Message----- > >> > From: John Omernik [mailto:j...@omernik.com] > >> > Sent: Thursday, February 08, 2018 1:38 PM > >> > To: user <user@drill.apache.org> > >> > Subject: MapR Drill 1.12 Mismatch between Native and Library Versions > >> > > >> > I am running MapR's 1.12 drill on a node that only has posix client > >> > installed (and thus has a MapR client from that). > >> > > >> > I've recently had to work with MapR Support to get a fix to posix, and > >> > that fixed one issue, but now when I try to start a drill bit, I get > >> this > >> > error. > >> > The fix was a small patch that only updated the posix library... I > >> guess I > >> > am confused why I am seeing one "non-patched" version (in the java > >> library) > >> > and one "patched" version in the native library and why I can't start > >> > drill. I was debating where to post this, MapR community or here, but > >> it's > >> > only Drill I having an issue... any thoughts? > >> > > >> > John > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > 2018-02-08 15:20:54,3305 ERROR JniCommon > >> > fs/client/fileclient/cc/jni_MapRClient.cc:687 Thread: 71 Mismatch > found > >> > for java and native libraries java build version > >> 6.0.0.20171109191718.GA, > >> > native build version 6.0.0.20171229015939.GA java patch vserion $Id: > >> > mapr-version: 6.0.0.20171109191718.GA e892229b271c98c75ccb, native > >> patch > >> > version $Id: mapr-version: > >> > 6.0.0.20171229015939.GA bd8dae73f45572194c89 > >> > 2018-02-08 15:20:54,3305 ERROR JniCommon > >> > fs/client/fileclient/cc/jni_MapRClient.cc:704 Thread: 71 Client > >> > initialization failed. > >> > Exception in thread "main" > >> > org.apache.drill.exec.exception.DrillbitStartupException: Failure > while > >> > initializing values in Drillbit. > >> > > >> > > > > >
