Adding to andries suggestion, check if you have one of the below 2 cases 1. Hbase storage plugin in enabled but hbase is not actually installed 2. You have an incompatible version of hbase installed (eg : hbase-0.94)
In both of the above cases, the queries you mentioned would be very slow. They would return after the hbase timeout (configured when setting up hbase) has expired. - Rahul On Mon, May 11, 2015 at 5:14 PM, Andries Engelbrecht < [email protected]> wrote: > You set up the Storage plugin to use the loopback NFS to read the file, > which is not the best way to do it. > > Why not use the MapR-FS or HDFS plug ins , this is a setup that uses the > MapR-FS plugin which will be considerably better than the loopback NFS for > Drill > > > > { > > "type": "file", > > "enabled": true, > > "connection": “maprfs:///", > > "workspaces": { > > "schema1": { > > "location": "/user/theUser/test", > > "writable": true, > > "defaultInputFormat": null > > } > > }, > > > —Andries > > > On May 11, 2015, at 5:07 PM, Minnow Noir <[email protected]> wrote: > > > Hanifi: The web UI is extremely slow, and the style sheets aren't > loading, > > but here is what I see for "show databases" > > > > Major FragmentMinor Fragments ReportingFirst StartLast StartFirst EndLast > > Endtmintavgtmaxlast updatelast progressmemmax 00-xx-xx1 / > > 10.213s0.213s0.670s0.670s0.457s0.457s0.457s12:15:3012:15:303MB > > Major Fragment: 00-xx-xx > > < > http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#fragment-0 > > > > Minor FragmentHostStartEndTotal TimeMax RecordsMax BatchesLast > UpdateLast > > ProgressPeak MemoryState > 00-00-xxip-10-101-197-197.ec2.internal0.213s0.670s > > 0.457s6112:15:3012:15:303MBFINISHED > > Operator Profiles > > Overview > > < > http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-overview > > > > OperatorTypeSetup (min)Setup (avg)Setup (max)Process (min)Process > > (avg)Process > > (max)Wait (min)Wait (avg)Wait (max)Mem (avg)Mem (max) > 00-xx-00SCREEN0.000s > > 0.000s0.000s0.000s0.000s0.000s0.000s0.000s0.000s-- 00-xx-01PROJECT0.422s > > 0.422s0.422s0.001s0.001s0.001s0.000s0.000s0.000s-- 00-xx-02 > > > INFO_SCHEMA_SUB_SCAN0.000s0.000s0.000s0.020s0.020s0.020s0.000s0.000s0.000s > > 17MB17MB > > 00-xx-00 - SCREEN > > < > http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-0 > > > > Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-00 > > 0.000s0.000s0.000s16- > > 00-xx-01 - PROJECT > > < > http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-1 > > > > Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-01 > > 0.422s0.001s0.000s16- > > 00-xx-02 - INFO_SCHEMA_SUB_SCAN > > < > http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-2 > > > > Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem > > 00-00-020.000s0.020s0.000s0017MB > > > > Andries: That storage plugin is very simple. I've obfuscated some > > proprietary info. (Note that the web UI was slow/hanging before I added > > the storage plugin.) > > > > { > > "type": "file", > > "enabled": true, > > "connection": "file:///", > > "workspaces": { > > "schema1": { > > "location": "/mapr/theClusterName/user/theUser/test", > > "writable": true, > > "defaultInputFormat": null > > } > > }, > > "formats": { > > "csv": { > > "type": "text", > > "extensions": [ > > "csv" > > ], > > "delimiter": "," > > }, > > "tsv": { > > "type": "text", > > "extensions": [ > > "tsv" > > ], > > "delimiter": "\t" > > }, > > "parquet": { > > "type": "parquet" > > } > > } > > } > > > > Unfortunately, because the Web UI is having issues, it is presenting an > > unstyled page with no disable links for the storage plugin so I can't > > disable it. With the web UI not working, I don't know how to delete the > > plugin. > > > > Abdel: There are no errors in the logs. > > > > > > On Mon, May 11, 2015 at 7:22 PM, Hanifi Gunes <[email protected]> > wrote: > > > >> I would be interested in knowing where the time has been spent. Can you > >> inspect query profile from web ui for `show databases` and let us know > how > >> the profile looks like? > >> > >> On Mon, May 11, 2015 at 4:10 PM, Abdel Hakim Deneche < > >> [email protected]> > >> wrote: > >> > >>> any errors in the logs ? > >>> > >>> On Mon, May 11, 2015 at 4:03 PM, Minnow Noir <[email protected]> > >> wrote: > >>> > >>>> Yes. The bit and server are idle, with no jobs of any sort running on > >>>> them. > >>>> > >>>> I just did started sqlline and it happened again. I had to cancel the > >>> show > >>>> databases command after 30s. > >>>> > >>>> sqlline version 1.1.6 > >>>> 0: jdbc:drill:zk=theServer:5181> show databases; > >>>> +-------------+ > >>>> | SCHEMA_NAME | > >>>> +-------------+ > >>>> | INFORMATION_SCHEMA | > >>>> | cp.default | > >>>> | dfs.default | > >>>> | dfs.root | > >>>> | dfs.tmp | > >>>> | test.default | > >>>> | test.schema1 | > >>>> | sys | > >>>> +-------------+ > >>>> 8 rows selected (29.264 seconds) > >>>> > >>>> On Mon, May 11, 2015 at 6:57 PM, Hanifi Gunes <[email protected]> > >>> wrote: > >>>> > >>>>> Can you confirm if this happens even when the particular 0.9 bit is > >>> idle? > >>>>> > >>>>> -Hanifi > >>>>> > >>>>> On Mon, May 11, 2015 at 3:53 PM, Minnow Noir <[email protected]> > >>>> wrote: > >>>>> > >>>>>> Experiencing some slowness after upgrading one of our servers from > >>> 0.8 > >>>> to > >>>>>> 0.9. In particular, despite restarting the Drillbit, and even the > >>>>> server, > >>>>>> several times, the Drill web UI and sqlline are both so slow as to > >> be > >>>>>> unusable. > >>>>>> > >>>>>> For example, doing a show databases hangs indefinitely, until I > >>> CTRL-C > >>>>> the > >>>>>> command (e.g., after a minute). > >>>>>> > >>>>>> show databases; > >>>>>> +-------------+ > >>>>>> | SCHEMA_NAME | > >>>>>> +-------------+ > >>>>>> | INFORMATION_SCHEMA | > >>>>>> | cp.default | > >>>>>> | dfs.default | > >>>>>> | dfs.root | > >>>>>> | dfs.tmp | > >>>>>> | test.default | > >>>>>> | test.schema1 | > >>>>>> | sys | > >>>>>> +-------------+ > >>>>>> 8 rows selected (59.044 seconds) > >>>>>> > >>>>>> I tried changing schemas, but had to cancel after waiting several > >>>>> minutes. > >>>>>> > >>>>>> use test.schema1; > >>>>>> +------------+------------+ > >>>>>> | ok | summary | > >>>>>> +------------+------------+ > >>>>>> | true | Default schema changed to 'test.schema1' | > >>>>>> +------------+------------+ > >>>>>> 1 row selected (208.089 seconds) > >>>>>> > >>>>>> There are no errors or obvious issues in the logs. The server has > >>>> plenty > >>>>> of > >>>>>> disk, CPU, and RAM free. It's running a recent version of MAPR > >>> Hadoop > >>>> on > >>>>>> CentOS 6.5 on EC2. It feels like something is hanging > >>>>>> > >>>>>> Note: This is just after starting sqlline; no actual work, which > >>> might > >>>>> have > >>>>>> slowed things down, has happened yet. > >>>>>> > >>>>>> This is what's running from a Drill perspective: > >>>>>> > >>>>>> ps aux | grep drill > >>>>>> ec2-user 24662 0.0 0.0 106100 1328 pts/0 S 12:29 0:00 > >> bash > >>>>>> bin/drillbit.sh internal_start drillbit > >>>>>> ec2-user 24697 0.3 2.3 6486240 730568 pts/0 Sl 12:29 0:36 > >>>>>> /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/bin/java > >>>>>> -Dlog.path=/opt/mapr/drill/drill-0.9.0/logs/drillbit.log -Xms1G > >>> -Xmx4G > >>>>>> -XX:MaxDirectMemorySize=8G -XX:MaxPermSize=512M > >>>>>> -XX:ReservedCodeCacheSize=1G -ea > >>>>>> -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf > >>>>>> -Dzookeeper.sasl.client=false -XX:+CMSClassUnloadingEnabled > >>>>>> -XX:+UseConcMarkSweepGC -cp > >>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > /opt/mapr/drill/drill-0.9.0/conf:/opt/mapr/drill/drill-0.9.0/jars/*:/opt/mapr/drill/drill-0.9.0/jars/ext/*:/opt/mapr/drill/drill-0.9.0/jars/3rdparty/*:/opt/mapr/drill/drill-0.9.0/jars/classb/* > >>>>>> org.apache.drill.exec.server.Drillbit > >>>>>> > >>>>>> Drill has been started thusly: > >>>>>> > >>>>>> bin/sqlline -u jdbc:drill:zk=theServer:5181 > >>>>>> > >>>>>> Any idea how I could troubleshoot this? > >>>>>> > >>>>>> Anyone else experiencing slowness after upgrading? > >>>>>> > >>>>>> Thank you > >>>>>> > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> > >>> Abdelhakim Deneche > >>> > >>> Software Engineer > >>> > >>> <http://www.mapr.com/> > >>> > >>> > >>> Now Available - Free Hadoop On-Demand Training > >>> < > >>> > >> > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > >>>> > >>> > >> > >
