Maybe try sort.external.spill.directories: [ "/var/mapr/local/$hostname/drillspill" ],
—Andries > On Sep 24, 2015, at 12:38 PM, Andy Pernsteiner <[email protected]> > wrote: > > I’m trying to do some experimentation and set the > drill.exec.sort.external.spill.directories value. Since this option appears > as a ‘boot’ option ( https://drill.apache.org/docs/start-up-options/ ) , I > believe the right way is to set this in drill-override.conf on each node. > > I tried doing this via the following: > > > drill.exec: { > cluster-id: "se1-drillbits", > zk.connect: "10.10.15.10:5181,10.10.15.11:5181,10.10.15.12:5181", > sys.store.provider.zk.blobroot: "maprfs:///user/mapr/profiles", > sort.external.spill.directories: [ "/var/mapr/$hostname/drillspill" ], > sort.external.spill.fs: "maprfs:///", > impersonation: { > enabled: true, > max_chained_user_hops: 3 > } > } > > I also tried setting via: > > sort: { > purge.threshold : 100, > external: { > batch.size : 4000, > spill: { > batch.size : 4000, > group.size : 100, > threshold : 200, > directories : [ "/var/mapr/$hostname/drillspill" ], > fs : “maprfs:///" > } > } > }, > > > But then looking at the sys.boot table after restarting the drill bits, I > still see the default values: > > 0: jdbc:drill:> select * from sys.boot where name like '%spill%'; > +------+------+------+--------+---------+------------+----------+-----------+ > | name | kind | type | status | num_val | string_val | bool_val | float_val | > +------+------+------+--------+---------+------------+----------+-----------+ > | drill.exec.sort.external.spill.batch.size | LONG | BOOT | BOOT | 4000 | > null | null | null | > | drill.exec.sort.external.spill.directories | STRING | BOOT | BOOT | null | [ > # > jar:file:/opt/mapr/drill/drill-1.1.0/jars/drill-java-exec-1.1.0.jar!/drill-module.conf: > 145 > "/tmp/drill/spill" > ] | null | null | > | drill.exec.sort.external.spill.fs | STRING | BOOT | BOOT | null | > "file:///" | null | null | > | drill.exec.sort.external.spill.group.size | LONG | BOOT | BOOT | 40000 | > null | null | null | > | drill.exec.sort.external.spill.threshold | LONG | BOOT | BOOT | 40000 | > null | null | null | > +------+------+------+--------+---------+------------+----------+-----------+ > > Note that I’ve tried removing the shell ‘$hostname’ variable (in case it > causes issues), no dice. > > What’s the right way to set these values? > > > > > > > Andy Pernsteiner > Manager, Field Enablement > ph: 206.228.0737 > > www.mapr.com > Now Available - Free Hadoop On-Demand Training > >
