Thank you. After a bit more digging I see that Tachyon is not HDFS compatible (explains the "tachyon://" connection prefix) and I doubt that the it will work at all until Drill starts officially supporting it.
I will do this and report any positive progress. It's a shame that it does not work as Tachyon+S3 could be a really great Drill setup where frequently used files are available "locally" and S3 is used more like a deep storage than anything else. Regards, -Stefán On Sun, Jul 26, 2015 at 3:11 PM, David Tucker <[email protected]> wrote: > One quick hint : > The file $DRILL_HOME/bin/hadoop-excludes.txt has a list of jars that ARE > NOT loaded during the bootstrap of Drill … and jets3t is one of them. > Commenting out the jets3t line in that file and restarting the drill bits > will at least get you past the first java dependency problem. > > — David > > > On Jul 26, 2015, at 4:52 AM, Stefán Baxter <[email protected]> > wrote: > > > > Hi, > > > > I'm trying to run Drill with Tachyon on top of S3. > > > > My Drill-Source config looks like this: > > { > > "type": "file", > > "enabled": true, > > "connection": "tachyon://localhost:19998/", > > "workspaces": { > > "root": { > > "location": "/", > > "writable": true, > > "defaultInputFormat": null > > } > > }, > > "formats": { > > "psv": { > > "type": "text", > > "extensions": [ > > "tbl" > > ], > > "delimiter": "|" > > }, > > "csv": { > > "type": "text", > > "extensions": [ > > "csv" > > ], > > "delimiter": "," > > }, > > "tsv": { > > "type": "text", > > "extensions": [ > > "tsv" > > ], > > "delimiter": "\t" > > }, > > "parquet": { > > "type": "parquet" > > }, > > "json": { > > "type": "json" > > }, > > "avro": { > > "type": "avro" > > } > > } > > } > > > > Initilly I get: > > Error: SYSTEM ERROR: IOException: No FileSystem for scheme: tachyon > > > > When I copy the tachyon client (tachyon-client-0.7.0.jar) to the > > jar/3rdparty the system complains about missing tependencies (jets3). > > > > If I copy the full Tachyon client > > (tachyon-client-0.7.0-jar-with-dependencies.jar) to the jar/3rds party > the > > system will not start. > > > > Exception in thread "main" java.lang.ExceptionInInitializerError > > at org.apache.log4j.LogManager.getLogger(LogManager.java:44) > > at > org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64) > > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:277) > > at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:288) > > at org.apache.drill.exec.server.Drillbit.<clinit>(Drillbit.java:58) > > Caused by: java.lang.IllegalStateException: Detected both > > log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting > > StackOverflowError. See also > > http://www.slf4j.org/codes.html#log4jDelegationLoop for more details. > > at > org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:51) > > ... 5 more > > > > > > Has anyone here done this already and is willing to share some info? > > > > Regards, > > -Stefán > >
