John, I've got some sample working code in this notebook for connecting to Drill via pyodbc: https://github.com/cjmatta/DrillPandasReddit
I'd be interested in helping integrate this as well. Let us know if you push out a repo. -- Chris Matta 215-701-3146 [email protected] On Fri, May 13, 2016 at 2:28 PM, Ted Dunning <[email protected]> wrote: > The limit 0 form is highly to be preferred since that will give you > information about columns for a view. > > This can be set in the dialect for SQL Alchemy. > > > > On Fri, May 13, 2016 at 11:03 AM, Neeraja Rentachintala < > [email protected]> wrote: > > > Great, thanks John. I wil look forward for an update on how drill queuing > > part goes : ) > > Btw with regards to metadata queries, Drill already supports metadata > (both > > the limit 0 form and also show tables/show schemas which are served from > > the Information_schema). > > > > On Friday, May 13, 2016, John Omernik <[email protected]> wrote: > > > > > So with that Docker file, I got caravel working easily with test data > (no > > > drill yet) that will be weekend fun (and the pyodbc is already > installed > > in > > > the container, so now it's time to play!) > > > > > > So I started my docker image with: > > > > > > sudo docker run -it --rm --net=host > > > -v=/mapr/brewpot/apps/prod/caravel/working:/app/working:rw > > > -v=/mapr/brewpot/apps/prod/caravel/cache:/app/cache:rw zeta/caravel > > > /bin/bash > > > > > > > > > Now, I passed through a couple of volumes that I am not sure I will > > need, I > > > want to play so that my "State" and initialization is saved in those > > > directories in the running container (this is just early testing) I > just > > > run bash, and then run the commands below and it works. I was lazy here > > and > > > just did net host, it would likely work with bridged mode, but I am in > an > > > airport and wanted to see if I could get it working... the fun part > will > > be > > > working with Drill over the weekend. Thanks again Neeraja for sharing > > this! > > > > > > > > > > > > > > > Then I ran these commands(per the docs) and could explore... pretty > easy > > > actually! > > > > > > # Create an admin userfabmanager create-admin --app caravel > > > # Initialize the databasecaravel db upgrade > > > # Create default roles and permissionscaravel init > > > # Load some data to play withcaravel load_examples > > > # Start the development web servercaravel runserver -d > > > > > > > > > > > > > > > On Fri, May 13, 2016 at 11:27 AM, John Omernik <[email protected]> > wrote: > > > > > > > So, without running this, but having it build successfully, this > seems > > > > like a good place to start, it has caravel, and pyodbc all installed > > > here. > > > > I will be playing more this weekend.... > > > > > > > > FROM ubuntu > > > > > > > > RUN apt-get update && apt-get install -y build-essential libssl-dev > > > > libffi-dev python-dev python-pip > > > > > > > > RUN apt-get install -y unixodbc-dev unixodbc-bin > > > > > > > > RUN pip install pyodbc > > > > > > > > RUN pip install caravel > > > > > > > > CMD ["python -v"] > > > > > > > > On Fri, May 13, 2016 at 10:44 AM, John Omernik <[email protected]> > > wrote: > > > > > > > >> A little more googling and I found the pyodbc, that looks promising. > > > >> > > > >> On Fri, May 13, 2016 at 10:41 AM, John Omernik <[email protected]> > > > wrote: > > > >> > > > >>> "SQL Alchemy already understands Drill" I was just looking for > that, > > is > > > >>> there already some docs/blogs on that? I was going to start there > as > > > well > > > >>> to determine how it worked and then look into the dialect writing > and > > > see > > > >>> how big that project was. I didn't find much on the Drill + > Alchemy, > > > but I > > > >>> am in an airport and I blame wifi gremlins. > > > >>> > > > >>> > > > >>> > > > >>> On Fri, May 13, 2016 at 10:25 AM, Ted Dunning < > [email protected] > > > > > > >>> wrote: > > > >>> > > > >>>> SQLAlchemy generates SQL queries and passes them on to Drill. > Since > > > >>>> SQLAlchemy already understands Drill, most of what will be needed > is > > > >>>> slight > > > >>>> tuning for SQL dialect and providing a mechanism for SQLAlchemy to > > get > > > >>>> meta-data from views. Tableau does the meta-data discovery using > > > limit > > > >>>> 0 > > > >>>> queries to get column names. We would hope that similar methods > > would > > > >>>> work. > > > >>>> > > > >>>> > > > >>>> On Fri, May 13, 2016 at 6:13 AM, Erik Antelman < > [email protected] > > > > > > >>>> wrote: > > > >>>> > > > >>>> > Isn't this a matter of Drill<->SQLAlchemy. Such a support could > > > likely > > > >>>> > enable other frameworks. > > > >>>> > > > > >>>> > Would one think that adaptation of SQLAlchemy to Drill is > specific > > > to > > > >>>> > Caravel? What subset of features from a RDBMS ORM is > meaningfull, > > > >>>> feasable > > > >>>> > and usefull to map to Drill. This sounds like a broad general > > > >>>> question. I > > > >>>> > am sure there are orms from other language camps that might want > > > Drill > > > >>>> > backends. > > > >>>> > On May 13, 2016 7:33 AM, "John Omernik" <[email protected]> > wrote: > > > >>>> > > > > >>>> > > I will be looking into this as well, thanks for sharing! > > > >>>> > > On May 13, 2016 2:01 AM, "Nirav Shah" < > [email protected] > > > > > > >>>> wrote: > > > >>>> > > > > > >>>> > > > I Hi Neeraja, > > > >>>> > > > > > > >>>> > > > I am interested in contributing if integration is not > > available. > > > >>>> > > > Kindly let me know > > > >>>> > > > > > > >>>> > > > Regards, > > > >>>> > > > Nirav > > > >>>> > > > > > > >>>> > > > On Thu, May 12, 2016 at 9:19 PM, Neeraja Rentachintala < > > > >>>> > > > [email protected]> wrote: > > > >>>> > > > > > > >>>> > > > > Hi Folks > > > >>>> > > > > > > > >>>> > > > > Caravel is nice visualization tool recently open sourced > by > > > >>>> airbnb. > > > >>>> > Did > > > >>>> > > > > anyone try to integrate Drill and/or interested in > > > contributing > > > >>>> to > > > >>>> > > making > > > >>>> > > > > this work with Drill. > > > >>>> > > > > > > > >>>> > > > > https://github.com/airbnb/caravel > > > >>>> > > > > > > > >>>> > > > > > > > >>>> > > > > -Thanks > > > >>>> > > > > Neeraja > > > >>>> > > > > > > > >>>> > > > > > > >>>> > > > > > >>>> > > > > >>>> > > > >>> > > > >>> > > > >> > > > > > > > > > >
