Yes that's the first idea ;) Regards JB
On Dec 7, 2016, 17:27, at 17:27, Vinoth Chandar <vin...@uber.com> wrote: >Interesting. So all the planning & execution is done by Hive, and Beam >will >process the results of the query? > >On Wed, Dec 7, 2016 at 8:24 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >wrote: > >> Hi >> >> The HiveIO will directly use the native API and HiveQL. That's the >plan on >> which we are working right now. >> >> Regards >> JB >> >> On Dec 7, 2016, 17:18, at 17:18, Vinoth Chandar <vin...@uber.com> >wrote: >> >Hi, >> > >> >I am not looking for a way to actually execute the query on Hive. I >> >would >> >like to do something similar to Spark SQL/HiveContext, but with >Beam. >> >Just >> >have a HiveIO that reads metadata from Hive metastore, and then >later >> >use a >> >Spark runner to execute the query. So, HiveJDBC is not an option I >> >would >> >like to pursue. Thanks for the pointer, though! >> > >> >And does the HiveIO that is being planned, work similarly as above? >> > >> > >> >Thanks >> >Vinoth >> > >> > >> > >> >On Tue, Dec 6, 2016 at 4:55 AM, Ismaël Mejía <ieme...@gmail.com> >wrote: >> > >> >> Hello, >> >> >> >> If you really need to read/write via Hive, remember that you can >use >> >the >> >> Hive Jdbc driver, and achieve this with Beam using the JdbcIO >(this >> >is >> >> probably less efficient for the streaming case but still a valid >> >solution). >> >> >> >> Ismaël >> >> >> >> >> >> On Tue, Dec 6, 2016 at 12:04 PM, Vinoth Chandar <vin...@uber.com> >> >wrote: >> >> >> >> > Great. Thanks! >> >> > >> >> > Thanks, >> >> > Vinoth >> >> > >> >> > > On Dec 6, 2016, at 2:06 AM, Jean-Baptiste Onofré >> ><j...@nanthrax.net> >> >> > wrote: >> >> > > >> >> > > Hi, >> >> > > >> >> > > Ismaël and I started HiveIO. >> >> > > >> >> > > I have several IOs ready to propose as PR, but, in order to >limit >> >the >> >> > number of open PRs, I would like to merge the pending ones. >> >> > > >> >> > > I will let you know when the branches/PRs will be available. >> >> > > >> >> > > Regards >> >> > > JB >> >> > > >> >> > >> On 12/05/2016 11:40 PM, Vinoth Chandar wrote: >> >> > >> Hi guys, >> >> > >> >> >> > >> Saw a post around HiveIO on the users list with a PR >followup. I >> >am >> >> > >> interested in this too and can pitch in on developement and >> >testing.. >> >> > >> >> >> > >> Who & where is this work happening? >> >> > >> >> >> > >> Thanks >> >> > >> VInoth >> >> > >> >> >> > > >> >> > > -- >> >> > > Jean-Baptiste Onofré >> >> > > jbono...@apache.org >> >> > > http://blog.nanthrax.net >> >> > > Talend - http://www.talend.com >> >> > >> >> >>
