Hi Dan, Sorry for the late response.
I agreed with you for the use cases that you mentioned. Advice me and please share if there is any sample code to join two data sets in Beam that are sharing some common keys. Regards, Prabeesh K. On 6 February 2017 at 10:38, Dan Halperin <[email protected]> wrote: > Definitely, using BigQuery for what BigQuery is really good at (big scans > and cost-based joins) is nearly always a good idea. A strong endorsement of > Ankur's answer. > > Pushing the right amount of work into a database is an art, however -- > there are some scenarios where you'd rather scan in BQ and join in Beam > because the join result is very large and you can better filter it in Beam, > or because you need to do some pre-join-filtering based on an external API > call (and you don't want to load the results of that API call into > BigQuery)... > > I've only seen a few, rare, cases of the latter. > > Thanks, > Dan > > On Sun, Feb 5, 2017 at 9:19 PM, Prabeesh K. <[email protected]> wrote: > >> Hi Ankur, >> >> Thank you for your response. >> >> On 5 February 2017 at 23:59, Ankur Chauhan <[email protected]> wrote: >> >>> I have found doing joins in bigquery using sql is a lot faster and >>> easier to iterate upon. >>> >>> >>> Ankur Chauhan >>> On Sat, Feb 4, 2017 at 22:05 Prabeesh K. <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> Which is the better way to join two tables in apache beam? >>>> >>>> Regards, >>>> Prabeesh K. >>>> >>> >> >
