Yes, it certainly is a trade-off. Hence we want the execution engine to support different types of connectors/channels between the operators. In-memory is one option. On-disk (eg, what Hadoop's MapReduce uses) is another option.
On Mon, Aug 27, 2012 at 12:05 PM, David Gruzman <[email protected]>wrote: > I think that execution engine should trade fault tolerance to low latency. > If query takes a few seconds - it > easier to re-run it then build intra query fault tolerance. > David > > On Mon, Aug 27, 2012 at 9:53 PM, Julien Le Dem <[email protected]> wrote: > > > Hi Ted > > As long as this plan has primitives like "group by" and "filter" it > > would be relatively easy to wrap Pig operators into it. We did > > something similar to run Pig on Spark. If performance is an issue, > > separate execution engines can also coexist. > > Julien > > > > On Sun, Aug 26, 2012 at 2:22 PM, Ted Dunning <[email protected]> > > wrote: > > > Camuel, > > > > > > Do you have a grammar test suite that demonstrates the range of > > expressions? > > > > > > Also, I believe that some have a goal to use additional languages > besides > > > SQL like languages. A limited version of pig, for instance, would be > > very > > > interesting. To do this, it will be important to have a logical plan > > > structure that is common for different syntaxes and is not limited to > the > > > idiosyncracies of any particular syntax. > > > > > > How do you think that should be handled? Do you have an idea for a > > logical > > > plan structure? > > > > > > On Sun, Aug 26, 2012 at 4:11 PM, Camuel Gilyadov <[email protected]> > > wrote: > > > > > >> I've written and attached ANTLR grammar for DrQL which I assume is > same > > as > > >> BigQuery language described in Query Reference on BigQuery website. > This > > >> grammar includes AST production rules. > > >> > > >> > > >> > > > -- Tomer Shiran Director of Product Management | MapR Technologies | 650-804-8657
