Can someone elaborate on what happens underneath if I poll every second (Specifically related to my questions in my previous email)?
Thanks! On Thu, Nov 9, 2017 at 7:56 AM, Ted Dunning <[email protected]> wrote: > Confluent has a non-Apache product, I think, for streaming SQL. > > > On Thu, Nov 9, 2017 at 4:50 PM, Saurabh Mahapatra <[email protected]> > wrote: > > > Isn't there the new Kafka plugin? What does that exactly do? > > > > Best, > > Saurabh > > > > Sent from my iPhone > > > > > > > > > On Nov 9, 2017, at 5:15 AM, kant kodali <[email protected]> wrote: > > > > > > Hi Tug, > > > > > > It's Parquet data on HDFS and the data to HDFS is constantly written by > > > spark while consuming from Kafka. > > > > > > Is polling a common technique for say real time analytics dashboard ? > > More > > > importantly if I poll does Drill due the scan every time? if the answer > > is > > > no, how does it know which is the new data? since the data is written > > HDFS > > > constantly as a stream (The query can be the same however the new data > > will > > > be appended or updated to HDFS in parquet format as a stream). > > > > > > Thanks! > > > > > >> On Thu, Nov 9, 2017 at 4:47 AM, Tugdual Grall <[email protected]> > > wrote: > > >> > > >> Hello, > > >> > > >> > > >> Today Drill cannot do continuous/streaming query, so as you mentioned > > you > > >> will have to use a polling technique. > > >> > > >> > > >> Just out of curiosity, Which data source are you planning to use ? > > >> > > >> Regards > > >> Tug > > >> > > >> > > >> > > >> > > >>> On Thu 9 Nov 2017 at 04:31, kant kodali <[email protected]> wrote: > > >>> > > >>> Hi All, > > >>> > > >>> I am new to Apache Drill. I am wondering if Apache Drill can perform > > >>> Streaming Queries? For example, I have a constant stream of data in > 24 > > >> hour > > >>> period and I would like to get updates as soon as I receive them. > > >>> > > >>> Do I need to have a polling thread that issues a Drill query every > > >> second? > > >>> > > >>> Thanks! > > >>> > > >> > > >
