Isn't there the new Kafka plugin? What does that exactly do? Best, Saurabh
Sent from my iPhone > On Nov 9, 2017, at 5:15 AM, kant kodali <[email protected]> wrote: > > Hi Tug, > > It's Parquet data on HDFS and the data to HDFS is constantly written by > spark while consuming from Kafka. > > Is polling a common technique for say real time analytics dashboard ? More > importantly if I poll does Drill due the scan every time? if the answer is > no, how does it know which is the new data? since the data is written HDFS > constantly as a stream (The query can be the same however the new data will > be appended or updated to HDFS in parquet format as a stream). > > Thanks! > >> On Thu, Nov 9, 2017 at 4:47 AM, Tugdual Grall <[email protected]> wrote: >> >> Hello, >> >> >> Today Drill cannot do continuous/streaming query, so as you mentioned you >> will have to use a polling technique. >> >> >> Just out of curiosity, Which data source are you planning to use ? >> >> Regards >> Tug >> >> >> >> >>> On Thu 9 Nov 2017 at 04:31, kant kodali <[email protected]> wrote: >>> >>> Hi All, >>> >>> I am new to Apache Drill. I am wondering if Apache Drill can perform >>> Streaming Queries? For example, I have a constant stream of data in 24 >> hour >>> period and I would like to get updates as soon as I receive them. >>> >>> Do I need to have a polling thread that issues a Drill query every >> second? >>> >>> Thanks! >>> >>
