Re: Segregating Foreman from leaf-worker fleet

2018-11-13 Thread Paul Rogers
Hi Lokendra, Your usecase is a typical old school sharded DB app. The design itself is fine. However, as Tim noted, Drill is not designed for this case. Still, perhaps Drill could be extended. As Tim suggested, Drill assumes any Drillbit can operate in any role. So, in your setup, you would

Re: Segregating Foreman from leaf-worker fleet

2018-11-13 Thread Lokendra Singh Panwar
Hi Tim, Thanks for the reply. My usecase is following: - My main DB table is huge so it is sharded amongs multiple storage-nodes. - Each stroage-node is storing the assigned shard in a local relational db engine. I was planning to use Drill as a distributed query engine that can

Re: Segregating Foreman from leaf-worker fleet

2018-11-13 Thread Timothy Farkas
Hi Lokendra, All Drillbits can function as a foreman if a query is sent to them, and all drillbits are considered worker nodes. This ingrained deeply into the design of Drill and it was done with the intention of making Drill symmetric. Symmetric here means that each Drillbit is identical to all

Segregating Foreman from leaf-worker fleet

2018-11-13 Thread Lokendra Singh Panwar
Hi, Is it possible to configure Drill such that the Foreman and leaf-worker fleets are separate fleets of nodes? Or if this needs changing the source of Drill, any pointers are appreciated too. Thanks, Lokendra