Yes, this is the only way today. However, with completion of DRILL-77 <https://issues.apache.org/jira/browse/DRILL-77>, you can use REST APIs as well.
On Sat, Jun 28, 2014 at 10:44 PM, mufy <[email protected]> wrote: > I have seen that. Is that the only way of adding new storage plugins then? > > > --- > Mufeed Usman > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My > Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal > <http://mufeed.livejournal.com> > > > > > > On Sat, Jun 28, 2014 at 10:13 PM, Ramana Inukonda <[email protected]> > wrote: > > > Jason's comments are right for a clustered environment as well. Just > > substitute localhost with any of the drillbit nodes in the cluster to > > access the web UI. > > > > Regards > > Ramana > > > > On Saturday, June 28, 2014, Jason Altekruse <[email protected]> > > wrote: > > > > > We have added a web server to Drill for showing query status > information > > as > > > well as setting configuration options like storage plugins. To view the > > > storage plugin management interface you can go to localhost:8047 if you > > are > > > running in local mode. If you are running from a cluster I believe you > > > should be able to just substitute any of the IP addresses of the nodes, > > but > > > I haven't been working with Drill in a clustered environment, so > someone > > > else might have to chime in with more info. > > > > > > -Jason > > > > > > > > > On Sat, Jun 28, 2014 at 7:12 AM, mufy <[email protected] > > > <javascript:;>> wrote: > > > > > > > So as an end-user where exactly do I add my new storage plugin > > > > configurations if I choose to add them in the future? > > > > > > > > > > > > --- > > > > Mufeed Usman > > > > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | > My > > > > Social Cause <http://www.vision2016.org.in/> | My Blogs : > LiveJournal > > > > <http://mufeed.livejournal.com> > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Jun 28, 2014 at 3:17 AM, Aditya <[email protected] > > > <javascript:;>> wrote: > > > > > > > > > To be more precise, the configuration file "storage-plugins.json" > has > > > > been > > > > > replaced with "bootstrap-storage-plugins.json" which resides inside > > the > > > > > Drill jars. > > > > > > > > > > When a cluster is started for the first time and no existing > storage > > > > plugin > > > > > definition is found in the Zookeeper, this file will be read and > > copied > > > > > into the Zookeeper. From that point onward, the all Drillbits in > the > > > > > cluster will use only the information stored in the ZK and would > > > ignore " > > > > > bootstrap-storage-plugins.json" as well. > > > > > > > > > > > > > > > On Fri, Jun 27, 2014 at 2:36 PM, Aditya <[email protected] > > > <javascript:;>> wrote: > > > > > > > > > > > The configuration file "storage-plugins.json" is no longer used > by > > > > Drill. > > > > > > > > > > > > Any change in the plugin configuration is stored only in the > > > persistent > > > > > > store (Zookeeper by default). > > > > > > > > > > > > > > > > > > On Fri, Jun 27, 2014 at 12:09 AM, mufy <[email protected] > > > <javascript:;>> wrote: > > > > > > > > > > > >> Hello Anil, > > > > > >> > > > > > >> Hmm... so the ZK will have the info. Agreed. But what about the > > > > > statement > > > > > >> where it says that any new storage plugin added/updated will be > > > > > broadcast > > > > > >> to all the nodes running drillbit service. I took that statement > > to > > > > > mean - > > > > > >> the config file will be synchronized across all the nodes > running > > > the > > > > > >> drillbit service. Did I get it wrong? > > > > > >> > > > > > >> > > > > > >> --- > > > > > >> Mufeed Usman > > > > > >> My LinkedIn < > http://www.linkedin.com/pub/mufeed-usman/28/254/400> > > | > > > > My > > > > > >> Social Cause <http://www.vision2016.org.in/> | My Blogs : > > > LiveJournal > > > > > >> <http://mufeed.livejournal.com> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> On Fri, Jun 27, 2014 at 11:57 AM, AnilKumar B < > > > [email protected] <javascript:;>> > > > > > >> wrote: > > > > > >> > > > > > >> > Hi Murfy, > > > > > >> > > > > > > >> > Based on my understanding, registered storage plugin info will > > be > > > > > >> stored on > > > > > >> > ZK. As below. > > > > > >> > > > > > > >> > [zk: localhost:2181(CONNECTED) 10] ls > /drill/sys.storage_plugins > > > > > >> > [hive, cp, dfs, hbase] > > > > > >> > [zk: localhost:2181(CONNECTED) 11] get > > > > > /drill/sys.storage_plugins/hbase > > > > > >> > { > > > > > >> > "type" : "hbase", > > > > > >> > "config" : { > > > > > >> > "hbase.zookeeper.quorum" : "localhost", > > > > > >> > "hbase.zookeeper.property.clientPort" : "2181" > > > > > >> > }, > > > > > >> > "enabled" : true > > > > > >> > } > > > > > >> > cZxid = 0x428 > > > > > >> > ctime = Tue Jun 24 13:30:18 IST 2014 > > > > > >> > mZxid = 0x450 > > > > > >> > mtime = Tue Jun 24 13:31:41 IST 2014 > > > > > >> > pZxid = 0x428 > > > > > >> > cversion = 0 > > > > > >> > dataVersion = 1 > > > > > >> > aclVersion = 0 > > > > > >> > ephemeralOwner = 0x0 > > > > > >> > dataLength = 157 > > > > > >> > numChildren = 0 > > > > > >> > [zk: localhost:2181(CONNECTED) 12] get > > > /drill/sys.storage_plugins/cp > > > > > >> > { > > > > > >> > "type" : "file", > > > > > >> > "enabled" : true, > > > > > >> > "connection" : "classpath:///", > > > > > >> > "workspaces" : null, > > > > > >> > "formats" : null > > > > > >> > } > > > > > >> > cZxid = 0x426 > > > > > >> > ctime = Tue Jun 24 13:30:18 IST 2014 > > > > > >> > mZxid = 0x426 > > > > > >> > mtime = Tue Jun 24 13:30:18 IST 2014 > > > > > >> > pZxid = 0x426 > > > > > >> > cversion = 0 > > > > > >> > dataVersion = 0 > > > > > >> > aclVersion = 0 > > > > > >> > ephemeralOwner = 0x0 > > > > > >> > dataLength = 118 > > > > > >> > numChildren = 0 > > > > > >> > [zk: localhost:2181(CONNECTED) 13] > > > > > >> > > > > > > >> > Thanks, > > > > > >> > B Anil Kumar. > > > > > >> > > > > > > >> > Thanks & Regards, > > > > > >> > B Anil Kumar. > > > > > >> > > > > > > >> > > > > > > >> > On Fri, Jun 27, 2014 at 11:12 AM, mufy < > [email protected] > > > <javascript:;>> > > > > > wrote: > > > > > >> > > > > > > >> > > Stated here > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/DRILL/Connecting+Apache+Drill+to+Data+Sources > > > > > >> > > that, > > > > > >> > > > > > > > >> > > "When you add or update storage plugin instances on one > node, > > > > Drill > > > > > >> > > broadcasts the information to all of the other Drill nodes > so > > > they > > > > > >> have > > > > > >> > > identical storage plugin instances registered. You do not > need > > > to > > > > > >> restart > > > > > >> > > any of the Drillbits when you add or update a storage plugin > > > > > >> instance." > > > > > >> > > > > > > > >> > > Does that mean if I were to have the following on one node: > > > > > >> > > [root@mufeed-drill1 ~]# cat > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > /opt/mapr/drill/drill-1.0.0/apache-drill-1.0.0-m2-incubating-SNAPSHOT/conf/storage-plugins.json > > > > > >> > > { > > > > > >> > > "storage": { > > > > > >> > > > > > > > >> > > dfs: { > > > > > >> > > type: "file", > > > > > >> > > connection: "maprfs:///" > > > > > >> > > }, > > > > > >> > > > > > > > >> > > cp: { > > > > > >> > > type: "file", > > > > > >> > > connection: "classpath:///" > > > > > >> > > }, > > > > > >> > > > > > > > >> > > cp1: { > > > > > >> > > type: "file", > > > > > >> > > connection: "classpath:///" > > > > > >> > > }, > > > > > >> > > } > > > > > >> > > } > > > > > >> > > > > > > > >> > > The same content would get replicated to other nodes, on a > > file > > > > > >> content > > > > > >> > > level? If yes, I do not see that happening. If not, please > > > > clarify. > > > > > >> > > > > > > > >> > > --- > > > > > >> > > Mufeed Usman > > > > > >> > > My LinkedIn < > > > http://www.linkedin.com/pub/mufeed-usman/28/254/400> > > > > | > > > > > >> My > > > > > >> > > Social Cause <http://www.vision2016.org.in/> | My Blogs : > > > > > LiveJournal > > > > > >> > > <http://mufeed.livejournal.com> > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > >
