Kathy's patch was exposing metrics, which is fairly different because it was exposing metrics from the backend. The java data source would have just had to call back into C++. It also needed to run on every backend, so would have required extending the data source API. That might be an argument for a C++ data source API (or at least a well-defined interface for implementing system tables as classes in C++, which it was heading towards).
IIRC MJ had a very old prototype of using the external data source to expose local catalog state (which is all Java). On Wed, Feb 7, 2018 at 11:38 AM, Daniel Hecht <dhe...@cloudera.com> wrote: > On Wed, Feb 7, 2018 at 9:18 AM, Marcel Kornacker <marc...@gmail.com> > wrote: > >> What is gained by removing the code? >> >> > The motivation was that we should always be looking for opportunities to > remove code, since we can't just keep adding or else the complexity of > Impala will get out of control over the long term. This was an opportunity > to remove ~5k lines, which is a nice benefit if the feature wasn't being > used. > > Regarding information schema, I definitely agree that is an interesting > and useful feature. There was a prototype of parts of that here > https://gerrit.cloudera.org/#/c/3863/19 but it did not rely on the > external data source. I don't remember whether there was a technical reason > why and that's something we can try to understand to see if external > datasource needs to evolve or we'd be better starting from scratch. > > Shant, thanks for chiming in with your take as well and that you have > found it useful and are in favor of keeping it. And Matt, thanks for some > of the historical context and your opinion. Of course, feel free to > contribute. >
