Hi Bosco, Do you know if there's any kind of table with all compatible version between Hive and Ranger?
Thanks On Sat, 23 Mar 2019 at 16:18, Don Bosco Durai <bo...@apache.org> wrote: > Hi Odon > > > > Good it worked. > > > > Hive is has been more stable recently. In Hive 1.2, the API to get remote > address is not there, so you would get runtime error, which you can capture > and ignore them. > > > > Hive team have added more hook points in recent versions. Some of them are > for row level filtering and column masking. So if you are not going to use > them, then you should be fine. > > > > Bosco > > > > > > *From: *Odon Copon <odonco...@gmail.com> > *Reply-To: *<user@ranger.apache.org> > *Date: *Friday, March 22, 2019 at 11:11 AM > *To: *<user@ranger.apache.org> > *Subject: *Re: Hadoop version > > > > Hi Bosco, > > Just noticed I never updated the thread with the final results. Well, it > worked in the end with Ranger latest version and Hadoop >= 2.7. > > What you said about the interfaces and the versions was the key. After > that, with newer versions, it worked fine. > > > > At the moment, I'm testing Ranger + Hive. Do you know if versions are > being affected here as well between Hive and Ranger? > > > > Thanks! > > > > > > On Wed, 16 Jan 2019 at 22:05, Don Bosco Durai <bo...@apache.org> wrote: > > > Tried with Ranger 1.2.0 and Hadoop 2.7 and seems that loads the plugin, > because saw some info logs regarding Ranger. > > This might work. I don’t think the interface changed within Hadoop 2.7 and > Hadoop 3x. If the interface changed, you would have got runtime error and > namenode would have not come up. You can look at the Ranger Admin portal > and see if the Audit->Plugin got any request from the HDFS plugin. > > > > > If I pick and old version, should I modify also the pom or that doesn't > influence? > > Sorry, just the changing the pom will not work. It is entirely different > implementation. > > > > Bosco > > > > > > > > *From: *Odon Copon <odonco...@gmail.com> > *Reply-To: *<user@ranger.apache.org> > *Date: *Wednesday, January 16, 2019 at 1:53 PM > *To: *<user@ranger.apache.org> > *Subject: *Re: Hadoop version > > > > Thanks, that makes sense. > > Tried with Ranger 1.2.0 and Hadoop 2.7 and seems that loads the plugin, > because saw some info logs regarding Ranger. Does that make sense, or will > load but not work eventually due to the old Hadoop version? > > Will try tomorrow with 2.6 and an old Ranger version. > > If I pick and old version, should I modify also the pom or that doesn't > influence? > > On Wed, 16 Jan 2019, 21:44 Don Bosco Durai <bo...@apache.org wrote: > > This might explain why the latest Ranger is not working for you. During > Hadoop 2.6, we didn’t have official interface from the Hadoop team, so we > implemented using code injection at runtime. So for Hadoop 2.6 you might > have to go back to Ranger release 0.4. Just recompiling the latest with the > older Hadoop version will not work… > > > > Bosco > > > > > > *From: *Odon Copon <odonco...@gmail.com> > *Reply-To: *<user@ranger.apache.org> > *Date: *Wednesday, January 16, 2019 at 12:17 PM > *To: *<user@ranger.apache.org> > *Subject: *Re: Hadoop version > > > > Thanks Bosco, > > I'm using Hadoop 2.6.0 at the moment, and thinking on upgrading to 2.7.1 > at some point soon. > > What version should I try while I'm using 2.6? > > Should I modify the pom as well to reflect the exact same Hadoop version I > have install in my system when building Ranger? Because I see 2.7 in the > pom for almost all Ranger releases. > > > > On Wed, 16 Jan 2019 at 19:56, Don Bosco Durai <bo...@apache.org> wrote: > > If you are using Hadoop 2.7, then Ranger 0.7 is the best bet. Let us know > which version you are using… > > > > As Loïc mentioned, where possible we take effort to provide backward > compatibility. But sometimes, the plugin interface from component/service > might change dramatically between releases. In those cases, we might not be > able to provide backward compatibility. But we ensured Ranger Admin API > level backward compatibility, so that you old plugins can communicate with > new version of Ranger Admin. > > > > Bosco > > > > > > *From: *Odon Copon <odonco...@gmail.com> > *Reply-To: *<user@ranger.apache.org> > *Date: *Wednesday, January 16, 2019 at 8:26 AM > *To: *<user@ranger.apache.org> > *Subject: *Re: Hadoop version > > > > Thanks. > > Knowing which Hadoop version supports each Ranger version would be > amazing. Because at the moment I'm having troubles with the installation > and I'm starting thinking it must be due to versioning issues. > > > > On Wed, 16 Jan 2019 at 15:33, Loïc Chanel <loic.cha...@telecomnancy.net> > wrote: > > Yes, this is just for Hortonworks distribution but you can assume that as > far as Ranger is concerned some backward compliancy have been implemented, > so Ranger 0.6.0 supports Hadoop 2.7.3 and older versions of Hadoop, and > Ranger 1.0 supports Hadoop 3.1.0 and older ... but Bosco know this better > than I do so I'll let him confirm. > > Best regards, > > > > > Loïc CHANEL > System Big Data engineer > Vision 360 Degrés (Lyon, France) > > > > > > Le mer. 16 janv. 2019 à 14:58, Odon Copon <odonco...@gmail.com> a écrit : > > Oh true, didn't see that Hadoop column. > > But this is just for Hortonworks distribution, right? I can see per > example, in Ranger POM file on 1.2.0 release that contains a reference to > Hadoop 2.7.1, so I'm assuming Apache Ranger 1.2.0 supports Hadoop 2.7.1. > > > > On Wed, 16 Jan 2019 at 13:49, Loïc Chanel <loic.cha...@telecomnancy.net> > wrote: > > Hi, > > > > I think there was a misunderstanding. What you see on the left border is > the HDP version. As HDP is a stack (a product packaged by Hortonworks), it > comes with a lot of components (Hive, HBase, Spark ...) including Ranger > and Hadoop (in the HDP core column, on the left side of the picture). > > Therefore, you can see on the picture that Hadoop 2.7.3 is supported by > both Ranger 0.6.0 and Ranger 0.7.0 versions, as they are packaged in the > same HDP stack. > > > > Hope this helps. > > > > > Loïc CHANEL > System Big Data engineer > Vision 360 Degrés (Lyon, France) > > > > > > Le mer. 16 janv. 2019 à 13:38, Odon Copon <odonco...@gmail.com> a écrit : > > Thanks Loic! > > That's definitely really helpful. But let me confirm and clarify what I'm > seeing there: > > - ranger 0.6 just works with hadoop 2.5.* > > - ranger 0.7 just works with hadoop 2.6.4 and 2.6.5 > > - ranger 1.1.0 just works with hadoop 3.0.0 > > - ranger 1.2.0 just works with hadoop 3.1.* > > > > Is this correct? > So Hadoop 2.7.* is not supported? > > > > > > On Wed, 16 Jan 2019 at 12:07, Loïc Chanel <loic.cha...@telecomnancy.net> > wrote: > > Hi Odon, > > > > Here it is : > > *Error! Filename not specified.* > > (https://fr.hortonworks.com/products/data-platforms/hdp/ ) > > > > Best regards, > > > > > Loïc CHANEL > System Big Data engineer > Vision 360 Degrés (Lyon, France) > > > > > > Le mer. 16 janv. 2019 à 11:53, Odon Copon <odonco...@gmail.com> a écrit : > > Thanks! > > Do you know if there's a table with Hadoop versions and the corresponding > Ranger version? > > > > On Wed, 16 Jan 2019 at 10:41, Don Bosco Durai <bo...@apache.org> wrote: > > Ranger implements the interface published by the service component. So > every Ranger version can only support the components which it is built > against. > > > > For older version of components, you will have to use older version of > Ranger which supports it. > > > > Bosco > > > > > > *From: *Odon Copon <odonco...@gmail.com> > *Reply-To: *<user@ranger.apache.org> > *Date: *Wednesday, January 16, 2019 at 2:03 AM > *To: *<user@ranger.apache.org> > *Subject: *Hadoop version > > > > Hi, > > Is there any requirement on the Hadoop version supported by Apache Ranger? > I can see the pom.xml contains a 2.7.1 pinned version for Hadoop, but my > question is, would it work on older versions like 2.6? Or is a hard > requirement to be >= 2.7.1? > > Thanks. > >
