Hi Odon
Good it worked. Hive is has been more stable recently. In Hive 1.2, the API to get remote address is not there, so you would get runtime error, which you can capture and ignore them. Hive team have added more hook points in recent versions. Some of them are for row level filtering and column masking. So if you are not going to use them, then you should be fine. Bosco From: Odon Copon <odonco...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Friday, March 22, 2019 at 11:11 AM To: <user@ranger.apache.org> Subject: Re: Hadoop version Hi Bosco, Just noticed I never updated the thread with the final results. Well, it worked in the end with Ranger latest version and Hadoop >= 2.7. What you said about the interfaces and the versions was the key. After that, with newer versions, it worked fine. At the moment, I'm testing Ranger + Hive. Do you know if versions are being affected here as well between Hive and Ranger? Thanks! On Wed, 16 Jan 2019 at 22:05, Don Bosco Durai <bo...@apache.org> wrote: > Tried with Ranger 1.2.0 and Hadoop 2.7 and seems that loads the plugin, > because saw some info logs regarding Ranger. This might work. I don’t think the interface changed within Hadoop 2.7 and Hadoop 3x. If the interface changed, you would have got runtime error and namenode would have not come up. You can look at the Ranger Admin portal and see if the Audit->Plugin got any request from the HDFS plugin. > If I pick and old version, should I modify also the pom or that doesn't > influence? Sorry, just the changing the pom will not work. It is entirely different implementation. Bosco From: Odon Copon <odonco...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Wednesday, January 16, 2019 at 1:53 PM To: <user@ranger.apache.org> Subject: Re: Hadoop version Thanks, that makes sense. Tried with Ranger 1.2.0 and Hadoop 2.7 and seems that loads the plugin, because saw some info logs regarding Ranger. Does that make sense, or will load but not work eventually due to the old Hadoop version? Will try tomorrow with 2.6 and an old Ranger version. If I pick and old version, should I modify also the pom or that doesn't influence? On Wed, 16 Jan 2019, 21:44 Don Bosco Durai <bo...@apache.org wrote: This might explain why the latest Ranger is not working for you. During Hadoop 2.6, we didn’t have official interface from the Hadoop team, so we implemented using code injection at runtime. So for Hadoop 2.6 you might have to go back to Ranger release 0.4. Just recompiling the latest with the older Hadoop version will not work… Bosco From: Odon Copon <odonco...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Wednesday, January 16, 2019 at 12:17 PM To: <user@ranger.apache.org> Subject: Re: Hadoop version Thanks Bosco, I'm using Hadoop 2.6.0 at the moment, and thinking on upgrading to 2.7.1 at some point soon. What version should I try while I'm using 2.6? Should I modify the pom as well to reflect the exact same Hadoop version I have install in my system when building Ranger? Because I see 2.7 in the pom for almost all Ranger releases. On Wed, 16 Jan 2019 at 19:56, Don Bosco Durai <bo...@apache.org> wrote: If you are using Hadoop 2.7, then Ranger 0.7 is the best bet. Let us know which version you are using… As Loïc mentioned, where possible we take effort to provide backward compatibility. But sometimes, the plugin interface from component/service might change dramatically between releases. In those cases, we might not be able to provide backward compatibility. But we ensured Ranger Admin API level backward compatibility, so that you old plugins can communicate with new version of Ranger Admin. Bosco From: Odon Copon <odonco...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Wednesday, January 16, 2019 at 8:26 AM To: <user@ranger.apache.org> Subject: Re: Hadoop version Thanks. Knowing which Hadoop version supports each Ranger version would be amazing. Because at the moment I'm having troubles with the installation and I'm starting thinking it must be due to versioning issues. On Wed, 16 Jan 2019 at 15:33, Loïc Chanel <loic.cha...@telecomnancy.net> wrote: Yes, this is just for Hortonworks distribution but you can assume that as far as Ranger is concerned some backward compliancy have been implemented, so Ranger 0.6.0 supports Hadoop 2.7.3 and older versions of Hadoop, and Ranger 1.0 supports Hadoop 3.1.0 and older ... but Bosco know this better than I do so I'll let him confirm. Best regards, Loïc CHANEL System Big Data engineer Vision 360 Degrés (Lyon, France) Le mer. 16 janv. 2019 à 14:58, Odon Copon <odonco...@gmail.com> a écrit : Oh true, didn't see that Hadoop column. But this is just for Hortonworks distribution, right? I can see per example, in Ranger POM file on 1.2.0 release that contains a reference to Hadoop 2.7.1, so I'm assuming Apache Ranger 1.2.0 supports Hadoop 2.7.1. On Wed, 16 Jan 2019 at 13:49, Loïc Chanel <loic.cha...@telecomnancy.net> wrote: Hi, I think there was a misunderstanding. What you see on the left border is the HDP version. As HDP is a stack (a product packaged by Hortonworks), it comes with a lot of components (Hive, HBase, Spark ...) including Ranger and Hadoop (in the HDP core column, on the left side of the picture). Therefore, you can see on the picture that Hadoop 2.7.3 is supported by both Ranger 0.6.0 and Ranger 0.7.0 versions, as they are packaged in the same HDP stack. Hope this helps. Loïc CHANEL System Big Data engineer Vision 360 Degrés (Lyon, France) Le mer. 16 janv. 2019 à 13:38, Odon Copon <odonco...@gmail.com> a écrit : Thanks Loic! That's definitely really helpful. But let me confirm and clarify what I'm seeing there: - ranger 0.6 just works with hadoop 2.5.* - ranger 0.7 just works with hadoop 2.6.4 and 2.6.5 - ranger 1.1.0 just works with hadoop 3.0.0 - ranger 1.2.0 just works with hadoop 3.1.* Is this correct? So Hadoop 2.7.* is not supported? On Wed, 16 Jan 2019 at 12:07, Loïc Chanel <loic.cha...@telecomnancy.net> wrote: Hi Odon, Here it is : Error! Filename not specified. (https://fr.hortonworks.com/products/data-platforms/hdp/ ) Best regards, Loïc CHANEL System Big Data engineer Vision 360 Degrés (Lyon, France) Le mer. 16 janv. 2019 à 11:53, Odon Copon <odonco...@gmail.com> a écrit : Thanks! Do you know if there's a table with Hadoop versions and the corresponding Ranger version? On Wed, 16 Jan 2019 at 10:41, Don Bosco Durai <bo...@apache.org> wrote: Ranger implements the interface published by the service component. So every Ranger version can only support the components which it is built against. For older version of components, you will have to use older version of Ranger which supports it. Bosco From: Odon Copon <odonco...@gmail.com> Reply-To: <user@ranger.apache.org> Date: Wednesday, January 16, 2019 at 2:03 AM To: <user@ranger.apache.org> Subject: Hadoop version Hi, Is there any requirement on the Hadoop version supported by Apache Ranger? I can see the pom.xml contains a 2.7.1 pinned version for Hadoop, but my question is, would it work on older versions like 2.6? Or is a hard requirement to be >= 2.7.1? Thanks.
