Hi Rajat,

The exception is thrown when deserializing Kafka notification message. This is 
probably because of mismatch between libraries used by Atlas 1.0 for 
serialization and used by Ranger 0.7 for deserialization.

You may add JSON serialization/deserialization libraries used by Atlas to 
tagsync-install-dir/lib directory and retest.

-Abhay


From: Rajat Goel <rajat.g...@guavus.com<mailto:rajat.g...@guavus.com>>
Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" 
<user@ranger.apache.org<mailto:user@ranger.apache.org>>
Date: Tuesday, June 12, 2018 at 10:57 AM
To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" 
<user@ranger.apache.org<mailto:user@ranger.apache.org>>
Subject: Re: Custom service integration and Tag based policies

Hi Abhay,

I did everything mentioned in the steps below – Created my custom 
implementation of AtlasResourceMapper, added it in the lib and configured the 
Tagsync config parameter. However, I am seeing an exception in my 
<>/tagsync.log file:

12 Jun 2018 13:06:39 ERROR AtlasTagSource [Thread-7] - 187 Caught exception..:
org.json4s.package$MappingException: No usable value for values
Expected object but got JNothing
        at org.json4s.reflect.package$.fail(package.scala:96)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:462)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$14.apply(Extraction.scala:482)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$14.apply(Extraction.scala:482)
        at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
        at scala.collection.AbstractTraversable.map(Traversable.scala:104)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$instantiate(Extraction.scala:470)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$result$6.apply(Extraction.scala:515)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$result$6.apply(Extraction.scala:512)
        at 
org.json4s.Extraction$.org$json4s$Extraction$$customOrElse(Extraction.scala:524)
        at 
org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:512)
        at org.json4s.Extraction$.extract(Extraction.scala:351)
        at org.json4s.Extraction$.extract(Extraction.scala:42)
        at 
org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
        at org.json4s.native.Serialization$.read(Serialization.scala:71)
        at 
org.apache.atlas.typesystem.json.InstanceSerialization$.fromJsonReferenceable(InstanceSerialization.scala:371)
        at 
org.apache.atlas.typesystem.json.InstanceSerialization.fromJsonReferenceable(InstanceSerialization.scala)
        at 
org.apache.atlas.notification.AbstractMessageDeserializer$ReferenceableDeserializer.deserialize(AbstractMessageDeserializer.java:162)
        at 
org.apache.atlas.notification.AbstractMessageDeserializer$ReferenceableDeserializer.deserialize(AbstractMessageDeserializer.java:157)
        at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
        at com.google.gson.Gson.fromJson(Gson.java:803)
        at com.google.gson.Gson.fromJson(Gson.java:868)
        at com.google.gson.Gson$1.deserialize(Gson.java:126)
        at 
org.apache.atlas.notification.entity.EntityMessageDeserializer$EntityNotificationDeserializer.deserialize(EntityMessageDeserializer.java:73)
        at 
org.apache.atlas.notification.entity.EntityMessageDeserializer$EntityNotificationDeserializer.deserialize(EntityMessageDeserializer.java:69)
        at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
        at com.google.gson.Gson.fromJson(Gson.java:803)
        at com.google.gson.Gson.fromJson(Gson.java:868)
        at com.google.gson.Gson$1.deserialize(Gson.java:126)
        at 
org.apache.atlas.notification.entity.EntityMessageDeserializer$EntityNotificationDeserializer.deserialize(EntityMessageDeserializer.java:73)
        at 
org.apache.atlas.notification.entity.EntityMessageDeserializer$EntityNotificationDeserializer.deserialize(EntityMessageDeserializer.java:69)
        at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
        at com.google.gson.Gson.fromJson(Gson.java:803)
        at com.google.gson.Gson.fromJson(Gson.java:768)
        at com.google.gson.Gson.fromJson(Gson.java:717)
        at 
org.apache.atlas.notification.AtlasNotificationMessageDeserializer.deserialize(AtlasNotificationMessageDeserializer.java:210)
        at 
org.apache.atlas.kafka.AtlasKafkaConsumer.receive(AtlasKafkaConsumer.java:72)
        at 
org.apache.ranger.tagsync.source.atlas.AtlasTagSource$ConsumerRunnable.run(AtlasTagSource.java:165)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.json4s.package$MappingException: Expected object but got JNothing
        at org.json4s.Extraction$.extract(Extraction.scala:336)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:450)
        ... 39 more

My Apache Ranger version is 0.7.0 and Atlas version is 1.0.0.

I thought this may be due to incompatible Atlas Notification message in Kafka 
so I added the following config in Atlas:

 atlas.notification.entity.version=v1

Then I deleted ATLAS_ENTITIES kafka topic to flush old messages, restarted 
Atlas, Ranger and retried. Again, I am seeing similar exception:

12 Jun 2018 20:19:22 ERROR AtlasTagSource [Thread-7] - 187 Caught exception..:
org.json4s.package$MappingException: No usable value for traits
No usable value for values
Expected object but got JNull
        at org.json4s.reflect.package$.fail(package.scala:96)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:462)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$14.apply(Extraction.scala:482)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$14.apply(Extraction.scala:482)
        at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
        at scala.collection.AbstractTraversable.map(Traversable.scala:104)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$instantiate(Extraction.scala:470)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$result$6.apply(Extraction.scala:515)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$result$6.apply(Extraction.scala:512)
        at 
org.json4s.Extraction$.org$json4s$Extraction$$customOrElse(Extraction.scala:524)
        at 
org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:512)
        at org.json4s.Extraction$.extract(Extraction.scala:351)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$mkWithTypeHint(Extraction.scala:507)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$result$6.apply(Extraction.scala:514)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$result$6.apply(Extraction.scala:512)
        at 
org.json4s.Extraction$.org$json4s$Extraction$$customOrElse(Extraction.scala:524)
        at 
org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:512)
        at org.json4s.Extraction$.extract(Extraction.scala:351)
        at org.json4s.Extraction$.extract(Extraction.scala:42)
        at 
org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
        at org.json4s.native.Serialization$.read(Serialization.scala:71)
        at 
org.apache.atlas.typesystem.json.InstanceSerialization$.fromJsonReferenceable(InstanceSerialization.scala:371)
        at 
org.apache.atlas.typesystem.json.InstanceSerialization.fromJsonReferenceable(InstanceSerialization.scala)
        at 
org.apache.atlas.notification.AbstractMessageDeserializer$ReferenceableDeserializer.deserialize(AbstractMessageDeserializer.java:162)

        at 
org.apache.atlas.notification.AbstractMessageDeserializer$ReferenceableDeserializer.deserialize(AbstractMessageDeserializer.java:157)
        at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
        at com.google.gson.Gson.fromJson(Gson.java:803)
        at com.google.gson.Gson.fromJson(Gson.java:868)
        at com.google.gson.Gson$1.deserialize(Gson.java:126)
        at 
org.apache.atlas.notification.entity.EntityMessageDeserializer$EntityNotificationDeserializer.deserialize(EntityMessageDeserializer.java:73)
        at 
org.apache.atlas.notification.entity.EntityMessageDeserializer$EntityNotificationDeserializer.deserialize(EntityMessageDeserializer.java:69)
        at com.google.gson.TreeTypeAdapter.read(TreeTypeAdapter.java:58)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
        at 
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
        at com.google.gson.Gson.fromJson(Gson.java:803)
        at com.google.gson.Gson.fromJson(Gson.java:768)
        at com.google.gson.Gson.fromJson(Gson.java:717)
        at 
org.apache.atlas.notification.AtlasNotificationMessageDeserializer.deserialize(AtlasNotificationMessageDeserializer.java:210)
        at 
org.apache.atlas.kafka.AtlasKafkaConsumer.receive(AtlasKafkaConsumer.java:72)
        at 
org.apache.ranger.tagsync.source.atlas.AtlasTagSource$ConsumerRunnable.run(AtlasTagSource.java:165)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.json4s.package$MappingException: No usable value for values
Expected object but got JNull
        at org.json4s.reflect.package$.fail(package.scala:96)
        at 
org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:462)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$14.apply(Extraction.scala:482)
        at 
org.json4s.Extraction$ClassInstanceBuilder$$anonfun$14.apply(Extraction.scala:482)
        at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)

        ….

Can you/someone please tell me how can I fix this?

Thanks & Regards,

Rajat

From: Abhay Kulkarni 
<akulka...@hortonworks.com<mailto:akulka...@hortonworks.com>>
Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" 
<user@ranger.apache.org<mailto:user@ranger.apache.org>>
Date: Wednesday, 23 May 2018 at 10:05 PM
To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" 
<user@ranger.apache.org<mailto:user@ranger.apache.org>>
Subject: Re: Custom service integration and Tag based policies

Hi Rajat,

Here are the steps required to enable Tagsync to map custom Atlas Entities to 
Ranger Entities (aka Service Resources).

  *   Provide an implementation of AtlasResourceMapper Java class. You may look 
up other implementation classes such as AtlasHdfsResourceMapper or 
AtlasHiveResourceMapper which map hfds and hive Atlas entities to Ranger 
Service Resources respectively. Specifically, you need to implement 
buildResource() member function where your component’s RangerAtlasEntity object 
needs to be converted to generic RangerServiceResource object.
  *   Set up the following Tagsync configuration parameter in 
ranger-tagsync-site.xml file. If using Ambari, you may do this by going to 
'Ranger->Configs->Advanced->Custom ranger-tagsync-site->Add Property’.

ranger.tagsync.atlas.custom.resource.mappers=<fully-qualified-name-of-the-implementation-class>

  *   Ensure that the implementation class is in the class-path of tagsync. One 
way to do this is to place the JAR containing the implementation class in 
/usr/hdp/current/ranger-tagsync/lib directory.
  *   Restart tagsync.
  *   Check /var/log/ranger/tagsync/tagsync.out and 
/var/log/ranger/tagsync/tagsync.log files to ensure that there are no errors in 
initializing AtlasTagSource.

Hope this helps.

Thanks,
-Abhay




From: Rajat Goel <rajat.g...@guavus.com<mailto:rajat.g...@guavus.com>>
Reply-To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" 
<user@ranger.apache.org<mailto:user@ranger.apache.org>>
Date: Wednesday, May 23, 2018 at 7:02 AM
To: "user@ranger.apache.org<mailto:user@ranger.apache.org>" 
<user@ranger.apache.org<mailto:user@ranger.apache.org>>
Subject: Custom service integration and Tag based policies

Hi,

I am new to Ranger. I am trying to integrate my custom web service with Ranger 
for authorisation and policy management. I have successfully tested Resource 
based policy management integration of my web service with Ranger. Now I want 
to integrate and enable Tag based policy. I have added my custom types and 
entities in Atlas. How can I provide tag based policy and authorisation?

While reading some documentation and code, I found that I will have to add 
implementation of ‘AtlasResourceMapper.java’ in Ranger Tag sync process on the 
lines of AtlasKafkaResourceMapper.java. Is my understanding correct? Can 
someone point me to any documentation on how to write a custom class which 
extends AtlasResourceMapper? Also, are there more steps/code that needs to be 
written or just the AtlasResourceMapper implementation will suffice?

Thanks & Regards,
Rajat

Reply via email to