Hi Revanth,
I took a quick look and don't think you can override existing mappings. You 
should see a warning like this -
18/04/15 19:50:27 WARN CodecRegistry: Ignoring codec CustomTSTypeCodec [date 
<-> java.lang.Long] because it collides with previously registered codec 
CustomTSTypeCodec [date <-> java.lang.Long]
The issue is that CodecRegistry does not allow you to override existing type 
mappings[1][2]. You can however register a codec that converts between a 
CqlType and your custom class.


    On Sunday, April 15, 2018, 1:06:53 AM PDT, Revanth Reddy 
<revanthkuma...@gmail.com> wrote:  
 Hi Dinesh,
Thanks a lot for your response . I had tried with the existing codec 
(SimpleTimestampCodec) ,but it says the codec has been registered but it is not 
converting the timestamp column to long . also i don't observe any error.
Below are the steps that i had followed and also attached the code that i had 
I had used this property to set the codec in spark 

Since there is no direct way to refer the custom codec in spark , i had 
followed the above approach.
I had used the following dependencies :
              <dependency> <groupId>com.datastax.spark</groupId> 
<version>2.0.1</version> </dependency> <dependency> 
<artifactId>cassandra-driver-extras</artifactId> <version>3.1.4</version> 
</dependency> <dependency> <groupId>joda-time</groupId> 
<artifactId>joda-time</artifactId> <version>2.9.1</version> </dependency>
Cassandra table used for testing :

CREATE TABLE dev.my_table (

    partition_key text PRIMARY KEY,

    some_timestamp timestamp


cqlsh:dev> select * from my_table;

 partition_key | some_timestamp


           key | 2015-12-25 18:30:00+0000

           foo | 2018-04-11 06:22:29+0000

Thanks & RegardsS.Revanth kumar reddyhttps://github.com/Re1tReddy/



On Sun, Apr 15, 2018 at 11:36 AM, Dinesh Joshi <dinesh.jo...@yahoo.com> wrote:

Hi Revanth,
How do you register the custom codec? Do you get any errors? Have you tried 
using a pre-existing codec? It would be helpful if you can give more 
Dinesh    On Saturday, April 14, 2018, 7:29:30 PM PDT, Revanth Reddy 
<revanthkuma...@gmail.com> wrote:  

Hi Team ,
I want to write a custom cassandra codec and i want to use that codec in my 
spark application while reading the data from cassandra table .
Basically the custom codecs are used to convert one column type to another 
while reading from cassandra. for example i have a timestamp column in 
cassandra table ,but i want to read it as long and save as parquet file to HDFS 
, then this cutom codecs will be usefull instead of doing an additional 
transformation in the spark code.
I want to know how to set/register this custom cassandra codecs in spark ?
 I had tried with solution provided in the below link but it does not work , 
any ideas around this will be helpfull.
https://stackoverflow.com/ questions/40363611/adding- custom-codec-to- 
cassandraconnector/49750791? noredirect=1#comment86515847_ 49750791

Thanks & RegardsS.Revanth kumar reddyhttps://github.com/Re1tReddy/ crazyusers/login




Reply via email to