Hi Chamila, Execute this query before you execute above one "Drop table Chamila" . Basically hive maintain meta information when it creates a new virtual table. Once you create a table it won't change again unless you drop that virtual table and create a new one with edited parameters.
Regards, Gihan On Mon, Dec 23, 2013 at 9:51 AM, Chamila Wijayarathna <[email protected]>wrote: > Hii all, > How can I create a column family, so that it will suite my need? I tried > > CREATE EXTERNAL TABLE IF NOT EXISTS Chamila ( > remote_ip STRING, > request_date STRING,request STRING, > httpCode STRING,length STRING, > version STRING) > STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' > WITH SERDEPROPERTIES ( > "wso2.carbon.datasource.name" = "WSO2BAM_CASSANDRA_DATASOURCE" , > "cassandra.cf.name" = "chamila" , > "cassandra.columns.mapping" = > *"*remote_ip, request_date, request,httpCode,length, version" ); > > So now I have all columns in column family has mapped to my hive table, > but still I am getting same error[1]. > > Do I have to configure column family somewhere else before running this > code? If yes, how can I do that? > > 1. Backend Console Log - https://gist.github.com/cdwijayarathna/8091591 > > Thank You! > > > > > > > On Sun, Dec 22, 2013 at 1:00 PM, Maninda Edirisooriya <[email protected]>wrote: > >> Hi Chamila, >> >> And also note that the error displayed on the UI when a Hive script is >> executed is not enough to figure out the problem. You need to look at the >> backend console log as well. >> Anyway for the given problem Inosh's correction will work. >> >> >> *Maninda Edirisooriya* >> Software Engineer >> >> *WSO2, Inc. *lean.enterprise.middleware. >> >> *Blog* : http://maninda.blogspot.com/ >> *Phone* : +94 777603226 >> >> >> On Sat, Dec 21, 2013 at 11:27 PM, Inosh Goonewardena <[email protected]>wrote: >> >>> On Sat, Dec 21, 2013 at 10:25 PM, Chamila Wijayarathna <[email protected] >>> > wrote: >>> >>>> Hii Maninda, >>>> Thanks for replying. >>>> I tried creating a table which will suite my need, with different set >>>> of columns from above example. >>>> >>>> CREATE EXTERNAL TABLE IF NOT EXISTS Chamila ( >>>> remote_ip STRING, >>>> request_date STRING,request STRING, >>>> httpCode STRING,length STRING, >>>> version STRING) >>>> STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' >>>> WITH SERDEPROPERTIES ( >>>> "wso2.carbon.datasource.name" = "WSO2BAM_CASSANDRA_DATASOURCE" , >>>> "cassandra.cf.name" = "processed_logs" , >>>> "cassandra.columns.mapping" = >>>> *":key*,remote_ip, request_date, request,httpCode,length, version" ); >>>> >>>> But I get an error saying, "Error while executing Hive script.Query >>>> returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 >>>> from org.apache.hadoop.hive.ql.exec.DDLTask". >>>> >>> >>> Seems like ':key' column has not been mapped to any column of hive your >>> table. For example, in original query it is mapped to logId. >>> >>> How can I create a table to suite what I want. As I understood, it has >>>> no meaning to create a virtual table without linking it with Column Family. >>>> >>>> Thank you! >>>> >>>> >>>> >>>> On Fri, Dec 20, 2013 at 5:58 PM, Maninda Edirisooriya <[email protected] >>>> > wrote: >>>> >>>>> On Fri, Dec 20, 2013 at 4:54 PM, Chamila Wijayarathna < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello All, >>>>>> I am running BAM sample 1. >>>>>> In the analytic script which get run when I sent log line event it >>>>>> has query >>>>>> >>>>>> CREATE EXTERNAL TABLE IF NOT EXISTS ProcessedLogs ( >>>>>> logId STRING,remote_ip STRING, >>>>>> request_date STRING,request STRING, >>>>>> httpCode STRING,length STRING, >>>>>> version STRING) >>>>>> STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' >>>>>> WITH SERDEPROPERTIES ( >>>>>> "wso2.carbon.datasource.name" = "WSO2BAM_CASSANDRA_DATASOURCE" , >>>>>> "cassandra.cf.name" = "processed_logs" , >>>>>> "cassandra.columns.mapping" = >>>>>> ":key,remote_ip, request_date, request,httpCode,length, version" ); >>>>>> >>>>>> What I understand about this query is that it creates a table and >>>>>> load data from "processed_logs" column family which is available at >>>>>> Cassandra Keyspaces / EVENT_KS. >>>>>> >>>>> In this query it created the "processed_logs" column family and >>>>> creates the Hive table (which is a virtual table), "ProcessedLogs" >>>>> wrapping >>>>> the column family, "processed_logs". >>>>> >>>>>> >>>>>> I need to analyze a log file which is in a format different from >>>>>> access.log given in sample. How can I create a different column family >>>>>> for >>>>>> my requirement and access them in my script??? >>>>>> >>>>> The script above can be used for creating a column family. >>>>> >>>>> Read [1] for more information on writing Hive queries. >>>>> >>>>> [1] >>>>> http://docs.wso2.org/display/BAM240/Creating+Hive+Queries+to+Analyze+Data >>>>> >>>>>> >>>>>> >>>>>> Thank You! >>>>>> -- >>>>>> *Chamila Wijayarathna* >>>>>> Engineering Intern, >>>>>> WSO2 Inc. >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> *Chamila Wijayarathna* >>>> Engineering Intern, >>>> WSO2 Inc. >>>> >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Regards, >>> >>> Inosh Goonewardena >>> Associate Technical Lead- WSO2 Inc. >>> Mobile: +94779966317 >>> >> >> > > > -- > *Chamila Wijayarathna* > Engineering Intern, > WSO2 Inc. > > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- W.G. Gihan Anuruddha Senior Software Engineer | WSO2, Inc. M: +94772272595
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
