@Rukshan, you may have to update the blog accordingly :-) (saying that auto-commit should be disabled)
On Wed, Nov 4, 2015 at 3:23 PM, Rushmin Fernando <rush...@wso2.com> wrote: > Hi Sinthuja, > > Yes the relaxAutoCommit parameter was not there in the beginning and the > table drop was probably due to that. > > Having the relaxAutoCommit in place and recreating the tables solved the > issue :-) > > Just wanted to know whether the auto-commit related error caused the table > drop. > > Thank you for your clarification. > > Thanks > Rushmin > > On Wed, Nov 4, 2015 at 2:20 PM, Sinthuja Ragendran <sinth...@wso2.com> > wrote: > >> Hi Rushmin, >> >> Did you create the table after encountering the exception, and adding >> the relaxAutoCommit parameter as well? Because during the exception phase >> it could have got dropped, and now it's again searching for the non- >> existing table. Hence I wanted to make sure, you ran the mysql script now, >> and still seeing this exception. >> >> Thanks, >> Sinthuja. >> >> On Wed, Nov 4, 2015 at 7:46 PM, Rushmin Fernando <rush...@wso2.com> >> wrote: >> >>> Hi Sinthuja, >>> >>> Thanks for the quick response. >>> >>> I did create the tables as per the instructions :-) >>> >>> But 'API_REQUEST_SUMMARY' table was gone after the auto-commit error, >>> causing the subsequent error messages. >>> >>> I could reproduce it consistently. >>> >>> Thanks >>> Rushhmin >>> >>> >>> On Wed, Nov 4, 2015 at 2:09 PM, Sinthuja Ragendran <sinth...@wso2.com> >>> wrote: >>> >>>> Hi Rushmin, >>>> >>>> Spark JDBC connector first drops the existing table and recreates, but >>>> anyhow to execute for the first time you need to have the table created. >>>> Therefore please create the tables in the database by running mysql script >>>> [1] which was given in the blog post. >>>> >>>> [1] >>>> https://github.com/ruks/WSO2-APIM_DAS_Analytics_CApp/tree/v1.0.2/dbscripts >>>> >>>> Thanks, >>>> Sinthuja. >>>> >>>> On Wed, Nov 4, 2015 at 7:24 PM, Rushmin Fernando <rush...@wso2.com> >>>> wrote: >>>> >>>>> Hi Devs, >>>>> >>>>> I was trying to publish APIM stats to DAS following this blog [1] >>>>> >>>>> And in DAS console I got the following error. >>>>> >>>>> [2015-11-04 12:10:08,163] ERROR >>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in >>>>> executing task: Can't call commit when autocommit=true >>>>> java.lang.RuntimeException: Can't call commit when autocommit=true >>>>> at >>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.insert(JDBCRelation.scala:193) >>>>> at >>>>> org.apache.spark.sql.sources.InsertIntoDataSource.run(commands.scala:53) >>>>> at >>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57) >>>>> at >>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57) >>>>> at >>>>> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68) >>>>> at >>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>> at >>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>> at >>>>> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >>>>> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >>>>> at >>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950) >>>>> at >>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950) >>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144) >>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128) >>>>> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >>>>> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) >>>>> >>>>> >>>>> Immediately after that error the below error was there. >>>>> >>>>> [2015-11-04 12:15:00,027] ERROR >>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in >>>>> executing task: Error while connecting to datasource WSO2AM_STATS_DB : >>>>> Table 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' doesn't exist >>>>> java.lang.RuntimeException: Error while connecting to datasource >>>>> WSO2AM_STATS_DB : Table 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' doesn't >>>>> exist >>>>> at >>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.liftedTree1$1(JDBCRelation.scala:143) >>>>> at >>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.<init>(JDBCRelation.scala:137) >>>>> at >>>>> org.apache.spark.sql.jdbc.carbon.AnalyticsJDBCRelationProvider.createRelation(JDBCRelation.scala:119) >>>>> at >>>>> org.apache.spark.sql.sources.ResolvedDataSource$.apply(ddl.scala:269) >>>>> at org.apache.spark.sql.sources.CreateTempTableUsing.run(ddl.scala:412) >>>>> at >>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57) >>>>> at >>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57) >>>>> at >>>>> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68) >>>>> at >>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>> at >>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>> at >>>>> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >>>>> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >>>>> at >>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950) >>>>> at >>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950) >>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144) >>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128) >>>>> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >>>>> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) >>>>> >>>>> >>>>> As the error message says the 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' >>>>> table was gone ! >>>>> >>>>> *I added the relaxAutoCommit=true property to the JDBC connection >>>>> string to solve the first error message and stat feature worked.* >>>>> >>>>> Is the table deletion, a result of the first error (trying to commit >>>>> when auto commit is enabled) >>>>> >>>>> Thanks >>>>> Rushmin >>>>> >>>>> >>>>> >>>>> >>>>> [1] - >>>>> http://blog.rukspot.com/2015/09/publishing-apim-runtime-statistics-to.html >>>>> >>>>> >>>>> >>>>> -- >>>>> *Rushmin Fernando* >>>>> *Technical Lead* >>>>> >>>>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>>>> >>>>> email : rush...@wso2.com >>>>> mobile : +94772310855 >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Sinthuja Rajendran* >>>> Associate Technical Lead >>>> WSO2, Inc.:http://wso2.com >>>> >>>> Blog: http://sinthu-rajan.blogspot.com/ >>>> Mobile: +94774273955 >>>> >>>> >>>> >>> >>> >>> -- >>> *Rushmin Fernando* >>> *Technical Lead* >>> >>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>> >>> email : rush...@wso2.com >>> mobile : +94772310855 >>> >>> >>> >> >> >> -- >> *Sinthuja Rajendran* >> Associate Technical Lead >> WSO2, Inc.:http://wso2.com >> >> Blog: http://sinthu-rajan.blogspot.com/ >> Mobile: +94774273955 >> >> >> > > > -- > *Rushmin Fernando* > *Technical Lead* > > WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware > > email : rush...@wso2.com > mobile : +94772310855 > > > -- *Rushmin Fernando* *Technical Lead* WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware email : rush...@wso2.com mobile : +94772310855
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
