Hi Rushmin/Sinthuja, I'll update the blog accordingly. Thanks for Pointing the issue and Solution.
Thanks and Regards. On Wed, Nov 4, 2015 at 9:09 PM, Rushmin Fernando <rush...@wso2.com> wrote: > Yes that explains ! Thanks Sinthuja :-) > > On Wed, Nov 4, 2015 at 3:36 PM, Sinthuja Ragendran <sinth...@wso2.com> > wrote: > >> Hi Rushmin, >> >> >> On Wed, Nov 4, 2015 at 8:53 PM, Rushmin Fernando <rush...@wso2.com> >> wrote: >> >>> Hi Sinthuja, >>> >>> Yes the relaxAutoCommit parameter was not there in the beginning and the >>> table drop was probably due to that. >>> >>> Having the relaxAutoCommit in place and recreating the tables solved the >>> issue :-) >>> >> >> Great! >> >> >>> >>> Just wanted to know whether the auto-commit related error caused the >>> table drop. >>> >> >> No, it didn't cause the table to drop, the default behaviour it self will >> drop the table and create it again; but due to the exception, the creation >> part didn't happen and only dropping the table was executed. >> >> Thanks, >> Sinthuja. >> >>> >>> Thank you for your clarification. >>> >>> Thanks >>> Rushmin >>> >>> On Wed, Nov 4, 2015 at 2:20 PM, Sinthuja Ragendran <sinth...@wso2.com> >>> wrote: >>> >>>> Hi Rushmin, >>>> >>>> Did you create the table after encountering the exception, and adding >>>> the relaxAutoCommit parameter as well? Because during the exception phase >>>> it could have got dropped, and now it's again searching for the non- >>>> existing table. Hence I wanted to make sure, you ran the mysql script now, >>>> and still seeing this exception. >>>> >>>> Thanks, >>>> Sinthuja. >>>> >>>> On Wed, Nov 4, 2015 at 7:46 PM, Rushmin Fernando <rush...@wso2.com> >>>> wrote: >>>> >>>>> Hi Sinthuja, >>>>> >>>>> Thanks for the quick response. >>>>> >>>>> I did create the tables as per the instructions :-) >>>>> >>>>> But 'API_REQUEST_SUMMARY' table was gone after the auto-commit error, >>>>> causing the subsequent error messages. >>>>> >>>>> I could reproduce it consistently. >>>>> >>>>> Thanks >>>>> Rushhmin >>>>> >>>>> >>>>> On Wed, Nov 4, 2015 at 2:09 PM, Sinthuja Ragendran <sinth...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Rushmin, >>>>>> >>>>>> Spark JDBC connector first drops the existing table and recreates, >>>>>> but anyhow to execute for the first time you need to have the table >>>>>> created. Therefore please create the tables in the database by running >>>>>> mysql script [1] which was given in the blog post. >>>>>> >>>>>> [1] >>>>>> https://github.com/ruks/WSO2-APIM_DAS_Analytics_CApp/tree/v1.0.2/dbscripts >>>>>> >>>>>> Thanks, >>>>>> Sinthuja. >>>>>> >>>>>> On Wed, Nov 4, 2015 at 7:24 PM, Rushmin Fernando <rush...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Devs, >>>>>>> >>>>>>> I was trying to publish APIM stats to DAS following this blog [1] >>>>>>> >>>>>>> And in DAS console I got the following error. >>>>>>> >>>>>>> [2015-11-04 12:10:08,163] ERROR >>>>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in >>>>>>> executing task: Can't call commit when autocommit=true >>>>>>> java.lang.RuntimeException: Can't call commit when autocommit=true >>>>>>> at >>>>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.insert(JDBCRelation.scala:193) >>>>>>> at >>>>>>> org.apache.spark.sql.sources.InsertIntoDataSource.run(commands.scala:53) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>>> at >>>>>>> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >>>>>>> at >>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950) >>>>>>> at >>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950) >>>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144) >>>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128) >>>>>>> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >>>>>>> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) >>>>>>> >>>>>>> >>>>>>> Immediately after that error the below error was there. >>>>>>> >>>>>>> [2015-11-04 12:15:00,027] ERROR >>>>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in >>>>>>> executing task: Error while connecting to datasource WSO2AM_STATS_DB : >>>>>>> Table 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' doesn't exist >>>>>>> java.lang.RuntimeException: Error while connecting to datasource >>>>>>> WSO2AM_STATS_DB : Table 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' doesn't >>>>>>> exist >>>>>>> at >>>>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.liftedTree1$1(JDBCRelation.scala:143) >>>>>>> at >>>>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.<init>(JDBCRelation.scala:137) >>>>>>> at >>>>>>> org.apache.spark.sql.jdbc.carbon.AnalyticsJDBCRelationProvider.createRelation(JDBCRelation.scala:119) >>>>>>> at >>>>>>> org.apache.spark.sql.sources.ResolvedDataSource$.apply(ddl.scala:269) >>>>>>> at >>>>>>> org.apache.spark.sql.sources.CreateTempTableUsing.run(ddl.scala:412) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>>> at >>>>>>> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >>>>>>> at >>>>>>> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >>>>>>> at >>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950) >>>>>>> at >>>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950) >>>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144) >>>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128) >>>>>>> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >>>>>>> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) >>>>>>> >>>>>>> >>>>>>> As the error message says the >>>>>>> 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' table was gone ! >>>>>>> >>>>>>> *I added the relaxAutoCommit=true property to the JDBC connection >>>>>>> string to solve the first error message and stat feature worked.* >>>>>>> >>>>>>> Is the table deletion, a result of the first error (trying to commit >>>>>>> when auto commit is enabled) >>>>>>> >>>>>>> Thanks >>>>>>> Rushmin >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> [1] - >>>>>>> http://blog.rukspot.com/2015/09/publishing-apim-runtime-statistics-to.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Rushmin Fernando* >>>>>>> *Technical Lead* >>>>>>> >>>>>>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>>>>>> >>>>>>> email : rush...@wso2.com >>>>>>> mobile : +94772310855 >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> Dev@wso2.org >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Sinthuja Rajendran* >>>>>> Associate Technical Lead >>>>>> WSO2, Inc.:http://wso2.com >>>>>> >>>>>> Blog: http://sinthu-rajan.blogspot.com/ >>>>>> Mobile: +94774273955 >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Rushmin Fernando* >>>>> *Technical Lead* >>>>> >>>>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>>>> >>>>> email : rush...@wso2.com >>>>> mobile : +94772310855 >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Sinthuja Rajendran* >>>> Associate Technical Lead >>>> WSO2, Inc.:http://wso2.com >>>> >>>> Blog: http://sinthu-rajan.blogspot.com/ >>>> Mobile: +94774273955 >>>> >>>> >>>> >>> >>> >>> -- >>> *Rushmin Fernando* >>> *Technical Lead* >>> >>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>> >>> email : rush...@wso2.com >>> mobile : +94772310855 >>> >>> >>> >> >> >> -- >> *Sinthuja Rajendran* >> Associate Technical Lead >> WSO2, Inc.:http://wso2.com >> >> Blog: http://sinthu-rajan.blogspot.com/ >> Mobile: +94774273955 >> >> >> > > > -- > *Rushmin Fernando* > *Technical Lead* > > WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware > > email : rush...@wso2.com > mobile : +94772310855 > > > -- Rukshan Chathuranga. Software Engineer. WSO2, Inc.
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
