Yes that explains ! Thanks Sinthuja :-) On Wed, Nov 4, 2015 at 3:36 PM, Sinthuja Ragendran <[email protected]> wrote:
> Hi Rushmin, > > > On Wed, Nov 4, 2015 at 8:53 PM, Rushmin Fernando <[email protected]> wrote: > >> Hi Sinthuja, >> >> Yes the relaxAutoCommit parameter was not there in the beginning and the >> table drop was probably due to that. >> >> Having the relaxAutoCommit in place and recreating the tables solved the >> issue :-) >> > > Great! > > >> >> Just wanted to know whether the auto-commit related error caused the >> table drop. >> > > No, it didn't cause the table to drop, the default behaviour it self will > drop the table and create it again; but due to the exception, the creation > part didn't happen and only dropping the table was executed. > > Thanks, > Sinthuja. > >> >> Thank you for your clarification. >> >> Thanks >> Rushmin >> >> On Wed, Nov 4, 2015 at 2:20 PM, Sinthuja Ragendran <[email protected]> >> wrote: >> >>> Hi Rushmin, >>> >>> Did you create the table after encountering the exception, and adding >>> the relaxAutoCommit parameter as well? Because during the exception phase >>> it could have got dropped, and now it's again searching for the non- >>> existing table. Hence I wanted to make sure, you ran the mysql script now, >>> and still seeing this exception. >>> >>> Thanks, >>> Sinthuja. >>> >>> On Wed, Nov 4, 2015 at 7:46 PM, Rushmin Fernando <[email protected]> >>> wrote: >>> >>>> Hi Sinthuja, >>>> >>>> Thanks for the quick response. >>>> >>>> I did create the tables as per the instructions :-) >>>> >>>> But 'API_REQUEST_SUMMARY' table was gone after the auto-commit error, >>>> causing the subsequent error messages. >>>> >>>> I could reproduce it consistently. >>>> >>>> Thanks >>>> Rushhmin >>>> >>>> >>>> On Wed, Nov 4, 2015 at 2:09 PM, Sinthuja Ragendran <[email protected]> >>>> wrote: >>>> >>>>> Hi Rushmin, >>>>> >>>>> Spark JDBC connector first drops the existing table and recreates, but >>>>> anyhow to execute for the first time you need to have the table created. >>>>> Therefore please create the tables in the database by running mysql script >>>>> [1] which was given in the blog post. >>>>> >>>>> [1] >>>>> https://github.com/ruks/WSO2-APIM_DAS_Analytics_CApp/tree/v1.0.2/dbscripts >>>>> >>>>> Thanks, >>>>> Sinthuja. >>>>> >>>>> On Wed, Nov 4, 2015 at 7:24 PM, Rushmin Fernando <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Devs, >>>>>> >>>>>> I was trying to publish APIM stats to DAS following this blog [1] >>>>>> >>>>>> And in DAS console I got the following error. >>>>>> >>>>>> [2015-11-04 12:10:08,163] ERROR >>>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in >>>>>> executing task: Can't call commit when autocommit=true >>>>>> java.lang.RuntimeException: Can't call commit when autocommit=true >>>>>> at >>>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.insert(JDBCRelation.scala:193) >>>>>> at >>>>>> org.apache.spark.sql.sources.InsertIntoDataSource.run(commands.scala:53) >>>>>> at >>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57) >>>>>> at >>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57) >>>>>> at >>>>>> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68) >>>>>> at >>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>> at >>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>> at >>>>>> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >>>>>> at >>>>>> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >>>>>> at >>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950) >>>>>> at >>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950) >>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144) >>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128) >>>>>> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >>>>>> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) >>>>>> >>>>>> >>>>>> Immediately after that error the below error was there. >>>>>> >>>>>> [2015-11-04 12:15:00,027] ERROR >>>>>> {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} - Error in >>>>>> executing task: Error while connecting to datasource WSO2AM_STATS_DB : >>>>>> Table 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' doesn't exist >>>>>> java.lang.RuntimeException: Error while connecting to datasource >>>>>> WSO2AM_STATS_DB : Table 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' doesn't >>>>>> exist >>>>>> at >>>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.liftedTree1$1(JDBCRelation.scala:143) >>>>>> at >>>>>> org.apache.spark.sql.jdbc.carbon.JDBCRelation.<init>(JDBCRelation.scala:137) >>>>>> at >>>>>> org.apache.spark.sql.jdbc.carbon.AnalyticsJDBCRelationProvider.createRelation(JDBCRelation.scala:119) >>>>>> at >>>>>> org.apache.spark.sql.sources.ResolvedDataSource$.apply(ddl.scala:269) >>>>>> at >>>>>> org.apache.spark.sql.sources.CreateTempTableUsing.run(ddl.scala:412) >>>>>> at >>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:57) >>>>>> at >>>>>> org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:57) >>>>>> at >>>>>> org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:68) >>>>>> at >>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>> at >>>>>> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:88) >>>>>> at >>>>>> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) >>>>>> at >>>>>> org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:87) >>>>>> at >>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:950) >>>>>> at >>>>>> org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:950) >>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:144) >>>>>> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:128) >>>>>> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >>>>>> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:755) >>>>>> >>>>>> >>>>>> As the error message says the >>>>>> 'TP_WSO2AM_STATS_DB.API_REQUEST_SUMMARY' table was gone ! >>>>>> >>>>>> *I added the relaxAutoCommit=true property to the JDBC connection >>>>>> string to solve the first error message and stat feature worked.* >>>>>> >>>>>> Is the table deletion, a result of the first error (trying to commit >>>>>> when auto commit is enabled) >>>>>> >>>>>> Thanks >>>>>> Rushmin >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> [1] - >>>>>> http://blog.rukspot.com/2015/09/publishing-apim-runtime-statistics-to.html >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Rushmin Fernando* >>>>>> *Technical Lead* >>>>>> >>>>>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>>>>> >>>>>> email : [email protected] >>>>>> mobile : +94772310855 >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Sinthuja Rajendran* >>>>> Associate Technical Lead >>>>> WSO2, Inc.:http://wso2.com >>>>> >>>>> Blog: http://sinthu-rajan.blogspot.com/ >>>>> Mobile: +94774273955 >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Rushmin Fernando* >>>> *Technical Lead* >>>> >>>> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >>>> >>>> email : [email protected] >>>> mobile : +94772310855 >>>> >>>> >>>> >>> >>> >>> -- >>> *Sinthuja Rajendran* >>> Associate Technical Lead >>> WSO2, Inc.:http://wso2.com >>> >>> Blog: http://sinthu-rajan.blogspot.com/ >>> Mobile: +94774273955 >>> >>> >>> >> >> >> -- >> *Rushmin Fernando* >> *Technical Lead* >> >> WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware >> >> email : [email protected] >> mobile : +94772310855 >> >> >> > > > -- > *Sinthuja Rajendran* > Associate Technical Lead > WSO2, Inc.:http://wso2.com > > Blog: http://sinthu-rajan.blogspot.com/ > Mobile: +94774273955 > > > -- *Rushmin Fernando* *Technical Lead* WSO2 Inc. <http://wso2.com/> - Lean . Enterprise . Middleware email : [email protected] mobile : +94772310855
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
