Thanks for the clarification Puneet. Yes, it’s pretty clear now. Regards Pawan Malwal
From: Puneet Gupta [mailto:[email protected]] Sent: 16 September 2016 11:31 To: [email protected] Cc: Sharanabasappa G Keriwaddi <[email protected]> Subject: Re: NoCandidateDimAvailableException Hi Pawan When you submit a query, estimate is run on all the drivers and the one which returns yes with least cost is selected. The estimate operation is done in parallel and you can see the logs have estimate thread name (example [estimate-3] ). Each estimate thread caters to a different driver. Since the estimate fails on some of the drivers that don't have the required data, you are seeing the logs. For the correct driver ( jdbc/jdbc3 in your case) you should not see any exceptions. Hope this helps. Thanks, Puneet Gupta On Fri, Sep 16, 2016 at 11:05 AM, Pawan Malwal <[email protected]<mailto:[email protected]>> wrote: Rajat, I already have it in jdbcdriver-site.xml file as shown below and I am even able to get the results of the query run on dimension using the right driver created for mysql. But still I see “NoCandidateDimAvailableException” logged in lens logs for rest of the drivers. So these exceptions does not affect anything but I was wondering if there is any configuration issue due to which I am getting all these exceptions. <property> <name>lens.cube.query.driver.supported.storages</name> <value>mysql_storage</value> <final>true</final> </property> Regards Pawan Malwal From: Rajat Khandelwal [mailto:[email protected]<mailto:[email protected]>] Sent: 15 September 2016 20:55 To: [email protected]<mailto:[email protected]> Cc: Sharanabasappa G Keriwaddi <[email protected]<mailto:[email protected]>> Subject: Re: NoCandidateDimAvailableException You have to add "mysql" in the list of supported storages in the driver config file (jdbcdriver-site.xml). On Thu, Sep 15, 2016 at 8:05 PM Pawan Malwal <[email protected]<mailto:[email protected]>> wrote: Hi Lens Developers, Here is an observation on Lens behavior. Can you please clarify ? I have configured jdbc/jdbc3 driver for MySQL and created the required entities in Lens cli. I see some ERROR and “NoCandidateDimAvailableException” generated when I run a select query on dimension. Ideally lens should check all the drivers for expected entity and go for the driver which has required dimension registered to it. Not sure why does it log these ERRORS/ exceptions for other drivers hive/hive1 , jdbc/jdbc1 and jdbc/jdbc2? 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-3] ERROR org.apache.lens.cube.parse.CubeQueryContext - Query rewrite failed due to NO_CANDIDATE_DIM_AVAILABLE, Cause PruneCauses.BriefAndDetailedError(brief=No candidate storages for any table, details={employee_table=[CandidateTablePruneCause(cause=NO_CANDIDATE_STORAGES, storageCauses={mysql=CandidateTablePruneCause.SkipStorageCause(cause=UNSUPPORTED, updatePeriodRejectionCause=null, nonExistantPartCols=null)}, missingPartitions=null, missingUpdatePeriods=null, missingColumns=null, missingExpressions=null, joinColumns=null, columnsMissingDefaultAggregate=null, unsupportedTimeDims=null, maxTimeCovered=null, invalidRanges=null)]}) 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-2] ERROR org.apache.lens.cube.parse.CubeQueryContext - Query rewrite failed due to NO_CANDIDATE_DIM_AVAILABLE, Cause PruneCauses.BriefAndDetailedError(brief=No candidate storages for any table, details={employee_table=[CandidateTablePruneCause(cause=NO_CANDIDATE_STORAGES, storageCauses={mysql=CandidateTablePruneCause.SkipStorageCause(cause=UNSUPPORTED, updatePeriodRejectionCause=null, nonExistantPartCols=null)}, missingPartitions=null, missingUpdatePeriods=null, missingColumns=null, missingExpressions=null, joinColumns=null, columnsMissingDefaultAggregate=null, unsupportedTimeDims=null, maxTimeCovered=null, invalidRanges=null)]}) 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-4] ERROR org.apache.lens.cube.parse.CubeQueryContext - Query rewrite failed due to NO_CANDIDATE_DIM_AVAILABLE, Cause PruneCauses.BriefAndDetailedError(brief=No candidate storages for any table, details={employee_table=[CandidateTablePruneCause(cause=NO_CANDIDATE_STORAGES, storageCauses={mysql=CandidateTablePruneCause.SkipStorageCause(cause=UNSUPPORTED, updatePeriodRejectionCause=null, nonExistantPartCols=null)}, missingPartitions=null, missingUpdatePeriods=null, missingColumns=null, missingExpressions=null, joinColumns=null, columnsMissingDefaultAggregate=null, unsupportedTimeDims=null, maxTimeCovered=null, invalidRanges=null)]}) 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-3] WARN org.apache.lens.server.rewrite.RewriteUtil - Driver : jdbc/jdbc2 Skipped for the query rewriting due to org.apache.lens.cube.error.NoCandidateDimAvailableException: NO_CANDIDATE_DIM_AVAILABLE[No candidate storages for any table] at org.apache.lens.cube.parse.CubeQueryContext.pickCandidateDimsToQuery(CubeQueryContext.java:820) ~[lens-cube-2.5.0-beta.jar:2.5.0-beta] at org.apache.lens.cube.parse.CubeQueryContext.toHQL(CubeQueryContext.java:886) ~[lens-cube-2.5.0-beta.jar:2.5.0-beta] at org.apache.lens.server.rewrite.RewriteUtil$DriverRewriterRunnable.run(RewriteUtil.java:391) ~[classes/:na] at org.apache.lens.server.query.QueryExecutionServiceImpl$RewriteEstimateRunnable.run(QueryExecutionServiceImpl.java:1416) [classes/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_40] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40] 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-2] WARN org.apache.lens.server.rewrite.RewriteUtil - Driver : hive/hive1 Skipped for the query rewriting due to org.apache.lens.cube.error.NoCandidateDimAvailableException: NO_CANDIDATE_DIM_AVAILABLE[No candidate storages for any table] at org.apache.lens.cube.parse.CubeQueryContext.pickCandidateDimsToQuery(CubeQueryContext.java:820) ~[lens-cube-2.5.0-beta.jar:2.5.0-beta] at org.apache.lens.cube.parse.CubeQueryContext.toHQL(CubeQueryContext.java:886) ~[lens-cube-2.5.0-beta.jar:2.5.0-beta] at org.apache.lens.server.rewrite.RewriteUtil$DriverRewriterRunnable.run(RewriteUtil.java:391) ~[classes/:na] at org.apache.lens.server.query.QueryExecutionServiceImpl$RewriteEstimateRunnable.run(QueryExecutionServiceImpl.java:1416) [classes/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_40] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40] 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-4] WARN org.apache.lens.server.rewrite.RewriteUtil - Driver : jdbc/jdbc1 Skipped for the query rewriting due to org.apache.lens.cube.error.NoCandidateDimAvailableException: NO_CANDIDATE_DIM_AVAILABLE[No candidate storages for any table] at org.apache.lens.cube.parse.CubeQueryContext.pickCandidateDimsToQuery(CubeQueryContext.java:820) ~[lens-cube-2.5.0-beta.jar:2.5.0-beta] at org.apache.lens.cube.parse.CubeQueryContext.toHQL(CubeQueryContext.java:886) ~[lens-cube-2.5.0-beta.jar:2.5.0-beta] at org.apache.lens.server.rewrite.RewriteUtil$DriverRewriterRunnable.run(RewriteUtil.java:391) ~[classes/:na] at org.apache.lens.server.query.QueryExecutionServiceImpl$RewriteEstimateRunnable.run(QueryExecutionServiceImpl.java:1416) [classes/:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_40] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_40] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40] 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-3] ERROR org.apache.lens.server.query.QueryExecutionServiceImpl - Estimate skipped since rewrite failed for driver jdbc/jdbc2 cause: Driver :jdbc/jdbc2 Cause :NO_CANDIDATE_DIM_AVAILABLE[No candidate storages for any table] 15 Sep 2016 14:20:28 [96204879-9158-4ef4-82d7-3c7b47474ee9] [estimate-4] ERROR org.apache.lens.server.query.QueryExecutionServiceImpl - Estimate skipped since rewrite failed for driver jdbc/jdbc1 cause: Driver :jdbc/jdbc1 Cause :NO_CANDIDATE_DIM_AVAILABLE[No candidate storages for any table] Regards Pawan Malwal _____________________________________________________________ The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
