[
https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059468#comment-15059468
]
Hadoop QA commented on HBASE-14849:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777865/HBASE-14849.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777865
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ sqlContext.sparkContext.getConf.getInt(HBaseSparkConf.CACHE_SIZE,
HBaseSparkConf.defaultCachingSize))
+
scan.setCaching(sqlContext.sparkContext.getConf.getInt(HBaseSparkConf.BATCH_NUM,
HBaseSparkConf.defaultBatchNum))
+ sparkConf.getBoolean(HBaseSparkConf.BLOCK_CACHE_ENABLE,
HBaseSparkConf.defaultBlockCacheEnable))
+ Map("cacheSize" -> "100", "batchNum" -> "100", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "1000", "batchNum" -> "100", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "100", "batchNum" -> "1000", "blockCacheingEnable" ->
"true", "rowNum" -> "10"))
+ Map("cacheSize" -> "100", "batchNum" -> "100", "blockCacheingEnable" ->
"false", "rowNum" -> "10"))
+case class DummyScan(cacheSize: Int, batchNum: Int, blockCachingEnable:
Boolean, rowNum: Int)(@transient val sqlContext: SQLContext)
+ override def buildScan(): RDD[Row] = sqlContext.sparkContext.parallelize(0
until rowNum).map(Row(_)).map{
+ sparkConf.getInt(HBaseSparkConf.CACHE_SIZE,
HBaseSparkConf.defaultCachingSize) != cacheSize ||
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//testReport/
Release Findbugs (version 2.0.3) warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16870//console
This message is automatically generated.
> Add option to set block cache to false on SparkSQL executions
> -------------------------------------------------------------
>
> Key: HBASE-14849
> URL: https://issues.apache.org/jira/browse/HBASE-14849
> Project: HBase
> Issue Type: New Feature
> Components: spark
> Reporter: Ted Malaska
> Assignee: Zhan Zhang
> Attachments: HBASE-14849.patch
>
>
> I was working at a client with a ported down version of the Spark module for
> HBase and realized we didn't add an option to turn of block cache for the
> scans.
> At the client I just disabled all caching with Spark SQL, this is an easy but
> very impactful fix.
> The fix for this patch will make this configurable
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
