[jira] [Commented] (HIVE-15802) Changes to expected entries for dynamic bloomfilter runtime filtering

2017-02-09 Thread Gunther Hagleitner (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860062#comment-15860062
 ] 

Gunther Hagleitner commented on HIVE-15802:
---

+1 LGTM

> Changes to expected entries for dynamic bloomfilter runtime filtering
> -
>
> Key: HIVE-15802
> URL: https://issues.apache.org/jira/browse/HIVE-15802
> Project: Hive
>  Issue Type: Improvement
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15802.1.patch, HIVE-15802.2.patch, 
> HIVE-15802.3.patch
>
>
> - Estimate bloom filter size based on distinct values from column stats if 
> available



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15802) Changes to expected entries for dynamic bloomfilter runtime filtering

2017-02-09 Thread Jason Dere (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15860012#comment-15860012
 ] 

Jason Dere commented on HIVE-15802:
---

Test results look good.
[~gopalv] [~hagleitn] can you review?

> Changes to expected entries for dynamic bloomfilter runtime filtering
> -
>
> Key: HIVE-15802
> URL: https://issues.apache.org/jira/browse/HIVE-15802
> Project: Hive
>  Issue Type: Improvement
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15802.1.patch, HIVE-15802.2.patch, 
> HIVE-15802.3.patch
>
>
> - Estimate bloom filter size based on distinct values from column stats if 
> available



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15802) Changes to expected entries for dynamic bloomfilter runtime filtering

2017-02-08 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859154#comment-15859154
 ] 

Hive QA commented on HIVE-15802:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12851757/HIVE-15802.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10207 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=161)

[infer_bucket_sort_num_buckets.q,gen_udf_example_add10.q,insert_overwrite_directory2.q,orc_merge5.q,bucketmapjoin6.q,import_exported_table.q,vector_outer_join0.q,orc_merge4.q,temp_table_external.q,orc_merge_incompat1.q,root_dir_external_table.q,constprog_semijoin.q,auto_sortmerge_join_16.q,schemeAuthority.q,index_bitmap3.q,external_table_with_space_in_location_path.q,parallel_orderby.q,infer_bucket_sort_map_operators.q,bucketizedhiveinputformat.q,remote_script.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=123)

[groupby3_map.q,union26.q,mapreduce1.q,mapjoin_addjar.q,bucket_map_join_spark1.q,udf_example_add.q,multi_insert_with_join.q,sample7.q,auto_join_nulls.q,ppd_outer_join4.q,load_dyn_part8.q,alter_merge_orc.q,sample6.q,bucket_map_join_1.q,auto_sortmerge_join_9.q]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3455/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3455/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3455/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12851757 - PreCommit-HIVE-Build

> Changes to expected entries for dynamic bloomfilter runtime filtering
> -
>
> Key: HIVE-15802
> URL: https://issues.apache.org/jira/browse/HIVE-15802
> Project: Hive
>  Issue Type: Improvement
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15802.1.patch, HIVE-15802.2.patch, 
> HIVE-15802.3.patch
>
>
> - Estimate bloom filter size based on distinct values from column stats if 
> available
> - Cap the bloom filter expected entries size to 
> hive.tez.max.bloom.filter.entries if the estimated size from stats exceeds 
> that amount.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15802) Changes to expected entries for dynamic bloomfilter runtime filtering

2017-02-07 Thread Jason Dere (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857153#comment-15857153
 ] 

Jason Dere commented on HIVE-15802:
---

Looks like the golden file for 
TestMiniLlapLocalCliDriver.testCliDriver[mergejoin] needs to be updated.

> Changes to expected entries for dynamic bloomfilter runtime filtering
> -
>
> Key: HIVE-15802
> URL: https://issues.apache.org/jira/browse/HIVE-15802
> Project: Hive
>  Issue Type: Improvement
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15802.1.patch, HIVE-15802.2.patch
>
>
> - Estimate bloom filter size based on distinct values from column stats if 
> available
> - Cap the bloom filter expected entries size to 
> hive.tez.max.bloom.filter.entries if the estimated size from stats exceeds 
> that amount.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15802) Changes to expected entries for dynamic bloomfilter runtime filtering

2017-02-07 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857074#comment-15857074
 ] 

Hive QA commented on HIVE-15802:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12851479/HIVE-15802.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10206 tests 
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) 
(batchId=235)
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=162)

[scriptfile1.q,vector_outer_join5.q,file_with_header_footer.q,bucket4.q,input16_cc.q,bucket5.q,infer_bucket_sort_merge.q,constprog_partitioner.q,orc_merge2.q,reduce_deduplicate.q,schemeAuthority2.q,load_fs2.q,orc_merge8.q,orc_merge_incompat2.q,infer_bucket_sort_bucketed_table.q,vector_outer_join4.q,disable_merge_for_bucketing.q,vector_inner_join.q,orc_merge7.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=106)

[bucketsortoptimize_insert_4.q,multi_insert_mixed.q,vectorization_10.q,auto_join18_multi_distinct.q,join_cond_pushdown_3.q,custom_input_output_format.q,skewjoinopt5.q,vectorization_part_project.q,vector_count_distinct.q,skewjoinopt4.q,count.q,parallel.q,union33.q,union_lateralview.q,nullgroup4.q]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
 (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin] 
(batchId=150)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] 
(batchId=223)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3422/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3422/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3422/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12851479 - PreCommit-HIVE-Build

> Changes to expected entries for dynamic bloomfilter runtime filtering
> -
>
> Key: HIVE-15802
> URL: https://issues.apache.org/jira/browse/HIVE-15802
> Project: Hive
>  Issue Type: Improvement
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-15802.1.patch, HIVE-15802.2.patch
>
>
> - Estimate bloom filter size based on distinct values from column stats if 
> available
> - Cap the bloom filter expected entries size to 
> hive.tez.max.bloom.filter.entries if the estimated size from stats exceeds 
> that amount.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)