[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512361#comment-16512361 ] Zoltan Haindrich commented on HIVE-18448: - thank you [~jnh5y]! ...I always kept forgetting it... > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Fix For: 3.0.0 > > Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch, > HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498288#comment-16498288 ] Jim Hughes commented on HIVE-18448: --- Relative to this ticket... can someone update the [https://cwiki.apache.org/confluence/display/Hive/IndexDev] wiki page to indicate that the IndexHandler interface has been removed? > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch, > HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451409#comment-16451409 ] Vineet Garg commented on HIVE-18448: [~kgyrtkirk] Never mind it was already turned off your patch didn't turn it off. I looked at wrong diff :) > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch, > HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451376#comment-16451376
]
Vineet Garg commented on HIVE-18448:
[~kgyrtkirk] this patch turned {{hive.optimize.index.filter}} to false. This
has disabled ORC ppd by default. We should turn if on again as it might cause
performance regression
> Drop Support For Indexes From Apache Hive
> -
>
> Key: HIVE-18448
> URL: https://issues.apache.org/jira/browse/HIVE-18448
> Project: Hive
> Issue Type: Improvement
> Components: Indexing
>Reporter: BELUGA BEHR
>Assignee: Zoltan Haindrich
>Priority: Minor
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch,
> HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch
>
>
> If a user needs to look up a small subset of records quickly, they can use
> Apache HBase, if they need fast retrieval of larger sets of data, or fast
> joins, aggregations, they can use Apache Impala. It seems to me that Hive
> indexes do not serve much of a role in the future of Hive.
> Even without moving workloads to other products, columnar file formats with
> their statistics achieve similar goals as Hive indexes.
> Please consider dropping Indexes from the Apache Hive project.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363301#comment-16363301 ] Ashutosh Chauhan commented on HIVE-18448: - +1 > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch, > HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363227#comment-16363227
]
Hive QA commented on HIVE-18448:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910386/HIVE-18448.01.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 13100 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=179)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=250)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=205)
org.apache.hadoop.hive.ql.TestAcidOnTez.testGetSplitsLocks (batchId=224)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=187)
org.apache.hive.hcatalog.pig.TestSequenceFileHCatStorer.testWriteTimestamp
(batchId=192)
org.apache.hive.jdbc.TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
(batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9197/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9197/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9197/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12910386 - PreCommit-HIVE-Build
> Drop Support For Indexes From Apache Hive
> -
>
> Key: HIVE-18448
> URL: https://issues.apache.org/jira/browse/HIVE-18448
> Project: Hive
> Issue Type: Improvement
> Components: Indexing
>Reporter: BELUGA BEHR
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch,
> HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch
>
>
> If a user needs to look up a small subset of records quickly, they can use
> Apache HBase, if they need fast retrieval of larger sets of data, or fast
> joins, aggregations, they can use Apache Impala. It seems to me that Hive
> indexes do not serve much of a role in the future of Hive.
> Even without moving workloads to other products, columnar file formats with
> their statistics achieve similar goals as Hive indexes.
> Please consider dropping Indexes from the Apache Hive project.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362319#comment-16362319 ] Zoltan Haindrich commented on HIVE-18448: - patch#1: is bulk removal only; I plan to do the following in separate changes: * remove support from metastore as well * some parts of the index classes were used from StorageHandlers (hbase/accumulo) - I think that should be investigated separately - this patch may just hide those changes so I've not touched it > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch, > HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362306#comment-16362306 ] Zoltan Haindrich commented on HIVE-18448: - [~alangates] okay, in that case I will file a jira about removing it from the metastore as well; I guess that should be done on the separated branch > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01.patch, HIVE-18448.01wip02.patch, > HIVE-18448.01wip03.patch, HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361560#comment-16361560
]
Hive QA commented on HIVE-18448:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910262/HIVE-18448.01wip05.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 55 failed/errored test(s), 13119 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments]
(batchId=248)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_concatenate_indexed_table]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_index] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_index]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[database_drop]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_index] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_index_removes_partition_dirs]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_table_with_index]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[escape_comments]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_functions]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_indexes_edge_cases]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_indexes_syntax]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[special_character_in_tabnames_2]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitmap_and]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitmap_empty]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitmap_or]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[unicode_comments]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_view] (batchId=15)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=179)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_concatenate_indexed_table]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_create_index]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_drop_index]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_invalid_priv_v2]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_uri_index]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[bad_indextype]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_index_failure]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_negative_1]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[show_create_table_index]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[temp_table_index]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_indexed_table]
(batchId=94)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_view]
(batchId=111)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=250)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestFunctions.testGetFunctionNullDatabase[Embedded]
(batchId=205)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361184#comment-16361184 ] Alan Gates commented on HIVE-18448: --- Is there a use case for keeping it in the metastore if we remove it from the rest of Hive? Are we aware of non-Hive systems using the indices? > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01wip02.patch, HIVE-18448.01wip03.patch, > HIVE-18448.01wip04.patch, HIVE-18448.01wip05.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360693#comment-16360693
]
Hive QA commented on HIVE-18448:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 31m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 3m
4s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
2s{color} | {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
1s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
2s{color} | {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
2s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 4s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 2s{color}
| {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 1s{color}
| {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 2s{color}
| {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 2s{color}
| {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
4s{color} | {color:green} common: The patch generated 0 new + 0 unchanged -
2945 fixed = 0 total (was 2945) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
2s{color} | {color:green} root: The patch generated 0 new + 0 unchanged -
86060 fixed = 0 total (was 86060) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
2s{color} | {color:green} hcatalog/core: The patch generated 0 new + 0
unchanged - 1321 fixed = 0 total (was 1321) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
2s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 0
unchanged - 5951 fixed = 0 total (was 5951) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
2s{color} | {color:green} itests/util: The patch generated 0 new + 0 unchanged
- 1038 fixed = 0 total (was 1038) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
2s{color} | {color:green} ql: The patch generated 0 new + 0 unchanged - 45303
fixed = 0 total (was 45303) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
2s{color} | {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
1s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
2s{color} | {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
2s{color} | {color:red} hive-unit in the patch failed. {color} |
|
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360474#comment-16360474 ] Zoltan Haindrich commented on HIVE-18448: - [~pvary]: I'm not sure; I think about that as a separate change - if after this change nothing will use it; we might consider to remove it from the metastore as wellbut I know not much about metastore uses by other projects. cc: [~alangates] > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01wip02.patch, HIVE-18448.01wip03.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359064#comment-16359064
]
Hive QA commented on HIVE-18448:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909820/HIVE-18448.01wip03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9125/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9125/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9125/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-09 22:38:08.910
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9125/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-09 22:38:08.914
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
58bbfc7..717ef18 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 58bbfc7 HIVE-18580: Create tests to cover exchange partitions
(Marta Kuczora, reviewed by Adam Szita, Peter Vary)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 717ef18 HIVE-18607 : HBase HFile write does strange things
(Sergey Shelukhin, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-09 22:38:12.029
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9125/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java:36
Falling back to three-way merge...
Applied patch to 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java:36
Falling back to three-way merge...
Applied patch to 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java'
with conflicts.
U ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909820 - PreCommit-HIVE-Build
> Drop Support For Indexes From Apache Hive
> -
>
> Key: HIVE-18448
> URL: https://issues.apache.org/jira/browse/HIVE-18448
> Project: Hive
> Issue Type: Improvement
> Components: Indexing
>Reporter: BELUGA BEHR
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-18448.01wip02.patch, HIVE-18448.01wip03.patch
>
>
> If a user needs to look up a small subset of records quickly, they can use
> Apache HBase, if they need fast retrieval of larger sets of data, or fast
> joins, aggregations, they can use Apache Impala. It seems to me that Hive
> indexes do not serve much of a role in the future of Hive.
> Even without moving workloads to other products, columnar file formats with
> their statistics achieve similar goals as Hive indexes.
> Please consider dropping Indexes from the Apache Hive project.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358455#comment-16358455 ] Peter Vary commented on HIVE-18448: --- Hi [~kgyrtkirk], Thinking, wondering.. just a question :) Are we removing the indexes only from the HS2 side of the project, and we plan to keep it in the MetaStore? Thanks, Peter > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01wip02.patch, HIVE-18448.01wip03.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[
https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356858#comment-16356858
]
Hive QA commented on HIVE-18448:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909435/HIVE-18448.01wip02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9092/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9092/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9092/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-08 12:09:04.411
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9092/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-08 12:09:04.414
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 6e9b63e HIVE-18350 : load data should rename files consistent
with insert statements. (Deepak Jaiswal, reviewed by Sergey Shelukhin and
Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 6e9b63e HIVE-18350 : load data should rename files consistent
with insert statements. (Deepak Jaiswal, reviewed by Sergey Shelukhin and
Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-08 12:09:05.048
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9092/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestDDLWithRemoteMetastoreSecondNamenode.java:30
Falling back to three-way merge...
Applied patch to
'itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestDDLWithRemoteMetastoreSecondNamenode.java'
with conflicts.
error: patch failed:
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java:95
Falling back to three-way merge...
Applied patch to
'itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java:52
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java:209
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java' cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:65
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java' cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java:72
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java' cleanly.
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java:19
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java' with
conflicts.
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java:35
Falling back to three-way merge...
Applied patch to 'ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestDDLWithRemoteMetastoreSecondNamenode.java:30
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353805#comment-16353805 ] Zoltan Haindrich commented on HIVE-18448: - I always wanted to file a ticket for this...its sometimes problematic to keep the index related things around; since 3.0 will be a major release currently we might have a chance to drop support for it. > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Assignee: Zoltan Haindrich >Priority: Minor > Attachments: HIVE-18448.01wip01.patch > > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18448) Drop Support For Indexes From Apache Hive
[ https://issues.apache.org/jira/browse/HIVE-18448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332578#comment-16332578 ] Ashutosh Chauhan commented on HIVE-18448: - This is a good idea. Its not clear if indexes are ever useful in Hive. Further, its a source of confusion for users since they expect it to work in a certain way. > Drop Support For Indexes From Apache Hive > - > > Key: HIVE-18448 > URL: https://issues.apache.org/jira/browse/HIVE-18448 > Project: Hive > Issue Type: Improvement > Components: Indexing >Reporter: BELUGA BEHR >Priority: Minor > > If a user needs to look up a small subset of records quickly, they can use > Apache HBase, if they need fast retrieval of larger sets of data, or fast > joins, aggregations, they can use Apache Impala. It seems to me that Hive > indexes do not serve much of a role in the future of Hive. > Even without moving workloads to other products, columnar file formats with > their statistics achieve similar goals as Hive indexes. > Please consider dropping Indexes from the Apache Hive project. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
