[jira] [Commented] (HIVE-15202) Concurrent compactions for the same partition may generate malformed folder structure
[
https://issues.apache.org/jira/browse/HIVE-15202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672903#comment-15672903
]
Rui Li commented on HIVE-15202:
---
Thank you [~ekoifman]. That's great to know.
> Concurrent compactions for the same partition may generate malformed folder
> structure
> -
>
> Key: HIVE-15202
> URL: https://issues.apache.org/jira/browse/HIVE-15202
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>
> If two compactions run concurrently on a single partition, it may generate
> folder structure like this: (nested base dir)
> {noformat}
> drwxr-xr-x - root supergroup 0 2016-11-14 22:23
> /user/hive/warehouse/test/z=1/base_007/base_007
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_0
> -rw-r--r-- 3 root supergroup611 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_1
> -rw-r--r-- 3 root supergroup614 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_2
> -rw-r--r-- 3 root supergroup621 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_3
> -rw-r--r-- 3 root supergroup621 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_4
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_5
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_6
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_7
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_8
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_9
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15227) Optimize join + gby into semijoin
[ https://issues.apache.org/jira/browse/HIVE-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672817#comment-15672817 ] Ashutosh Chauhan commented on HIVE-15227: - yes.. there is. You may note (from the plan in attached patch) that Gby is only in mapper, not in reducer. This is possible since LSJ operator understands the semantics, such that GBy in reducer is obviated. Net perf win here is to not compute GBy on reducer side. > Optimize join + gby into semijoin > - > > Key: HIVE-15227 > URL: https://issues.apache.org/jira/browse/HIVE-15227 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-15227.patch > > > Calcite has a rule which can do this transformation. Lets take advantage of > this since Hive has native Left semi join operator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672593#comment-15672593
]
Sahil Takiar commented on HIVE-15199:
-
* Is the goal to trigger mvFile when the destination file is a blobstore? I
don't think thats the right approach because a {{FileUtils.copy}} will do a
client-side copy when running on S3, data will be downloaded from HDFS to HS2
and then uploaded to S3; the target should be to do a server-side copy (happens
internally on S3). A server side copy can only be triggered by called
{{FileSystem.rename}}.
* The listing optimization can be applied to HDFS too, right? It should
increase perf when running on HDFS too.
* A bit orthogonal to this JIRA, but {{mvFile}} should probably be called
copyFile because it always copies data.
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch, HIVE-15199.2.patch,
> HIVE-15199.3.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-14990) run all tests for MM tables and fix the issues that are found
[
https://issues.apache.org/jira/browse/HIVE-14990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668996#comment-15668996
]
Sergey Shelukhin edited comment on HIVE-14990 at 11/17/16 3:02 AM:
---
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{panel}
TestCliDriver:
stats_list_bucket
show_tablestatus
-vector_udf2-
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{panel}
was (Author: sershe):
Updated test list to fix/declare irrelevant before closing this. Only updated
the CliDriver list actually, haven't made my way thru it yet
{noformat}
TestCliDriver:
stats_list_bucket
show_tablestatus
vector_udf2
list_bucket_dml_14
autoColumnStats_9
stats_noscan_2
symlink_text_input_format
temp_table_precedence
offset_limit_global_optimizer
rand_partitionpruner2
materialized_view_authorization_sqlstd,materialized_*
merge_dynamic_partition, merge_dynamic_partition*
orc_vectorization_ppd
parquet_join2
repl_3_exim_metadata
sample6
sample_islocalmode_hook
smb_mapjoin_2,smb_mapjoin_3,smb_mapjoin_7
orc_createas1
exim_16_part_external,exim_17_part_managed,
TestEncryptedHDFSCliDriver:
encryption_ctas
encryption_drop_partition
encryption_insert_values
encryption_join_unencrypted_tbl
encryption_load_data_to_encrypted_tables
MiniLlapLocal:
exchgpartition2lel
cbo_rp_lineage2
create_merge_compressed
deleteAnalyze
delete_where_no_match
delete_where_non_partitioned
dynpart_sort_optimization
escape2
insert1
lineage2
lineage3
orc_llap
schema_evol_orc_nonvec_part
schema_evol_orc_vec_part
schema_evol_text_nonvec_part
schema_evol_text_vec_part
schema_evol_text_vecrow_part
smb_mapjoin_6
tez_dml
union_fast_stats
update_all_types
update_tmp_table
update_where_no_match
update_where_non_partitioned
vector_outer_join1
vector_outer_join4
MiniLlap:
load_fs2
orc_ppd_basic
external_table_with_space_in_location_path
file_with_header_footer
import_exported_table
schemeAuthority,schemeAuthority2
table_nonprintable
Minimr:
infer_bucket_sort_map_operators
infer_bucket_sort_merge
infer_bucket_sort_reducers_power_two
root_dir_external_table
scriptfile1
TestSymlinkTextInputFormat#testCombine
TestJdbcWithLocalClusterSpark, etc.
{noformat}
> run all tests for MM tables and fix the issues that are found
> -
>
> Key: HIVE-14990
> URL: https://issues.apache.org/jira/browse/HIVE-14990
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-14990.01.patch, HIVE-14990.02.patch,
> HIVE-14990.03.patch, HIVE-14990.04.patch, HIVE-14990.04.patch,
> HIVE-14990.05.patch, HIVE-14990.05.patch, HIVE-14990.06.patch,
> HIVE-14990.06.patch, HIVE-14990.07.patch, HIVE-14990.08.patch,
> HIVE-14990.09.patch, HIVE-14990.10.patch, HIVE-14990.10.patch,
> HIVE-14990.10.patch, HIVE-14990.patch
>
>
> Expected failures
> 1) All HCat tests (cannot write MM tables via the HCat writer)
> 2) Almost all merge tests (alter .. concat is not supported).
> 3) Tests that run dfs commands with specific paths (path changes).
> 4) Truncate column (not supported).
> 5) Describe formatted will have the new table fields in the output (before
> merging MM with ACID).
> 6) Many tests w/explain extended - diff in partition "base file name" (path
> changes).
> 7)
[jira] [Commented] (HIVE-14189) backport HIVE-13945 to branch-1
[
https://issues.apache.org/jira/browse/HIVE-14189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672491#comment-15672491
]
Sergey Shelukhin commented on HIVE-14189:
-
[~cartershanklin] [~jcamachorodriguez] [~ashutoshc] do you have access to the
text of a later SQL standard than 99? That one is pretty vague.
Seems like there's implicit cast to string in these cases on master:
{noformat}
POSTHOOK: query: select 'NaN' < 2.0 from src limit 1
false
POSTHOOK: query: select 1.0 < 'NaN' from src limit 1
true
{noformat}
branch-1 with this patch returns null by happenstance (because something else
inserts a cast to decimal from string), so the operation returns null.
I wonder if SQL standard has anything to say on this... 99 doesn't seem to.
> backport HIVE-13945 to branch-1
> ---
>
> Key: HIVE-14189
> URL: https://issues.apache.org/jira/browse/HIVE-14189
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Labels: TODOC1.3
> Attachments: HIVE-14189-branch-1.patch, HIVE-14189.01-branch-1.patch,
> HIVE-14189.02-branch-1.patch, HIVE-14189.03-branch-1.patch,
> HIVE-14189.04-branch-1.patch, HIVE-14189.05-branch-1.patch,
> HIVE-14189.06-branch-1.patch, HIVE-14189.07-branch-1.patch,
> HIVE-14189.08-branch-1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14189) backport HIVE-13945 to branch-1
[ https://issues.apache.org/jira/browse/HIVE-14189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672468#comment-15672468 ] Sergey Shelukhin commented on HIVE-14189: - Hmm, different types notwithstanding, this seems like the correct output. Master seems to be comparing strings. Let me see what SQL standard sayeth. > backport HIVE-13945 to branch-1 > --- > > Key: HIVE-14189 > URL: https://issues.apache.org/jira/browse/HIVE-14189 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC1.3 > Attachments: HIVE-14189-branch-1.patch, HIVE-14189.01-branch-1.patch, > HIVE-14189.02-branch-1.patch, HIVE-14189.03-branch-1.patch, > HIVE-14189.04-branch-1.patch, HIVE-14189.05-branch-1.patch, > HIVE-14189.06-branch-1.patch, HIVE-14189.07-branch-1.patch, > HIVE-14189.08-branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14189) backport HIVE-13945 to branch-1
[
https://issues.apache.org/jira/browse/HIVE-14189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672440#comment-15672440
]
Sergey Shelukhin commented on HIVE-14189:
-
Difference (with CBO disabled) on master vs branch-1
{noformat}
2016-11-16 17:37:28,788 DEBUG [main]: optimizer.ConstantPropagateProcFactory
(ConstantPropagateProcFactory.java:foldExprFull(310)) -
Folding expression:GenericUDFOPLessThan(GenericUDFToDecimal(Const string NaN),
Const int 2) -> Const boolean null
2016-11-16T18:06:46,788 DEBUG [5682f00b-91b1-4106-a205-b55e98f03ad0 main]
optimizer.ConstantPropagateProcFactory:
Folding expression:GenericUDFOPLessThan(Const string NaN, Const decimal(1,0) 2)
-> Const boolean false
{noformat}
> backport HIVE-13945 to branch-1
> ---
>
> Key: HIVE-14189
> URL: https://issues.apache.org/jira/browse/HIVE-14189
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Labels: TODOC1.3
> Attachments: HIVE-14189-branch-1.patch, HIVE-14189.01-branch-1.patch,
> HIVE-14189.02-branch-1.patch, HIVE-14189.03-branch-1.patch,
> HIVE-14189.04-branch-1.patch, HIVE-14189.05-branch-1.patch,
> HIVE-14189.06-branch-1.patch, HIVE-14189.07-branch-1.patch,
> HIVE-14189.08-branch-1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15225) QueryPlan.getJSONValue should code against empty string values
[
https://issues.apache.org/jira/browse/HIVE-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672413#comment-15672413
]
Hive QA commented on HIVE-15225:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839260/HIVE-15225.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10694 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1]
(batchId=90)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2161/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2161/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2161/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839260 - PreCommit-HIVE-Build
> QueryPlan.getJSONValue should code against empty string values
> --
>
> Key: HIVE-15225
> URL: https://issues.apache.org/jira/browse/HIVE-15225
> Project: Hive
> Issue Type: Bug
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-15225.1.patch
>
>
> The current {{QueryPlan.getJSONValue}} implementation is as below:
> {code}
> public String getJSONValue(Object value) {
> String v = "null";
> if (value != null) {
> v = value.toString();
> if (v.charAt(0) != '[' && v.charAt(0) != '{') {
> v = "\"" + v + "\"";
> }
> }
> return v;
> }
> {code}
> When {{value.toString()}} returns an empty string, a
> StringIndexOutOfRangeException would be thrown out when "v.charAt(0)" is
> evaluated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14189) backport HIVE-13945 to branch-1
[ https://issues.apache.org/jira/browse/HIVE-14189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14189: Attachment: HIVE-14189.08-branch-1.patch Most of the out file changes are valid; updated them. Some are spurious (and so are timeouts, I suspect) - rerunning. ops_comparison seems to be a real problem that doesn't happen on master version of the same patch, need to take a look > backport HIVE-13945 to branch-1 > --- > > Key: HIVE-14189 > URL: https://issues.apache.org/jira/browse/HIVE-14189 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC1.3 > Attachments: HIVE-14189-branch-1.patch, HIVE-14189.01-branch-1.patch, > HIVE-14189.02-branch-1.patch, HIVE-14189.03-branch-1.patch, > HIVE-14189.04-branch-1.patch, HIVE-14189.05-branch-1.patch, > HIVE-14189.06-branch-1.patch, HIVE-14189.07-branch-1.patch, > HIVE-14189.08-branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1478) Non-boolean expression in WHERE should be rejected
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672332#comment-15672332
]
Ashutosh Chauhan commented on HIVE-1478:
This should be closed as won't fix since as discussed on HIVE-13659 this is
already supported in Hive and there is no good reason to withdra support for
this.
By the way, on master these queries should not throw any exception. MR or
otherwise.
> Non-boolean expression in WHERE should be rejected
> --
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch, HIVE-1478.2.patch
>
>
> Automatically casting strings or other types into boolean may confuse even
> the user - and somehow it doesn't always work (HIVE-15089)
> sql2011 states that "where expression" should accept a boolean expression.
> Original reported problem:
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Boolean
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
> ... 5 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15227) Optimize join + gby into semijoin
[
https://issues.apache.org/jira/browse/HIVE-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672331#comment-15672331
]
Vineet Garg commented on HIVE-15227:
[~ashutoshc] I discussed this with [~jcamachorodriguez] today and it looks like
Hive does group by before doing left semi join anyway. e.g. {noformat} select a
from t1 left semi join t2 on t1.a=t2.b {noformat} will generate GBY on t2.b
before doing Left Semi Join.
Is there any improvement which physical Left Semi Join operator has in this
case ?
> Optimize join + gby into semijoin
> -
>
> Key: HIVE-15227
> URL: https://issues.apache.org/jira/browse/HIVE-15227
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-15227.patch
>
>
> Calcite has a rule which can do this transformation. Lets take advantage of
> this since Hive has native Left semi join operator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-1478) Non-boolean expression in WHERE should be rejected
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672326#comment-15672326
]
Hive QA commented on HIVE-1478:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839245/HIVE-1478.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10673 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[ptf_general_queries.q,auto_join_reordering_values.q,sample2.q,join1.q,decimal_join.q,mapjoin_subquery2.q,join32_lessSize.q,mapjoin1.q,order2.q,skewjoinopt18.q,union_remove_18.q,join25.q,groupby9.q,bucketsortoptimize_insert_6.q,ctas.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=91)
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropPartition
(batchId=204)
org.apache.hive.hcatalog.api.repl.commands.TestCommands.org.apache.hive.hcatalog.api.repl.commands.TestCommands
(batchId=166)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2159/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2159/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2159/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839245 - PreCommit-HIVE-Build
> Non-boolean expression in WHERE should be rejected
> --
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch, HIVE-1478.2.patch
>
>
> Automatically casting strings or other types into boolean may confuse even
> the user - and somehow it doesn't always work (HIVE-15089)
> sql2011 states that "where expression" should accept a boolean expression.
> Original reported problem:
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Boolean
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
> ... 5 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10901) Optimize multi column distinct queries
[ https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672321#comment-15672321 ] Ashutosh Chauhan commented on HIVE-10901: - +1 > Optimize multi column distinct queries > > > Key: HIVE-10901 > URL: https://issues.apache.org/jira/browse/HIVE-10901 > Project: Hive > Issue Type: New Feature > Components: CBO, Logical Optimizer >Affects Versions: 1.2.0 >Reporter: Mostafa Mokhtar >Assignee: Pengcheng Xiong > Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch, > HIVE-10901.04.patch, HIVE-10901.patch > > > HIVE-10568 is useful only when there is a distinct on one column. It can be > expanded for multiple column cases too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15227) Optimize join + gby into semijoin
[ https://issues.apache.org/jira/browse/HIVE-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15227: Status: Patch Available (was: Open) > Optimize join + gby into semijoin > - > > Key: HIVE-15227 > URL: https://issues.apache.org/jira/browse/HIVE-15227 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-15227.patch > > > Calcite has a rule which can do this transformation. Lets take advantage of > this since Hive has native Left semi join operator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15227) Optimize join + gby into semijoin
[ https://issues.apache.org/jira/browse/HIVE-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15227: Attachment: HIVE-15227.patch > Optimize join + gby into semijoin > - > > Key: HIVE-15227 > URL: https://issues.apache.org/jira/browse/HIVE-15227 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-15227.patch > > > Calcite has a rule which can do this transformation. Lets take advantage of > this since Hive has native Left semi join operator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10901) Optimize multi column distinct queries
[ https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672237#comment-15672237 ] Pengcheng Xiong commented on HIVE-10901: [~ashutoshc], could u take a look? Thanks. > Optimize multi column distinct queries > > > Key: HIVE-10901 > URL: https://issues.apache.org/jira/browse/HIVE-10901 > Project: Hive > Issue Type: New Feature > Components: CBO, Logical Optimizer >Affects Versions: 1.2.0 >Reporter: Mostafa Mokhtar >Assignee: Pengcheng Xiong > Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch, > HIVE-10901.04.patch, HIVE-10901.patch > > > HIVE-10568 is useful only when there is a distinct on one column. It can be > expanded for multiple column cases too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15219) LLAP: Allow additional slider global parameters to be set while creating the LLAP package
[ https://issues.apache.org/jira/browse/HIVE-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated HIVE-15219: -- Attachment: HIVE-15219.02.patch Updated patch. Works with --slider-appconfig-global propertName=value [~gopalv] - ready for review when you get a chance. Tested locally. > LLAP: Allow additional slider global parameters to be set while creating the > LLAP package > - > > Key: HIVE-15219 > URL: https://issues.apache.org/jira/browse/HIVE-15219 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15219.02.patch, HIVE-15219.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15226) Add a different masking comment to qtests blobstore output
[
https://issues.apache.org/jira/browse/HIVE-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672205#comment-15672205
]
Sahil Takiar commented on HIVE-15226:
-
Overall looks good. Only comment is that {{patternsWithMaskComments}} should
probably return a {{Pair}} instead of a {{String[][]}}.
{{Pattern.compile(...)}} is an expensive call, so you really only want to do it
once per Hive QA run.
> Add a different masking comment to qtests blobstore output
> --
>
> Key: HIVE-15226
> URL: https://issues.apache.org/jira/browse/HIVE-15226
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Minor
> Attachments: HIVE-15226.1.patch
>
>
> The output of TestBlobstoreCliDriver is masking all s3a patch so that we can
> use the tests with any other blobstore scheme.
> It should be good to have a specific masking comment for those paths instead
> of the generic " A masked pattern was here " so that we can verify
> that certain tests are indeed using the blobstore path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10901) Optimize multi column distinct queries
[
https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672177#comment-15672177
]
Hive QA commented on HIVE-10901:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839247/HIVE-10901.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10666 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[groupby_map_ppr.q,nullgroup4_multi_distinct.q,join_rc.q,union14.q,smb_mapjoin_12.q,vector_cast_constant.q,union_remove_4.q,auto_join11.q,load_dyn_part7.q,udaf_collect_set.q,vectorization_12.q,groupby_sort_skew_1.q,groupby_sort_skew_1_23.q,smb_mapjoin_25.q,skewjoinopt12.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=118)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2158/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2158/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2158/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839247 - PreCommit-HIVE-Build
> Optimize multi column distinct queries
>
>
> Key: HIVE-10901
> URL: https://issues.apache.org/jira/browse/HIVE-10901
> Project: Hive
> Issue Type: New Feature
> Components: CBO, Logical Optimizer
>Affects Versions: 1.2.0
>Reporter: Mostafa Mokhtar
>Assignee: Pengcheng Xiong
> Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch,
> HIVE-10901.04.patch, HIVE-10901.patch
>
>
> HIVE-10568 is useful only when there is a distinct on one column. It can be
> expanded for multiple column cases too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15226) Add a different masking comment to qtests blobstore output
[ https://issues.apache.org/jira/browse/HIVE-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-15226: --- Status: Patch Available (was: Open) [~stakiar] What do you think about this approach? It uses a specific mask comment, such as "### BLOBSTORE_PATH ###" when a s3a, swift or any other blobstore scheme is found on the output. > Add a different masking comment to qtests blobstore output > -- > > Key: HIVE-15226 > URL: https://issues.apache.org/jira/browse/HIVE-15226 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Minor > Attachments: HIVE-15226.1.patch > > > The output of TestBlobstoreCliDriver is masking all s3a patch so that we can > use the tests with any other blobstore scheme. > It should be good to have a specific masking comment for those paths instead > of the generic " A masked pattern was here " so that we can verify > that certain tests are indeed using the blobstore path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15226) Add a different masking comment to qtests blobstore output
[ https://issues.apache.org/jira/browse/HIVE-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-15226: --- Attachment: HIVE-15226.1.patch > Add a different masking comment to qtests blobstore output > -- > > Key: HIVE-15226 > URL: https://issues.apache.org/jira/browse/HIVE-15226 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Minor > Attachments: HIVE-15226.1.patch > > > The output of TestBlobstoreCliDriver is masking all s3a patch so that we can > use the tests with any other blobstore scheme. > It should be good to have a specific masking comment for those paths instead > of the generic " A masked pattern was here " so that we can verify > that certain tests are indeed using the blobstore path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15192) Use Calcite to de-correlate and plan subqueries
[ https://issues.apache.org/jira/browse/HIVE-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-15192: --- Status: Patch Available (was: Open) > Use Calcite to de-correlate and plan subqueries > --- > > Key: HIVE-15192 > URL: https://issues.apache.org/jira/browse/HIVE-15192 > Project: Hive > Issue Type: Task > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-15192.patch > > > Currently support of subqueries is limited [Link to original spec | > https://issues.apache.org/jira/secure/attachment/12614003/SubQuerySpec.pdf]. > Using Calcite to plan and de-correlate subqueries will help Hive get rid of > these limitations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15173) Allow dec as an alias for decimal
[ https://issues.apache.org/jira/browse/HIVE-15173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15173: Attachment: HIVE-15173.1.patch Updated test case. > Allow dec as an alias for decimal > - > > Key: HIVE-15173 > URL: https://issues.apache.org/jira/browse/HIVE-15173 > Project: Hive > Issue Type: Sub-task > Components: Parser >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-15173.1.patch, HIVE-15173.patch > > > Standard allows dec as an alias for decimal -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15173) Allow dec as an alias for decimal
[ https://issues.apache.org/jira/browse/HIVE-15173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15173: Status: Patch Available (was: Open) > Allow dec as an alias for decimal > - > > Key: HIVE-15173 > URL: https://issues.apache.org/jira/browse/HIVE-15173 > Project: Hive > Issue Type: Sub-task > Components: Parser >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-15173.1.patch, HIVE-15173.patch > > > Standard allows dec as an alias for decimal -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15173) Allow dec as an alias for decimal
[ https://issues.apache.org/jira/browse/HIVE-15173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15173: Status: Open (was: Patch Available) > Allow dec as an alias for decimal > - > > Key: HIVE-15173 > URL: https://issues.apache.org/jira/browse/HIVE-15173 > Project: Hive > Issue Type: Sub-task > Components: Parser >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-15173.patch > > > Standard allows dec as an alias for decimal -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14189) backport HIVE-13945 to branch-1
[
https://issues.apache.org/jira/browse/HIVE-14189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15672020#comment-15672020
]
Hive QA commented on HIVE-14189:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839226/HIVE-14189.07-branch-1.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 201 failed/errored test(s), 8083 tests
executed
*Failed tests:*
{noformat}
TestAcidOnTez - did not produce a TEST-*.xml file (likely timed out)
(batchId=377)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=359)
TestAuthorizationPreEventListener - did not produce a TEST-*.xml file (likely
timed out) (batchId=392)
TestAuthzApiEmbedAuthorizerInEmbed - did not produce a TEST-*.xml file (likely
timed out) (batchId=369)
TestAuthzApiEmbedAuthorizerInRemote - did not produce a TEST-*.xml file (likely
timed out) (batchId=375)
TestBeeLineWithArgs - did not produce a TEST-*.xml file (likely timed out)
(batchId=399)
TestCLIAuthzSessionContext - did not produce a TEST-*.xml file (likely timed
out) (batchId=417)
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=384)
TestClientSideAuthorizationProvider - did not produce a TEST-*.xml file (likely
timed out) (batchId=391)
TestCompactor - did not produce a TEST-*.xml file (likely timed out)
(batchId=380)
TestCreateUdfEntities - did not produce a TEST-*.xml file (likely timed out)
(batchId=379)
TestCustomAuthentication - did not produce a TEST-*.xml file (likely timed out)
(batchId=400)
TestDBTokenStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=343)
TestDDLWithRemoteMetastoreSecondNamenode - did not produce a TEST-*.xml file
(likely timed out) (batchId=378)
TestDynamicSerDe - did not produce a TEST-*.xml file (likely timed out)
(batchId=346)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=356)
TestEmbeddedThriftBinaryCLIService - did not produce a TEST-*.xml file (likely
timed out) (batchId=403)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=351)
TestFolderPermissions - did not produce a TEST-*.xml file (likely timed out)
(batchId=386)
TestHS2AuthzContext - did not produce a TEST-*.xml file (likely timed out)
(batchId=420)
TestHS2AuthzSessionContext - did not produce a TEST-*.xml file (likely timed
out) (batchId=421)
TestHS2ClearDanglingScratchDir - did not produce a TEST-*.xml file (likely
timed out) (batchId=407)
TestHS2ImpersonationWithRemoteMS - did not produce a TEST-*.xml file (likely
timed out) (batchId=408)
TestHiveAuthorizerCheckInvocation - did not produce a TEST-*.xml file (likely
timed out) (batchId=395)
TestHiveAuthorizerShowFilters - did not produce a TEST-*.xml file (likely timed
out) (batchId=394)
TestHiveHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=397)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=371)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=361)
TestHiveMetaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=374)
TestHiveServer2 - did not produce a TEST-*.xml file (likely timed out)
(batchId=423)
TestHiveServer2SessionTimeout - did not produce a TEST-*.xml file (likely timed
out) (batchId=424)
TestHiveSessionImpl - did not produce a TEST-*.xml file (likely timed out)
(batchId=404)
TestHs2Hooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=376)
TestJdbcDriver2 - did not produce a TEST-*.xml file (likely timed out)
(batchId=411)
TestJdbcMetadataApiAuth - did not produce a TEST-*.xml file (likely timed out)
(batchId=422)
TestJdbcWithLocalClusterSpark - did not produce a TEST-*.xml file (likely timed
out) (batchId=416)
TestJdbcWithMiniHS2 - did not produce a TEST-*.xml file (likely timed out)
(batchId=413)
TestJdbcWithMiniMr - did not produce a TEST-*.xml file (likely timed out)
(batchId=412)
TestJdbcWithSQLAuthUDFBlacklist - did not produce a TEST-*.xml file (likely
timed out) (batchId=418)
TestJdbcWithSQLAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=419)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=383)
TestMTQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=381)
TestMarkPartition - did not produce a TEST-*.xml file (likely timed out)
(batchId=368)
TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=372)
TestMetaStoreAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=357)
TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file (likely
timed out) (batchId=355)
TestMetaStoreEndFunctionListener - did not produce a TEST-*.xml file (likely
timed out)
[jira] [Updated] (HIVE-15225) QueryPlan.getJSONValue should code against empty string values
[
https://issues.apache.org/jira/browse/HIVE-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yibing Shi updated HIVE-15225:
--
Status: Patch Available (was: Open)
> QueryPlan.getJSONValue should code against empty string values
> --
>
> Key: HIVE-15225
> URL: https://issues.apache.org/jira/browse/HIVE-15225
> Project: Hive
> Issue Type: Bug
>Reporter: Yibing Shi
> Attachments: HIVE-15225.1.patch
>
>
> The current {{QueryPlan.getJSONValue}} implementation is as below:
> {code}
> public String getJSONValue(Object value) {
> String v = "null";
> if (value != null) {
> v = value.toString();
> if (v.charAt(0) != '[' && v.charAt(0) != '{') {
> v = "\"" + v + "\"";
> }
> }
> return v;
> }
> {code}
> When {{value.toString()}} returns an empty string, a
> StringIndexOutOfRangeException would be thrown out when "v.charAt(0)" is
> evaluated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HIVE-15225) QueryPlan.getJSONValue should code against empty string values
[
https://issues.apache.org/jira/browse/HIVE-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yibing Shi reassigned HIVE-15225:
-
Assignee: Yibing Shi
> QueryPlan.getJSONValue should code against empty string values
> --
>
> Key: HIVE-15225
> URL: https://issues.apache.org/jira/browse/HIVE-15225
> Project: Hive
> Issue Type: Bug
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-15225.1.patch
>
>
> The current {{QueryPlan.getJSONValue}} implementation is as below:
> {code}
> public String getJSONValue(Object value) {
> String v = "null";
> if (value != null) {
> v = value.toString();
> if (v.charAt(0) != '[' && v.charAt(0) != '{') {
> v = "\"" + v + "\"";
> }
> }
> return v;
> }
> {code}
> When {{value.toString()}} returns an empty string, a
> StringIndexOutOfRangeException would be thrown out when "v.charAt(0)" is
> evaluated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15225) QueryPlan.getJSONValue should code against empty string values
[
https://issues.apache.org/jira/browse/HIVE-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yibing Shi updated HIVE-15225:
--
Attachment: HIVE-15225.1.patch
Attach a quick patch
> QueryPlan.getJSONValue should code against empty string values
> --
>
> Key: HIVE-15225
> URL: https://issues.apache.org/jira/browse/HIVE-15225
> Project: Hive
> Issue Type: Bug
>Reporter: Yibing Shi
> Attachments: HIVE-15225.1.patch
>
>
> The current {{QueryPlan.getJSONValue}} implementation is as below:
> {code}
> public String getJSONValue(Object value) {
> String v = "null";
> if (value != null) {
> v = value.toString();
> if (v.charAt(0) != '[' && v.charAt(0) != '{') {
> v = "\"" + v + "\"";
> }
> }
> return v;
> }
> {code}
> When {{value.toString()}} returns an empty string, a
> StringIndexOutOfRangeException would be thrown out when "v.charAt(0)" is
> evaluated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14982) Remove some reserved keywords in 2.2
[ https://issues.apache.org/jira/browse/HIVE-14982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671972#comment-15671972 ] Ashutosh Chauhan commented on HIVE-14982: - +1 > Remove some reserved keywords in 2.2 > > > Key: HIVE-14982 > URL: https://issues.apache.org/jira/browse/HIVE-14982 > Project: Hive > Issue Type: Bug >Reporter: Pengcheng Xiong >Assignee: Pengcheng Xiong > Attachments: HIVE-14982.01.patch > > > It seems that CACHE, DAYOFWEEK, VIEWS are reserved keywords in master. This > conflicts with SQL2011 standard. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13282) GroupBy and select operator encounter ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-13282:
---
Target Version/s: 1.2.2, 2.2.0 (was: 1.2.2, 2.2.0, 2.1.1)
> GroupBy and select operator encounter ArrayIndexOutOfBoundsException
>
>
> Key: HIVE-13282
> URL: https://issues.apache.org/jira/browse/HIVE-13282
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.0.0, 2.1.0
>Reporter: Vikram Dixit K
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: HIVE-13282.01.patch, smb_fail_issue.patch,
> smb_groupby.q, smb_groupby.q.out
>
>
> The group by and select operators run into the ArrayIndexOutOfBoundsException
> when they incorrectly initialize themselves with tag 0 but the incoming tag
> id is different.
> {code}
> select count(*) from
> (select rt1.id from
> (select t1.key as id, t1.value as od from tab t1 group by key, value) rt1) vt1
> join
> (select rt2.id from
> (select t2.key as id, t2.value as od from tab_part t2 group by key, value)
> rt2) vt2
> where vt1.id=vt2.id;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-15199:
---
Attachment: HIVE-15199.3.patch
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch, HIVE-15199.2.patch,
> HIVE-15199.3.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-1478) Non-boolean expression in WHERE should be rejected
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-1478:
---
Attachment: HIVE-1478.2.patch
patch#2)
removed qtest cases which contained invalid where expressions.
it turned out that there was a ticket to do support this conversion for
vectorization(HIVE-13659)...not sure what to do now:
* close this as won't fix - and open an another issue that this fails for mr
* i've undone the {{VectorizationContext}} change of HIVE-13659 in this patch -
as it will became an unreachable codepath after this change.
I think it would be better to stick to the standard...because casting anything
into a boolean may help a bad query to hide in the woods ;)
[~ashutoshc],[~mmccline] what's your opinion?
> Non-boolean expression in WHERE should be rejected
> --
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch, HIVE-1478.2.patch
>
>
> Automatically casting strings or other types into boolean may confuse even
> the user - and somehow it doesn't always work (HIVE-15089)
> sql2011 states that "where expression" should accept a boolean expression.
> Original reported problem:
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Boolean
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
> ... 5 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-10901) Optimize mutli column distinct queries
[ https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-10901: --- Status: Patch Available (was: Open) > Optimize mutli column distinct queries > > > Key: HIVE-10901 > URL: https://issues.apache.org/jira/browse/HIVE-10901 > Project: Hive > Issue Type: New Feature > Components: CBO, Logical Optimizer >Affects Versions: 1.2.0 >Reporter: Mostafa Mokhtar >Assignee: Pengcheng Xiong > Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch, > HIVE-10901.04.patch, HIVE-10901.patch > > > HIVE-10568 is useful only when there is a distinct on one column. It can be > expanded for multiple column cases too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10901) Optimize multi column distinct queries
[ https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-10901: --- Summary: Optimize multi column distinct queries (was: Optimize mutli column distinct queries ) > Optimize multi column distinct queries > > > Key: HIVE-10901 > URL: https://issues.apache.org/jira/browse/HIVE-10901 > Project: Hive > Issue Type: New Feature > Components: CBO, Logical Optimizer >Affects Versions: 1.2.0 >Reporter: Mostafa Mokhtar >Assignee: Pengcheng Xiong > Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch, > HIVE-10901.04.patch, HIVE-10901.patch > > > HIVE-10568 is useful only when there is a distinct on one column. It can be > expanded for multiple column cases too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10901) Optimize mutli column distinct queries
[ https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-10901: --- Status: Open (was: Patch Available) > Optimize mutli column distinct queries > > > Key: HIVE-10901 > URL: https://issues.apache.org/jira/browse/HIVE-10901 > Project: Hive > Issue Type: New Feature > Components: CBO, Logical Optimizer >Affects Versions: 1.2.0 >Reporter: Mostafa Mokhtar >Assignee: Pengcheng Xiong > Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch, > HIVE-10901.04.patch, HIVE-10901.patch > > > HIVE-10568 is useful only when there is a distinct on one column. It can be > expanded for multiple column cases too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-10901) Optimize mutli column distinct queries
[ https://issues.apache.org/jira/browse/HIVE-10901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengcheng Xiong updated HIVE-10901: --- Attachment: HIVE-10901.04.patch > Optimize mutli column distinct queries > > > Key: HIVE-10901 > URL: https://issues.apache.org/jira/browse/HIVE-10901 > Project: Hive > Issue Type: New Feature > Components: CBO, Logical Optimizer >Affects Versions: 1.2.0 >Reporter: Mostafa Mokhtar >Assignee: Pengcheng Xiong > Attachments: HIVE-10901.02.patch, HIVE-10901.03.patch, > HIVE-10901.04.patch, HIVE-10901.patch > > > HIVE-10568 is useful only when there is a distinct on one column. It can be > expanded for multiple column cases too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671747#comment-15671747
]
Hive QA commented on HIVE-15199:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839221/HIVE-15199.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 209 failed/errored test(s), 10664 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=109)
[enforce_order.q,ppd_join2.q,smb_mapjoin_21.q,load_dyn_part15.q,udf_min.q,groupby_resolution.q,mapjoin_memcheck.q,subquery_exists.q,join27.q,alter_merge_stats_orc.q,union_remove_2.q,vector_orderby_5.q,groupby6_map_skew.q,join12.q,union9.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=128)
[union_remove_15.q,bucket_map_join_tez1.q,groupby7_noskew.q,bucketmapjoin1.q,subquery_multiinsert.q,auto_join8.q,auto_join6.q,groupby2_map_skew.q,lateral_view_explode2.q,join28.q,load_dyn_part1.q,skewjoinopt17.q,skewjoin_union_remove_1.q,union_remove_20.q,bucketmapjoin5.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[add_part_multiple]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge_2_orc]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_add_partition]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[array_map_access_nonconstant]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_3]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_add_column2]
(batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avrocountemptytbl]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ba_table_udfs]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_insert]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnStatsUpdateForStatsOptimizer_2]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[column_names_with_leading_and_trailing_spaces]
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[complex_alias]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[concat_op] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constantPropWhen]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constantfolding]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_when_case]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_merge_compressed]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_5] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_7] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_query1]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_precision]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_stats]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[except_all] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_01_nonpart]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_03_nonpart_over_compat]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_08_nonpart_rename]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_10_external_managed]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_12_external_location]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_13_managed_location]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_14_managed_location_over_existing]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_22_import_exist_authsuccess]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_24_import_nonexist_authsuccess]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[folder_predicate]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_distinct_samekey]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_window]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_nullvalues]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[implicit_decimal]
(batchId=61)
[jira] [Commented] (HIVE-15219) LLAP: Allow additional slider global parameters to be set while creating the LLAP package
[ https://issues.apache.org/jira/browse/HIVE-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671739#comment-15671739 ] Siddharth Seth commented on HIVE-15219: --- The intent was for the user to provide free-form text, site.global.prop1=val1,site.global.prop2=val2. Think I'm going to simplify this to allow specific properties to be set, using the same mechanism used for hiveconf. i.e. --slider_appconfig_global site.global.prop1=val1 --slider_appconfig_global site.global.prop2=val2 > LLAP: Allow additional slider global parameters to be set while creating the > LLAP package > - > > Key: HIVE-15219 > URL: https://issues.apache.org/jira/browse/HIVE-15219 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-15219.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13282) GroupBy and select operator encounter ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671710#comment-15671710
]
Matt McCline commented on HIVE-13282:
-
No, this patch is not needed for 2.1.1
> GroupBy and select operator encounter ArrayIndexOutOfBoundsException
>
>
> Key: HIVE-13282
> URL: https://issues.apache.org/jira/browse/HIVE-13282
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.0.0, 2.1.0
>Reporter: Vikram Dixit K
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: HIVE-13282.01.patch, smb_fail_issue.patch,
> smb_groupby.q, smb_groupby.q.out
>
>
> The group by and select operators run into the ArrayIndexOutOfBoundsException
> when they incorrectly initialize themselves with tag 0 but the incoming tag
> id is different.
> {code}
> select count(*) from
> (select rt1.id from
> (select t1.key as id, t1.value as od from tab t1 group by key, value) rt1) vt1
> join
> (select rt2.id from
> (select t2.key as id, t2.value as od from tab_part t2 group by key, value)
> rt2) vt2
> where vt1.id=vt2.id;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13282) GroupBy and select operator encounter ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HIVE-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671668#comment-15671668
]
Sergio Peña commented on HIVE-13282:
[~vikram.dixit] Is this patch still needed for 2.1.1? We're looking to release
an RC this or next week.
> GroupBy and select operator encounter ArrayIndexOutOfBoundsException
>
>
> Key: HIVE-13282
> URL: https://issues.apache.org/jira/browse/HIVE-13282
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.1, 2.0.0, 2.1.0
>Reporter: Vikram Dixit K
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: HIVE-13282.01.patch, smb_fail_issue.patch,
> smb_groupby.q, smb_groupby.q.out
>
>
> The group by and select operators run into the ArrayIndexOutOfBoundsException
> when they incorrectly initialize themselves with tag 0 but the incoming tag
> id is different.
> {code}
> select count(*) from
> (select rt1.id from
> (select t1.key as id, t1.value as od from tab t1 group by key, value) rt1) vt1
> join
> (select rt2.id from
> (select t2.key as id, t2.value as od from tab_part t2 group by key, value)
> rt2) vt2
> where vt1.id=vt2.id;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-1478) Non-boolean expression in WHERE should be rejected
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-1478:
---
Description:
Automatically casting strings or other types into boolean may confuse even the
user - and somehow it doesn't always work (HIVE-15089)
sql2011 states that "where expression" should accept a boolean expression.
Original reported problem:
If the expression in the where clause does not evaluate to a boolean, the job
will fail with the following exception in the task logs:
Query:
SELECT key FROM src WHERE 1;
Exception in mapper:
2010-07-21 17:00:31,460 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"key":"238","value":"val_238"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.Boolean
at
org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
... 5 more
was:
This problem is still present (
Currently:
If the expression in the where clause does not evaluate to a boolean, the job
will fail with the following exception in the task logs:
Query:
SELECT key FROM src WHERE 1;
Exception in mapper:
2010-07-21 17:00:31,460 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"key":"238","value":"val_238"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.Boolean
at
org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
... 5 more
> Non-boolean expression in WHERE should be rejected
> --
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch
>
>
> Automatically casting strings or other types into boolean may confuse even
> the user - and somehow it doesn't always work (HIVE-15089)
> sql2011 states that "where expression" should accept a boolean expression.
> Original reported problem:
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at
[jira] [Updated] (HIVE-1478) Non-boolean expression in WHERE should be rejected
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-1478:
---
Summary: Non-boolean expression in WHERE should be rejected (was:
Non-boolean expression in WHERE clause throws exception)
> Non-boolean expression in WHERE should be rejected
> --
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch
>
>
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Boolean
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
> ... 5 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-1478) Non-boolean expression in WHERE should be rejected
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-1478:
---
Description:
This problem is still present (
Currently:
If the expression in the where clause does not evaluate to a boolean, the job
will fail with the following exception in the task logs:
Query:
SELECT key FROM src WHERE 1;
Exception in mapper:
2010-07-21 17:00:31,460 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"key":"238","value":"val_238"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.Boolean
at
org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
... 5 more
was:
If the expression in the where clause does not evaluate to a boolean, the job
will fail with the following exception in the task logs:
Query:
SELECT key FROM src WHERE 1;
Exception in mapper:
2010-07-21 17:00:31,460 FATAL ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {"key":"238","value":"val_238"}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.Boolean
at
org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
... 5 more
> Non-boolean expression in WHERE should be rejected
> --
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch
>
>
> This problem is still present (
> Currently:
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Boolean
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
>
[jira] [Commented] (HIVE-15215) Files on S3 are deleted one by one in INSERT OVERWRITE queries
[
https://issues.apache.org/jira/browse/HIVE-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671633#comment-15671633
]
Sergio Peña commented on HIVE-15215:
It might be good to use approach #1 where it sets the staging directory under
the scratch directory only for blobstorage so Hive can do the final move at the
end. This move may be in parallel as renames are not allowed on S3, so it may
be faster.
> Files on S3 are deleted one by one in INSERT OVERWRITE queries
> --
>
> Key: HIVE-15215
> URL: https://issues.apache.org/jira/browse/HIVE-15215
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Sahil Takiar
>
> When running {{INSERT OVERWRITE}} queries the files to overwrite are deleted
> one by one. The reason is that, by default, hive.exec.stagingdir is inside
> the target table directory.
> Ideally Hive would just delete the entire table directory, but it can't do
> that since the staging data is also inside the directory. Instead it deletes
> each file one-by-one, which is very slow.
> There are a few ways to fix this:
> 1: Move the staging directory outside the table location. This can be done by
> setting hive.exec.stagingdir to a different location when running on S3. It
> would be nice if users didn't have to explicitly set this when running on S3
> and things just worked out-of-the-box. My understanding is that
> hive.exec.stagingdir was only added to support HDFS encryption zones. Since
> S3 doesn't have encryption zones, there should be no problem with using the
> value of hive.exec.scratchdir to store all intermediate data instead.
> 2: Multi-thread the delete operations
> 3: See if the {{S3AFileSystem}} can expose some type of bulk delete op
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15180) Extend JSONMessageFactory to store additional information about metadata objects on different table events
[
https://issues.apache.org/jira/browse/HIVE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671568#comment-15671568
]
Hive QA commented on HIVE-15180:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839218/HIVE-15180.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10694 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hive.hcatalog.api.TestHCatClientNotification.addPartition
(batchId=217)
org.apache.hive.hcatalog.api.TestHCatClientNotification.createTable
(batchId=217)
org.apache.hive.hcatalog.api.TestHCatClientNotification.dropPartition
(batchId=217)
org.apache.hive.hcatalog.api.TestHCatClientNotification.dropTable (batchId=217)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2155/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2155/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2155/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839218 - PreCommit-HIVE-Build
> Extend JSONMessageFactory to store additional information about metadata
> objects on different table events
> --
>
> Key: HIVE-15180
> URL: https://issues.apache.org/jira/browse/HIVE-15180
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-15180.1.patch, HIVE-15180.2.patch,
> HIVE-15180.3.patch, HIVE-15180.3.patch
>
>
> We want the {{NOTIFICATION_LOG}} table to capture additional information
> about the metadata objects when {{DbNotificationListener}} captures different
> events for a table (create/drop/alter).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-6536) Reduce dependencies of org.apache.hive:hive-jdbc maven module
[ https://issues.apache.org/jira/browse/HIVE-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671508#comment-15671508 ] Tero Paananen commented on HIVE-6536: - This issue has become much, much worse in the 2.1.0 version of the JDBC driver. There's something like 80MB of dependencies in the 2.1.0 version. I'm not sure how much bigger it is compared to 1.2.1 I was using before, but it's at least 2x - 4x bigger. > Reduce dependencies of org.apache.hive:hive-jdbc maven module > - > > Key: HIVE-6536 > URL: https://issues.apache.org/jira/browse/HIVE-6536 > Project: Hive > Issue Type: Improvement > Components: JDBC >Affects Versions: 0.12.0 > Environment: org.apache.hive:hive-jdbc:jar:0.12.0 >Reporter: Kevin Minder > Attachments: hive-jdbc-maven-dependencies-0-13.log, > hive-jdbc-maven-dependencies.log > > > The Hive JDBC driver maven module requires a significant number of > dependencies that are likely unnecessary and will result in bloating of > consumers. Most of this is a result of the dependency on > org.apache.hive:hive-cli. I have attached a portion of the output from mvn > depedency:tree output for a client that depends on the > org.apache.hive:hive-jdbc module. Note the extra 2.0.6.1-102 in the output > is the result of our local build and publish to a local nexus repo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14189) backport HIVE-13945 to branch-1
[ https://issues.apache.org/jira/browse/HIVE-14189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-14189: Attachment: HIVE-14189.07-branch-1.patch Trying again... > backport HIVE-13945 to branch-1 > --- > > Key: HIVE-14189 > URL: https://issues.apache.org/jira/browse/HIVE-14189 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC1.3 > Attachments: HIVE-14189-branch-1.patch, HIVE-14189.01-branch-1.patch, > HIVE-14189.02-branch-1.patch, HIVE-14189.03-branch-1.patch, > HIVE-14189.04-branch-1.patch, HIVE-14189.05-branch-1.patch, > HIVE-14189.06-branch-1.patch, HIVE-14189.07-branch-1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14902) HiveServer2: Cleanup code which checks for ThriftJDBCSerde usage
[ https://issues.apache.org/jira/browse/HIVE-14902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671439#comment-15671439 ] Ziyang Zhao commented on HIVE-14902: Failed tests passed in my local. Seems unrelated. > HiveServer2: Cleanup code which checks for ThriftJDBCSerde usage > > > Key: HIVE-14902 > URL: https://issues.apache.org/jira/browse/HIVE-14902 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC, ODBC >Affects Versions: 2.1.0 >Reporter: Vaibhav Gumashta >Assignee: Ziyang Zhao > Attachments: HIVE-14902.2.patch > > > As part of HIVE-14039, we simplified the condition checking by setting a > boolean in SessionState (from HiveSessionImpl) and then using that in other > layers to decide whether we should use ThriftJDBCSerde. However, there are > some parts of the code (e.g. FileSinkOperator.closeOp, Driver.getResults, > SQLOperation.getNextRowSet) where the condition checking is still complicated > - we should refactor that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-15199:
---
Attachment: HIVE-15199.2.patch
[~stakiar] Thanks. I updated the patch with:
- Using an hybrid solution that checks if a file exists on the list status or
if exists on the FS.
- Change the renameNonLocal to renameIsAllowed
For the Hive lock, yes, Hive should have a lock to avoid another client inserts
data on the same table.
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch, HIVE-15199.2.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15200) Support setOp in subQuery with parentheses
[
https://issues.apache.org/jira/browse/HIVE-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671428#comment-15671428
]
Hive QA commented on HIVE-15200:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839211/HIVE-15200.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 80 failed/errored test(s), 10665 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=104)
[skewjoin_union_remove_2.q,avro_decimal_native.q,skewjoinopt8.q,bucketmapjoin_negative3.q,union32.q,stats6.q,groupby2_map.q,stats_only_null.q,insert_into3.q,join18_multi_distinct.q,vectorization_6.q,cross_join.q,stats9.q,timestamp_1.q,join24.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=128)
[union_remove_15.q,bucket_map_join_tez1.q,groupby7_noskew.q,bucketmapjoin1.q,subquery_multiinsert.q,auto_join8.q,auto_join6.q,groupby2_map_skew.q,lateral_view_explode2.q,join28.q,load_dyn_part1.q,skewjoinopt17.q,skewjoin_union_remove_1.q,union_remove_20.q,bucketmapjoin5.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join4] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join5] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join6] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join7] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic2]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_bigdata]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_position]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input14] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input14_limit]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input17] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input18] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input20] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input33] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input34] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input35] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input36] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input37] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input38] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input5] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join4] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join5] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join6] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join7] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join8] (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonreserved_keywords_input37]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partcols1] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_transform]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_union] (batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rcfile_bigdata]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rcfile_null_value]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[regexp_extract]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[setop_subq] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subq] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_alias]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[transform_ppr1]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[transform_ppr2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union] (batchId=4)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[java_mr_example]
(batchId=218)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes2]
(batchId=218)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes3]
(batchId=218)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes4]
(batchId=218)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes5]
(batchId=218)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes]
(batchId=218)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr1]
(batchId=131)
[jira] [Commented] (HIVE-15217) Add watch mode to llap status tool
[ https://issues.apache.org/jira/browse/HIVE-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671407#comment-15671407 ] Prasanth Jayachandran commented on HIVE-15217: -- i see what you mean. Makes sense. Will update the patch. > Add watch mode to llap status tool > -- > > Key: HIVE-15217 > URL: https://issues.apache.org/jira/browse/HIVE-15217 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-15217.1.patch > > > There is few seconds overhead for launching the llap status command. To avoid > we can add "watch" mode to llap status tool that refreshes the status after > configured interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15180) Extend JSONMessageFactory to store additional information about metadata objects on different table events
[
https://issues.apache.org/jira/browse/HIVE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-15180:
Attachment: HIVE-15180.3.patch
> Extend JSONMessageFactory to store additional information about metadata
> objects on different table events
> --
>
> Key: HIVE-15180
> URL: https://issues.apache.org/jira/browse/HIVE-15180
> Project: Hive
> Issue Type: Sub-task
> Components: repl
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-15180.1.patch, HIVE-15180.2.patch,
> HIVE-15180.3.patch, HIVE-15180.3.patch
>
>
> We want the {{NOTIFICATION_LOG}} table to capture additional information
> about the metadata objects when {{DbNotificationListener}} captures different
> events for a table (create/drop/alter).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671380#comment-15671380
]
Deepesh Khandelwal commented on HIVE-15220:
---
Thanks [~thejas] for the review and commit!
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
> Fix For: 2.2.0
>
> Attachments: HIVE-15220.1.patch
>
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15217) Add watch mode to llap status tool
[ https://issues.apache.org/jira/browse/HIVE-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671355#comment-15671355 ] Sergey Shelukhin commented on HIVE-15217: - Why print output at all in wait mode? We can print the last one if anything at all. > Add watch mode to llap status tool > -- > > Key: HIVE-15217 > URL: https://issues.apache.org/jira/browse/HIVE-15217 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-15217.1.patch > > > There is few seconds overhead for launching the llap status command. To avoid > we can add "watch" mode to llap status tool that refreshes the status after > configured interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15217) Add watch mode to llap status tool
[ https://issues.apache.org/jira/browse/HIVE-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671346#comment-15671346 ] Prasanth Jayachandran commented on HIVE-15217: -- I will add another option that waits/watches until RUNNING (or any state) state. The problem with keeping shorter refresh interval is with the way we are viewing the output. Currently the output is printed as json which is only machine readable. The output that gets printed is way too much even for 10s of nodes. IMO we should change the output format to something like tez job monitor and update in-place but that's a broader change and optionally output json. I think both formats will be useful. Refreshing too often and spitting json out is completely unreadable. Thoughts? > Add watch mode to llap status tool > -- > > Key: HIVE-15217 > URL: https://issues.apache.org/jira/browse/HIVE-15217 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-15217.1.patch > > > There is few seconds overhead for launching the llap status command. To avoid > we can add "watch" mode to llap status tool that refreshes the status after > configured interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15217) Add watch mode to llap status tool
[ https://issues.apache.org/jira/browse/HIVE-15217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671298#comment-15671298 ] Sergey Shelukhin commented on HIVE-15217: - 1) Is it possible to wait for specific status, e.g. "running"? 2) I think the wait interval can be much shorter <1sec. > Add watch mode to llap status tool > -- > > Key: HIVE-15217 > URL: https://issues.apache.org/jira/browse/HIVE-15217 > Project: Hive > Issue Type: Improvement > Components: llap >Affects Versions: 2.2.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Minor > Attachments: HIVE-15217.1.patch > > > There is few seconds overhead for launching the llap status command. To avoid > we can add "watch" mode to llap status tool that refreshes the status after > configured interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-15220:
-
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Committed to master.
Thanks for the patch [~deepesh]!
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
> Fix For: 2.2.0
>
> Attachments: HIVE-15220.1.patch
>
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671234#comment-15671234
]
Sahil Takiar commented on HIVE-15199:
-
@spena a few comments:
* It may be better to take a hybrid of the list files approach + the exists
approach; for blobstores like S3 listfiles is only eventually consistent; this
means listfiles may not return all the files that are actually there. One way
to get around this is to first do the listfiles, and then checks if the
targetFilename exists or not. This has the advantage of the perf gains of using
listfiles, but avoids the consistency problems
* I remember we discussed offline about concerns w.r.t multiple INSERT INTO
queries running against the same table, but I just remembered that Hive Locking
(https://cwiki.apache.org/confluence/display/Hive/Locking) should prevent that
from ever happening, correct?
* It would be nice (although not necessary) if we changed the name of
{{renameNonLocal}} to something more descriptive
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671234#comment-15671234
]
Sahil Takiar edited comment on HIVE-15199 at 11/16/16 6:34 PM:
---
[~spena] a few comments:
* It may be better to take a hybrid of the list files approach + the exists
approach; for blobstores like S3 listfiles is only eventually consistent; this
means listfiles may not return all the files that are actually there. One way
to get around this is to first do the listfiles, and then checks if the
targetFilename exists or not. This has the advantage of the perf gains of using
listfiles, but avoids the consistency problems
* I remember we discussed offline about concerns w.r.t multiple INSERT INTO
queries running against the same table, but I just remembered that Hive Locking
(https://cwiki.apache.org/confluence/display/Hive/Locking) should prevent that
from ever happening, correct?
* It would be nice (although not necessary) if we changed the name of
{{renameNonLocal}} to something more descriptive
was (Author: stakiar):
@spena a few comments:
* It may be better to take a hybrid of the list files approach + the exists
approach; for blobstores like S3 listfiles is only eventually consistent; this
means listfiles may not return all the files that are actually there. One way
to get around this is to first do the listfiles, and then checks if the
targetFilename exists or not. This has the advantage of the perf gains of using
listfiles, but avoids the consistency problems
* I remember we discussed offline about concerns w.r.t multiple INSERT INTO
queries running against the same table, but I just remembered that Hive Locking
(https://cwiki.apache.org/confluence/display/Hive/Locking) should prevent that
from ever happening, correct?
* It would be nice (although not necessary) if we changed the name of
{{renameNonLocal}} to something more descriptive
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15200) Support setOp in subQuery with parentheses
[
https://issues.apache.org/jira/browse/HIVE-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15200:
---
Status: Patch Available (was: Open)
> Support setOp in subQuery with parentheses
> --
>
> Key: HIVE-15200
> URL: https://issues.apache.org/jira/browse/HIVE-15200
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15200.01.patch
>
>
> {code}
> explain select key from ((select key from src) union (select key from
> src))subq;
> {code}
> will throw
> {code}
> FAILED: ParseException line 1:47 cannot recognize input near 'union' '('
> 'select' in subquery source
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15200) Support setOp in subQuery with parentheses
[
https://issues.apache.org/jira/browse/HIVE-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15200:
---
Attachment: (was: HIVE-15200.01.patch)
> Support setOp in subQuery with parentheses
> --
>
> Key: HIVE-15200
> URL: https://issues.apache.org/jira/browse/HIVE-15200
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15200.01.patch
>
>
> {code}
> explain select key from ((select key from src) union (select key from
> src))subq;
> {code}
> will throw
> {code}
> FAILED: ParseException line 1:47 cannot recognize input near 'union' '('
> 'select' in subquery source
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15200) Support setOp in subQuery with parentheses
[
https://issues.apache.org/jira/browse/HIVE-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15200:
---
Attachment: HIVE-15200.01.patch
> Support setOp in subQuery with parentheses
> --
>
> Key: HIVE-15200
> URL: https://issues.apache.org/jira/browse/HIVE-15200
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15200.01.patch
>
>
> {code}
> explain select key from ((select key from src) union (select key from
> src))subq;
> {code}
> will throw
> {code}
> FAILED: ParseException line 1:47 cannot recognize input near 'union' '('
> 'select' in subquery source
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15200) Support setOp in subQuery with parentheses
[
https://issues.apache.org/jira/browse/HIVE-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15200:
---
Status: Open (was: Patch Available)
> Support setOp in subQuery with parentheses
> --
>
> Key: HIVE-15200
> URL: https://issues.apache.org/jira/browse/HIVE-15200
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15200.01.patch
>
>
> {code}
> explain select key from ((select key from src) union (select key from
> src))subq;
> {code}
> will throw
> {code}
> FAILED: ParseException line 1:47 cannot recognize input near 'union' '('
> 'select' in subquery source
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9482) Hive parquet timestamp compatibility
[ https://issues.apache.org/jira/browse/HIVE-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671191#comment-15671191 ] Vitalii Diravka commented on HIVE-9482: --- Why this hive.parquet.timestamp.skip.conversion option is enabled by default? Since according [parquet spec|https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md#timestamp_millis], parquet files don't keep local timezone. And we cann't distinguish from file what was the value of that option while parquet file was generating. > Hive parquet timestamp compatibility > > > Key: HIVE-9482 > URL: https://issues.apache.org/jira/browse/HIVE-9482 > Project: Hive > Issue Type: Bug > Components: File Formats >Affects Versions: 0.15.0 >Reporter: Szehon Ho >Assignee: Szehon Ho > Fix For: 1.2.0 > > Attachments: HIVE-9482.2.patch, HIVE-9482.patch, HIVE-9482.patch, > parquet_external_time.parq > > > In current Hive implementation, timestamps are stored in UTC (converted from > current timezone), based on original parquet timestamp spec. > However, we find this is not compatibility with other tools, and after some > investigation it is not the way of the other file formats, or even some > databases (Hive Timestamp is more equivalent of 'timestamp without timezone' > datatype). > This is the first part of the fix, which will restore compatibility with > parquet-timestamp files generated by external tools by skipping conversion on > reading. > Later fix will change the write path to not convert, and stop the > read-conversion even for files written by Hive itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15114) Remove extra MoveTask operators
[
https://issues.apache.org/jira/browse/HIVE-15114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671167#comment-15671167
]
Sergio Peña commented on HIVE-15114:
[~stakiar] The conditional task created on the GenMapRedUtils class also
creates tasks for Spark or Tez. The MoveTask should be linked to any task
regardless the execution engine.
The .q tests added to the TestBlobstorecliDriver can be executed with any
engine as well, so nothing to change. However, I did not verify those tests
were running correctly on them. I will run them and verify they work.
> Remove extra MoveTask operators
> ---
>
> Key: HIVE-15114
> URL: https://issues.apache.org/jira/browse/HIVE-15114
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 2.1.0
>Reporter: Sahil Takiar
>Assignee: Sergio Peña
> Attachments: HIVE-15114.WIP.1.patch, HIVE-15114.WIP.2.patch
>
>
> When running simple insert queries (e.g. {{INSERT INTO TABLE ... VALUES
> ...}}) there an extraneous {{MoveTask}s is created.
> This is problematic when the scratch directory is on S3 since renames require
> copying the entire dataset.
> For simple queries (like the one above), there are two MoveTasks. The first
> one moves the output data from one file in the scratch directory to another
> file in the scratch directory. The second MoveTask moves the data from the
> scratch directory to its final table location.
> The first MoveTask should not be necessary. The goal of this JIRA it to
> remove it. This should help improve performance when running on S3.
> It seems that the first Move might be caused by a dependency resolution
> problem in the optimizer, where a dependent task doesn't get properly removed
> when the task it depends on is filtered by a condition resolver.
> A dummy {{MoveTask}} is added in the
> {{GenMapRedUtils.createMRWorkForMergingFiles}} method. This method creates a
> conditional task which launches a job to merge tasks at the end of the file.
> At the end of the conditional job there is a MoveTask.
> Even though Hive decides that the conditional merge job is no needed, it
> seems the MoveTask is still added to the plan.
> Seems this extra {{MoveTask}} may have been added intentionally. Not sure why
> yet. The {{ConditionalResolverMergeFiles}} says that one of three tasks will
> be returned: move task only, merge task only, merge task followed by a move
> task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671136#comment-15671136
]
Sergio Peña commented on HIVE-15199:
Guys, [~ashutoshc], could you help me review the patch?
What it does is to use the alternative condition {{mvFile}} when the
destination filesystem is a blobstore. And, because this {{mvFile}} was calling
{{destFs.exists()}} for every file on S3, then I changed it to get a list of
files, and check whether the {{destf}} exists or not on that list.
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15202) Concurrent compactions for the same partition may generate malformed folder structure
[
https://issues.apache.org/jira/browse/HIVE-15202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671134#comment-15671134
]
Eugene Koifman commented on HIVE-15202:
---
When compactions are triggered by the system automatically they have logic to
avoid concurrent invocations but not when triggered by the user. It should and
will be fixed (almost certainly before 2.2 is released)
> Concurrent compactions for the same partition may generate malformed folder
> structure
> -
>
> Key: HIVE-15202
> URL: https://issues.apache.org/jira/browse/HIVE-15202
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>
> If two compactions run concurrently on a single partition, it may generate
> folder structure like this: (nested base dir)
> {noformat}
> drwxr-xr-x - root supergroup 0 2016-11-14 22:23
> /user/hive/warehouse/test/z=1/base_007/base_007
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_0
> -rw-r--r-- 3 root supergroup611 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_1
> -rw-r--r-- 3 root supergroup614 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_2
> -rw-r--r-- 3 root supergroup621 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_3
> -rw-r--r-- 3 root supergroup621 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_4
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_5
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_6
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_7
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_8
> -rw-r--r-- 3 root supergroup201 2016-11-14 21:46
> /user/hive/warehouse/test/z=1/base_007/bucket_9
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15144) JSON.org license is now CatX
[
https://issues.apache.org/jira/browse/HIVE-15144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671129#comment-15671129
]
Alan Gates commented on HIVE-15144:
---
We may have a bit more time in which to get this done. Based on a discussion
on legal-discuss we may have a 6 month grace period. This will allow us to
keep doing releases while we do the work to get this switched out and tested.
> JSON.org license is now CatX
>
>
> Key: HIVE-15144
> URL: https://issues.apache.org/jira/browse/HIVE-15144
> Project: Hive
> Issue Type: Bug
>Reporter: Robert Kanter
>Priority: Blocker
> Fix For: 2.2.0
>
>
> per [update resolved legal|http://www.apache.org/legal/resolved.html#json]:
> {quote}
> CAN APACHE PRODUCTS INCLUDE WORKS LICENSED UNDER THE JSON LICENSE?
> No. As of 2016-11-03 this has been moved to the 'Category X' license list.
> Prior to this, use of the JSON Java library was allowed. See Debian's page
> for a list of alternatives.
> {quote}
> I'm not sure when this dependency was first introduced, but it looks like
> it's currently used in a few places:
> https://github.com/apache/hive/search?p=1=%22org.json%22=%E2%9C%93
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15224) replace org.json usage in branch-1 with as minor changes as possible
[ https://issues.apache.org/jira/browse/HIVE-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671124#comment-15671124 ] Alan Gates commented on HIVE-15224: --- Based on a thread going on in legal discuss we may have a bit before we have to do this. I'm working on getting a 6 month grace period so we can still do maintenance releases without waiting until we've tested and validated a replacement JSON library. > replace org.json usage in branch-1 with as minor changes as possible > > > Key: HIVE-15224 > URL: https://issues.apache.org/jira/browse/HIVE-15224 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Daniel Voros > Fix For: 2.2.0 > > > branch-1 / master have diverged in many ways - StatsCollector have changed; > EximUtil supports new replication > ...so backporting any changes from master would be hard. > maybe we should use some drop-in replacement like the android one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-15199:
---
Status: Patch Available (was: Open)
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15199) INSERT INTO data on S3 is replacing the old rows with the new ones
[
https://issues.apache.org/jira/browse/HIVE-15199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-15199:
---
Attachment: HIVE-15199.1.patch
> INSERT INTO data on S3 is replacing the old rows with the new ones
> --
>
> Key: HIVE-15199
> URL: https://issues.apache.org/jira/browse/HIVE-15199
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Sergio Peña
>Assignee: Sergio Peña
>Priority: Critical
> Attachments: HIVE-15199.1.patch
>
>
> Any INSERT INTO statement run on S3 tables and when the scratch directory is
> saved on S3 is deleting old rows of the table.
> {noformat}
> hive> set hive.blobstore.use.blobstore.as.scratchdir=true;
> hive> create table t1 (id int, name string) location 's3a://spena-bucket/t1';
> hive> insert into table t1 values (1,'name1');
> hive> select * from t1;
> 1 name1
> hive> insert into table t1 values (2,'name2');
> hive> select * from t1;
> 2 name2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15220) WebHCat test driver not capturing end time of test accurately
[
https://issues.apache.org/jira/browse/HIVE-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671091#comment-15671091
]
Thejas M Nair commented on HIVE-15220:
--
+1
> WebHCat test driver not capturing end time of test accurately
> -
>
> Key: HIVE-15220
> URL: https://issues.apache.org/jira/browse/HIVE-15220
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Trivial
> Attachments: HIVE-15220.1.patch
>
>
> Webhcat e2e testsuite prints message while ending test run:
> {noformat}
> Ending test at 1479264720
> {noformat}
> Currently it is not capturing the end time correctly.
> NO PRECOMMIT TESTS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15057) Support other types of operators (other than SELECT)
[
https://issues.apache.org/jira/browse/HIVE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671073#comment-15671073
]
Hive QA commented on HIVE-15057:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839188/HIVE-15057.wip.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10696 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2152/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2152/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2152/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839188 - PreCommit-HIVE-Build
> Support other types of operators (other than SELECT)
>
>
> Key: HIVE-15057
> URL: https://issues.apache.org/jira/browse/HIVE-15057
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer, Physical Optimizer
>Reporter: Chao Sun
>Assignee: Chao Sun
> Attachments: HIVE-15057.wip.patch
>
>
> Currently only SELECT operators are supported for nested column pruning. We
> should add support for other types of operators so the optimization can work
> for complex queries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15208) Query string should be HTML encoded for Web UI
[ https://issues.apache.org/jira/browse/HIVE-15208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HIVE-15208: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.2.0 Status: Resolved (was: Patch Available) Pushed to the master branch. Thanks Xuefu for the review. > Query string should be HTML encoded for Web UI > -- > > Key: HIVE-15208 > URL: https://issues.apache.org/jira/browse/HIVE-15208 > Project: Hive > Issue Type: Bug > Components: Web UI >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 2.2.0 > > Attachments: HIVE-15208.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14582) Add trunc(numeric) udf
[
https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670933#comment-15670933
]
Hive QA commented on HIVE-14582:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839177/HIVE-14582.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10680 tests
executed
*Failed tests:*
{noformat}
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=120)
[groupby3_map.q,union11.q,union26.q,mapreduce1.q,mapjoin_addjar.q,bucket_map_join_spark1.q,udf_example_add.q,multi_insert_with_join.q,sample7.q,auto_join_nulls.q,ppd_outer_join4.q,load_dyn_part8.q,sample6.q,bucket_map_join_1.q,auto_sortmerge_join_9.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=91)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2151/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2151/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2151/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839177 - PreCommit-HIVE-Build
> Add trunc(numeric) udf
> --
>
> Key: HIVE-14582
> URL: https://issues.apache.org/jira/browse/HIVE-14582
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-14582.1.patch, HIVE-14582.2.patch, HIVE-14582.patch
>
>
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15057) Support other types of operators (other than SELECT)
[ https://issues.apache.org/jira/browse/HIVE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-15057: Attachment: HIVE-15057.wip.patch > Support other types of operators (other than SELECT) > > > Key: HIVE-15057 > URL: https://issues.apache.org/jira/browse/HIVE-15057 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Physical Optimizer >Reporter: Chao Sun >Assignee: Chao Sun > Attachments: HIVE-15057.wip.patch > > > Currently only SELECT operators are supported for nested column pruning. We > should add support for other types of operators so the optimization can work > for complex queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15057) Support other types of operators (other than SELECT)
[ https://issues.apache.org/jira/browse/HIVE-15057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HIVE-15057: Attachment: (was: HIVE-15057.wip.patch) > Support other types of operators (other than SELECT) > > > Key: HIVE-15057 > URL: https://issues.apache.org/jira/browse/HIVE-15057 > Project: Hive > Issue Type: Sub-task > Components: Logical Optimizer, Physical Optimizer >Reporter: Chao Sun >Assignee: Chao Sun > Attachments: HIVE-15057.wip.patch > > > Currently only SELECT operators are supported for nested column pruning. We > should add support for other types of operators so the optimization can work > for complex queries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-15224) replace org.json usage in branch-1 with as minor changes as possible
[ https://issues.apache.org/jira/browse/HIVE-15224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Voros reassigned HIVE-15224: --- Assignee: Daniel Voros > replace org.json usage in branch-1 with as minor changes as possible > > > Key: HIVE-15224 > URL: https://issues.apache.org/jira/browse/HIVE-15224 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Daniel Voros > Fix For: 2.2.0 > > > branch-1 / master have diverged in many ways - StatsCollector have changed; > EximUtil supports new replication > ...so backporting any changes from master would be hard. > maybe we should use some drop-in replacement like the android one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14582) Add trunc(numeric) udf
[ https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14582: Status: Patch Available (was: Open) > Add trunc(numeric) udf > -- > > Key: HIVE-14582 > URL: https://issues.apache.org/jira/browse/HIVE-14582 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Ashutosh Chauhan >Assignee: Chinna Rao Lalam > Attachments: HIVE-14582.1.patch, HIVE-14582.2.patch, HIVE-14582.patch > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14582) Add trunc(numeric) udf
[
https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinna Rao Lalam updated HIVE-14582:
Attachment: HIVE-14582.2.patch
{quote}
udf_trunc.q,
udf_trunc_number.q
{quote}
tests failed because of function description change, it was not updated in test
files. Remaining failures are not related to this patch.
Patch.2 Updated with new function description in test files.
> Add trunc(numeric) udf
> --
>
> Key: HIVE-14582
> URL: https://issues.apache.org/jira/browse/HIVE-14582
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-14582.1.patch, HIVE-14582.2.patch, HIVE-14582.patch
>
>
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14582) Add trunc(numeric) udf
[ https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14582: Status: Open (was: Patch Available) > Add trunc(numeric) udf > -- > > Key: HIVE-14582 > URL: https://issues.apache.org/jira/browse/HIVE-14582 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Ashutosh Chauhan >Assignee: Chinna Rao Lalam > Attachments: HIVE-14582.1.patch, HIVE-14582.patch > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14840) MSCK not adding the missing partitions to Hive Metastore when the partition names are not in lowercase
[ https://issues.apache.org/jira/browse/HIVE-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushil Kumar S updated HIVE-14840: -- Assignee: (was: Sushil Kumar S) > MSCK not adding the missing partitions to Hive Metastore when the partition > names are not in lowercase > -- > > Key: HIVE-14840 > URL: https://issues.apache.org/jira/browse/HIVE-14840 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1 >Reporter: Sushil Kumar S >Priority: Minor > Labels: hive > > Hi, > There's is a bug while running MSCK REPAIR TABLE EXTERNAL_TABLE_NAME on > Hive 1.2.1, all the partition that are not present in the metastore are being > listed but not added if the partition names are not in lowercase, in other > words if an external path has a camel case based name and value i.e > s3n://some_external_path/myPartition=01 it just gets listed as partition not > found in metastore but doesn’t add it. > However, am not able to run ALTER TABLE MY_EXTERNAL_TABLE RECOVER PARTITIONS; > on hive 1.2 and based on the source code from hive-exec am able to see under > org/apache/hadoop/hive/ql/parse/HiveParser.g:1001:1 that there's no token > matching in the grammar for RECOVER PARTITIONS. > Example: > - When external path = s3n://some_external_path/myPartition=01 >hive> MSCK REPAIR TABLE my_external_table; >Partitions not in metastore: my_external_table:mypartition=01 >Time taken: 1.729 seconds, Fetched: 2 row(s) > hive> show partitions foster.ola_raven_raven_users_raw; > OK > Time taken: 0.901 seconds, Fetched: 0 row(s) > - When external path = s3n://some_external_path/mypartition=01 > hive> MSCK REPAIR TABLE my_external_table; > Partitions not in metastore: my_external_table:mypartition=01 > Repair: Added partition to metastore my_external_table:mypartition=01 > Time taken: 1.729 seconds, Fetched: 2 row(s) > hive> show partitions my_external_table; > OK > mypartition=01 > Time taken: 1.101 seconds, Fetched: 1 row(s) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HIVE-13539) HiveHFileOutputFormat searching the wrong directory for HFiles
[
https://issues.apache.org/jira/browse/HIVE-13539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang reopened HIVE-13539:
Assignee: Tim Robertson (was: Matt McCline)
> HiveHFileOutputFormat searching the wrong directory for HFiles
> --
>
> Key: HIVE-13539
> URL: https://issues.apache.org/jira/browse/HIVE-13539
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 1.1.0
> Environment: Built into CDH 5.4.7
>Reporter: Tim Robertson
>Assignee: Tim Robertson
>Priority: Blocker
> Attachments: hive_hfile_output_format.q,
> hive_hfile_output_format.q.out
>
>
> When creating HFiles for a bulkload in HBase I believe it is looking in the
> wrong directory to find the HFiles, resulting in the following exception:
> {code}
> Error: java.lang.RuntimeException: Hive Runtime Error while closing
> operators: java.io.IOException: Multiple family directories found in
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:295)
> at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: Multiple family directories found in
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:188)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:958)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:598)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> ... 7 more
> Caused by: java.io.IOException: Multiple family directories found in
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
> at
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:158)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:185)
> ... 11 more
> {code}
> The issue is that is looks for the HFiles in
> {{hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary}}
> when I believe it should be looking in the task attempt subfolder, such as
> {{hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary/attempt_1461004169450_0002_r_00_1000}}.
> This can be reproduced in any HFile creation such as:
> {code:sql}
> CREATE TABLE coords_hbase(id INT, x DOUBLE, y DOUBLE)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping' = ':key,o:x,o:y',
> 'hbase.table.default.storage.type' = 'binary');
> SET hfile.family.path=/tmp/coords_hfiles/o;
> SET hive.hbase.generatehfiles=true;
> INSERT OVERWRITE TABLE coords_hbase
> SELECT id, decimalLongitude, decimalLatitude
> FROM source
> CLUSTER BY id;
> {code}
> Any advice greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13539) HiveHFileOutputFormat searching the wrong directory for HFiles
[
https://issues.apache.org/jira/browse/HIVE-13539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670315#comment-15670315
]
Chaoyu Tang commented on HIVE-13539:
The issue could only be reproduced when multiple reducers are used.
[~timrobertson100], would you like to provide a new qtest?
> HiveHFileOutputFormat searching the wrong directory for HFiles
> --
>
> Key: HIVE-13539
> URL: https://issues.apache.org/jira/browse/HIVE-13539
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 1.1.0
> Environment: Built into CDH 5.4.7
>Reporter: Tim Robertson
>Assignee: Matt McCline
>Priority: Blocker
> Attachments: hive_hfile_output_format.q,
> hive_hfile_output_format.q.out
>
>
> When creating HFiles for a bulkload in HBase I believe it is looking in the
> wrong directory to find the HFiles, resulting in the following exception:
> {code}
> Error: java.lang.RuntimeException: Hive Runtime Error while closing
> operators: java.io.IOException: Multiple family directories found in
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:295)
> at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: Multiple family directories found in
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:188)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:958)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:598)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> ... 7 more
> Caused by: java.io.IOException: Multiple family directories found in
> hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary
> at
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:158)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:185)
> ... 11 more
> {code}
> The issue is that is looks for the HFiles in
> {{hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary}}
> when I believe it should be looking in the task attempt subfolder, such as
> {{hdfs://c1n1.gbif.org:8020/user/hive/warehouse/tim.db/coords_hbase/_temporary/2/_temporary/attempt_1461004169450_0002_r_00_1000}}.
> This can be reproduced in any HFile creation such as:
> {code:sql}
> CREATE TABLE coords_hbase(id INT, x DOUBLE, y DOUBLE)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping' = ':key,o:x,o:y',
> 'hbase.table.default.storage.type' = 'binary');
> SET hfile.family.path=/tmp/coords_hfiles/o;
> SET hive.hbase.generatehfiles=true;
> INSERT OVERWRITE TABLE coords_hbase
> SELECT id, decimalLongitude, decimalLatitude
> FROM source
> CLUSTER BY id;
> {code}
> Any advice greatly appreciated
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13557) Make interval keyword optional while specifying DAY in interval arithmetic
[
https://issues.apache.org/jira/browse/HIVE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670308#comment-15670308
]
Zoltan Haindrich commented on HIVE-13557:
-
failures are unrelated; HIVE-15201 HIVE-15116 HIVE-15115
[~pxiong] can you please take a look at the new changes?
> Make interval keyword optional while specifying DAY in interval arithmetic
> --
>
> Key: HIVE-13557
> URL: https://issues.apache.org/jira/browse/HIVE-13557
> Project: Hive
> Issue Type: Sub-task
> Components: Types
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13557.1.patch, HIVE-13557.1.patch,
> HIVE-13557.1.patch, HIVE-13557.2.patch, HIVE-13557.3.patch, HIVE-13557.3.patch
>
>
> Currently we support expressions like: {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) - INTERVAL '30' DAY) AND
> DATE('2000-01-31')
> {code}
> We should support:
> {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) + (-30) DAY) AND
> DATE('2000-01-31')
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13557) Make interval keyword optional while specifying DAY in interval arithmetic
[
https://issues.apache.org/jira/browse/HIVE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670270#comment-15670270
]
Hive QA commented on HIVE-13557:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839158/HIVE-13557.3.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10710 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2150/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2150/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2150/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839158 - PreCommit-HIVE-Build
> Make interval keyword optional while specifying DAY in interval arithmetic
> --
>
> Key: HIVE-13557
> URL: https://issues.apache.org/jira/browse/HIVE-13557
> Project: Hive
> Issue Type: Sub-task
> Components: Types
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13557.1.patch, HIVE-13557.1.patch,
> HIVE-13557.1.patch, HIVE-13557.2.patch, HIVE-13557.3.patch, HIVE-13557.3.patch
>
>
> Currently we support expressions like: {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) - INTERVAL '30' DAY) AND
> DATE('2000-01-31')
> {code}
> We should support:
> {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) + (-30) DAY) AND
> DATE('2000-01-31')
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13557) Make interval keyword optional while specifying DAY in interval arithmetic
[
https://issues.apache.org/jira/browse/HIVE-13557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-13557:
Attachment: HIVE-13557.3.patch
> Make interval keyword optional while specifying DAY in interval arithmetic
> --
>
> Key: HIVE-13557
> URL: https://issues.apache.org/jira/browse/HIVE-13557
> Project: Hive
> Issue Type: Sub-task
> Components: Types
>Reporter: Ashutosh Chauhan
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13557.1.patch, HIVE-13557.1.patch,
> HIVE-13557.1.patch, HIVE-13557.2.patch, HIVE-13557.3.patch, HIVE-13557.3.patch
>
>
> Currently we support expressions like: {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) - INTERVAL '30' DAY) AND
> DATE('2000-01-31')
> {code}
> We should support:
> {code}
> WHERE SOLD_DATE BETWEEN ((DATE('2000-01-31')) + (-30) DAY) AND
> DATE('2000-01-31')
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14582) Add trunc(numeric) udf
[
https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670136#comment-15670136
]
Hive QA commented on HIVE-14582:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839148/HIVE-14582.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10695 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc_number]
(batchId=10)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hive.spark.client.TestSparkClient.testJobSubmission (batchId=272)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2149/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2149/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2149/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839148 - PreCommit-HIVE-Build
> Add trunc(numeric) udf
> --
>
> Key: HIVE-14582
> URL: https://issues.apache.org/jira/browse/HIVE-14582
> Project: Hive
> Issue Type: Sub-task
> Components: SQL
>Reporter: Ashutosh Chauhan
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-14582.1.patch, HIVE-14582.patch
>
>
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14582) Add trunc(numeric) udf
[ https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14582: Status: Patch Available (was: Open) > Add trunc(numeric) udf > -- > > Key: HIVE-14582 > URL: https://issues.apache.org/jira/browse/HIVE-14582 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Ashutosh Chauhan >Assignee: Chinna Rao Lalam > Attachments: HIVE-14582.1.patch, HIVE-14582.patch > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14582) Add trunc(numeric) udf
[ https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14582: Attachment: HIVE-14582.1.patch Hi [~ashutoshc], Please take a look at patch. Now trunc() will accept date or number type arguments and it will behave as below trunc(date, fmt) / trunc(N,D) - Returns If input is date returns date with the time portion of the day truncated to the unit specified by the format model fmt. If you omit fmt, then date is truncated to "the nearest day. It now only supports 'MONTH'/'MON'/'MM' and 'YEAR'/''/'YY' as format. If input is a number group returns N truncated to D decimal places. If D is omitted, then N is truncated to 0 places. D can be negative to truncate (make zero) D digits left of the decimal point. > Add trunc(numeric) udf > -- > > Key: HIVE-14582 > URL: https://issues.apache.org/jira/browse/HIVE-14582 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Ashutosh Chauhan >Assignee: Chinna Rao Lalam > Attachments: HIVE-14582.1.patch, HIVE-14582.patch > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-14582) Add trunc(numeric) udf
[ https://issues.apache.org/jira/browse/HIVE-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinna Rao Lalam updated HIVE-14582: Attachment: (was: HIVE-14582.1.patch) > Add trunc(numeric) udf > -- > > Key: HIVE-14582 > URL: https://issues.apache.org/jira/browse/HIVE-14582 > Project: Hive > Issue Type: Sub-task > Components: SQL >Reporter: Ashutosh Chauhan >Assignee: Chinna Rao Lalam > Attachments: HIVE-14582.patch > > > https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions200.htm -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15196) LLAP UI: HIVE-14984 broke LLAP UI
[
https://issues.apache.org/jira/browse/HIVE-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barna Zsombor Klara updated HIVE-15196:
---
Attachment: HIVE-15196.patch
Reuploading patch because pre-commit didn' run.
> LLAP UI: HIVE-14984 broke LLAP UI
> -
>
> Key: HIVE-15196
> URL: https://issues.apache.org/jira/browse/HIVE-15196
> Project: Hive
> Issue Type: Bug
> Components: llap, Web UI
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Barna Zsombor Klara
> Attachments: HIVE-15196.patch
>
>
> Hard-rewrite rule to the HS2 UI in HttpServer impl
> {code}
> RewriteRegexRule rootRule = new RewriteRegexRule();
> rootRule.setRegex("^/$");
> rootRule.setReplacement("/hiveserver2.jsp");
> rootRule.setTerminating(true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-15196) LLAP UI: HIVE-14984 broke LLAP UI
[
https://issues.apache.org/jira/browse/HIVE-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15669956#comment-15669956
]
Barna Zsombor Klara edited comment on HIVE-15196 at 11/16/16 9:34 AM:
--
Reuploading patch because pre-commit didn't run.
was (Author: zsombor.klara):
Reuploading patch because pre-commit didn' run.
> LLAP UI: HIVE-14984 broke LLAP UI
> -
>
> Key: HIVE-15196
> URL: https://issues.apache.org/jira/browse/HIVE-15196
> Project: Hive
> Issue Type: Bug
> Components: llap, Web UI
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Barna Zsombor Klara
> Attachments: HIVE-15196.patch
>
>
> Hard-rewrite rule to the HS2 UI in HttpServer impl
> {code}
> RewriteRegexRule rootRule = new RewriteRegexRule();
> rootRule.setRegex("^/$");
> rootRule.setReplacement("/hiveserver2.jsp");
> rootRule.setTerminating(true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-15196) LLAP UI: HIVE-14984 broke LLAP UI
[
https://issues.apache.org/jira/browse/HIVE-15196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barna Zsombor Klara updated HIVE-15196:
---
Attachment: (was: HIVE-15196.patch)
> LLAP UI: HIVE-14984 broke LLAP UI
> -
>
> Key: HIVE-15196
> URL: https://issues.apache.org/jira/browse/HIVE-15196
> Project: Hive
> Issue Type: Bug
> Components: llap, Web UI
>Affects Versions: 2.2.0
>Reporter: Gopal V
>Assignee: Barna Zsombor Klara
>
> Hard-rewrite rule to the HS2 UI in HttpServer impl
> {code}
> RewriteRegexRule rootRule = new RewriteRegexRule();
> rootRule.setRegex("^/$");
> rootRule.setReplacement("/hiveserver2.jsp");
> rootRule.setTerminating(true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-1478) Non-boolean expression in WHERE clause throws exception
[
https://issues.apache.org/jira/browse/HIVE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15669865#comment-15669865
]
Hive QA commented on HIVE-1478:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839133/HIVE-1478.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10695 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_empty_where]
(batchId=21)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=91)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4]
(batchId=91)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2148/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2148/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2148/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12839133 - PreCommit-HIVE-Build
> Non-boolean expression in WHERE clause throws exception
> ---
>
> Key: HIVE-1478
> URL: https://issues.apache.org/jira/browse/HIVE-1478
> Project: Hive
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Paul Yang
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-1478.1.patch
>
>
> If the expression in the where clause does not evaluate to a boolean, the job
> will fail with the following exception in the task logs:
> Query:
> SELECT key FROM src WHERE 1;
> Exception in mapper:
> 2010-07-21 17:00:31,460 FATAL ExecMapper:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"key":"238","value":"val_238"}
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:417)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:180)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Boolean
> at
> org.apache.hadoop.hive.ql.exec.FilterOperator.processOp(FilterOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:45)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:400)
> ... 5 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
