[jira] [Created] (HIVE-11804) Different describe formatted behavior depending on whether the table name is qualified with database name or not
Mark Grover created HIVE-11804:
--
Summary: Different describe formatted behavior depending on
whether the table name is qualified with database name or not
Key: HIVE-11804
URL: https://issues.apache.org/jira/browse/HIVE-11804
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Mark Grover
I have a simple text file based managed table on HDFS:
{quote}
show create table src;
+---+--+
|createtab_stmt
|
+---+--+
| CREATE TABLE `src`(
|
| `first` string,
|
| `word` string)
|
| PARTITIONED BY (
|
| `length` int)
|
| ROW FORMAT SERDE
|
| 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
|
| STORED AS INPUTFORMAT
|
| 'org.apache.hadoop.mapred.TextInputFormat'
|
| OUTPUTFORMAT
|
| 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
|
| LOCATION
|
| 'hdfs://name-node:8020/user/hive/warehouse/my.db/src' |
| TBLPROPERTIES (
|
| 'transient_lastDdlTime'='1441921577')
|
+---+--+
{quote}
The describe formatted with the database name returns:
{quote}
describe formatted my.src first partition(length=1);
+-+---+---+---++-+--+--++-+--+--+
|col_name | data_type | min |
max | num_nulls | distinct_count | avg_col_len | max_col_len | num_trues
| num_falses | comment |
+-+---+---+---++-+--+--++-+--+--+
| # col_name | data_type | comment |
| NULL | NULL| NULL | NULL | NULL |
NULL| NULL |
| | NULL | NULL |
NULL | NULL | NULL| NULL | NULL | NULL
| NULL| NULL |
| first | string| from deserializer |
NULL | NULL | NULL| NULL | NULL | NULL
| NULL| NULL |
+-+---+---+---++-+--+--++-+--+--+
{quote}
while without it returns:
{quote}
describe formatted src first partition(length=1);
+---+---+---+--+
| col_name| data_type
|comment|
+---+---+---+--+
| # col_name| data_type
| comment |
| | NULL
| NULL |
| first | string
| |
| word | string
| |
| | NULL
| NULL |
| # Partition Information | NULL
| NULL |
| # col_name| data_type
| comment |
| | NULL
[jira] [Created] (HIVE-10041) Set defaults for HBASE_HOME in a smarter way
Mark Grover created HIVE-10041: -- Summary: Set defaults for HBASE_HOME in a smarter way Key: HIVE-10041 URL: https://issues.apache.org/jira/browse/HIVE-10041 Project: Hive Issue Type: Bug Components: HCatalog Affects Versions: 1.1.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 1.1.0 Similar to SQOOP-2145, hcat binary script doesn't do smart detection of HBASE_HOME. It assumes it's always located in /usr/lib/hbase and if not HBASE_HOME variable needs to be exported. The reason is often times people have tarballs like ~/hive-hcatalog, ~/hbase, etc. and it would be good to have their hcat scripts work out of the book. This doesn't regress anything because it only sets the HBASE_HOME if it's not already set and the directory it's setting it to exists and is valid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9818) Print error messages on stderr instead of stdout
Mark Grover created HIVE-9818:
-
Summary: Print error messages on stderr instead of stdout
Key: HIVE-9818
URL: https://issues.apache.org/jira/browse/HIVE-9818
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 1.0.0
Reporter: Mark Grover
hcat script (see
[here|https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L103], for
example) prints error messages via echo which go on stdout the way they are
written. This is bad because downstream projects using it receive error
messages as expected output instead of actually receiving nothing (see
SQOOP-2147 as example).
We need to put error messages on stdout. Something like:
{code}
2 echo This is an error message
{code}
instead of a simple echo that we have been using so far.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-9000) LAST_VALUE Window function returns wrong results
Mark Grover created HIVE-9000:
-
Summary: LAST_VALUE Window function returns wrong results
Key: HIVE-9000
URL: https://issues.apache.org/jira/browse/HIVE-9000
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.13.1
Reporter: Mark Grover
Priority: Critical
Fix For: 0.14.1
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the wrong
result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s)
{code}
The result is:
{code}
t si last_value(i)
---
10 oscar allen 65662 65662
10 oscar carson65549 65549
{code}
LAST_VALUE(i) should have returned 65549 in both records, instead it simply
ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last value
should always be more (in terms of the specified 'order by s') than the lead by
1. While this doesn't directly apply to the above query, if the result set had
more rows, you would clearly see records where lead is higher than last_value
which is semantically incorrect.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9000) LAST_VALUE Window function returns wrong results
[
https://issues.apache.org/jira/browse/HIVE-9000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-9000:
--
Description:
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the wrong
result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s)
{code}
The result is:
{code}
t si last_value(i)
---
10 oscar allen 65662 65662
10 oscar carson65549 65549
{code}
{{LAST_VALUE( i )}} should have returned 65549 in both records, instead it
simply ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last value
should always be more (in terms of the specified 'order by s') than the lead by
1. While this doesn't directly apply to the above query, if the result set had
more rows, you would clearly see records where lead is higher than last_value
which is semantically incorrect.
was:
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the wrong
result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s)
{code}
The result is:
{code}
t si last_value(i)
---
10 oscar allen 65662 65662
10 oscar carson65549 65549
{code}
LAST_VALUE(i) should have returned 65549 in both records, instead it simply
ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last value
should always be more (in terms of the specified 'order by s') than the lead by
1. While this doesn't directly apply to the above query, if the result set had
more rows, you would clearly see records where lead is higher than last_value
which is semantically incorrect.
LAST_VALUE Window function returns wrong results
Key: HIVE-9000
URL: https://issues.apache.org/jira/browse/HIVE-9000
Project: Hive
Issue Type: Bug
Components: PTF-Windowing
Affects Versions: 0.13.1
Reporter: Mark Grover
Priority: Critical
Fix For: 0.14.1
LAST_VALUE Windowing function has been returning bad results, as far as I can
tell from day 1.
And, it seems like the tests are also asserting that LAST_VALUE gives the
wrong result.
Here's the test output:
https://github.com/apache/hive/blob/branch-0.14/ql/src/test/results/clientpositive/windowing_navfn.q.out#L587
The query is:
{code}
select t, s, i, last_value(i) over (partition by t order by s)
{code}
The result is:
{code}
t si last_value(i)
---
10oscar allen 65662 65662
10oscar carson65549 65549
{code}
{{LAST_VALUE( i )}} should have returned 65549 in both records, instead it
simply ends up returning i.
Another way you can make sure LAST_VALUE is bad is to verify it's result
against LEAD(i,1) over (partition by t order by s). LAST_VALUE being last
value should always be more (in terms of the specified 'order by s') than the
lead by 1. While this doesn't directly apply to the above query, if the
result set had more rows, you would clearly see records where lead is higher
than last_value which is semantically incorrect.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-6593) Create a maven assembly for hive-jdbc
[ https://issues.apache.org/jira/browse/HIVE-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924974#comment-13924974 ] Mark Grover commented on HIVE-6593: --- Thanks Szehon for taking this up and everyone for their input. Cos or I can review the patch. Create a maven assembly for hive-jdbc - Key: HIVE-6593 URL: https://issues.apache.org/jira/browse/HIVE-6593 Project: Hive Issue Type: Improvement Components: Build Infrastructure Affects Versions: 0.12.0 Reporter: Mark Grover Assignee: Szehon Ho Attachments: HIVE-6593.patch Currently in Apache Bigtop we bundle and distribute Hive. In particular, for users to not have to install the entirety of Hive on machines that are just jdbc clients, we have a special package which is a subset of hive, called hive-jdbc that bundles only the jdbc driver jar and it's dependencies. However, because Hive doesn't have an assembly for the jdbc jar, we have to hack and hardcode the list of jdbc jars and it's dependencies: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hive/SPECS/hive.spec#L361 As Hive moves to Maven, it would be pretty fantastic if Hive could leverage the maven-assembly-plugin and generate a .tar.gz assembly for what's required for jdbc gateway machines. That we can simply take that distribution and build a jdbc package from it without having to hard code jar names and dependencies. That would make the process much less error prone. NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6593) Create a maven assembly for hive-jdbc
Mark Grover created HIVE-6593: - Summary: Create a maven assembly for hive-jdbc Key: HIVE-6593 URL: https://issues.apache.org/jira/browse/HIVE-6593 Project: Hive Issue Type: Improvement Components: Build Infrastructure Affects Versions: 0.12.0 Reporter: Mark Grover Currently in Apache Bigtop we bundle and distribute Hive. In particular, for users to not have to install the entirety of Hive on machines that are just jdbc clients, we have a special package which is a subset of hive, called hive-jdbc that bundles only the jdbc driver jar and it's dependencies. However, because Hive doesn't have an assembly for the jdbc jar, we have to hack and hardcode the list of jdbc jars and it's dependencies: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hive/SPECS/hive.spec#L361 As Hive moves to Maven, it would be pretty fantastic if Hive could leverage the maven-assembly-plugin and generate a .tar.gz assembly for what's required for jdbc gateway machines. That we can simply take that distribution and build a jdbc package from it without having to hard code jar names and dependencies. That would make the process much less error prone. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-3819) Creating a table on Hive without Hadoop daemons running returns a misleading error
[
https://issues.apache.org/jira/browse/HIVE-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13817348#comment-13817348
]
Mark Grover commented on HIVE-3819:
---
Sorry, Xuefu, I haven't had the time. If you can't reproduce this, please go
ahead and mark this as Cant' reproduce.
Thanks for checking!
Creating a table on Hive without Hadoop daemons running returns a misleading
error
--
Key: HIVE-3819
URL: https://issues.apache.org/jira/browse/HIVE-3819
Project: Hive
Issue Type: Bug
Components: CLI, Metastore
Reporter: Mark Grover
Assignee: Xuefu Zhang
I was running hive without running the underlying hadoop daemon's running.
Hadoop was configured to run in pseudo-distributed mode. However, when I
tried to create a hive table, I got this rather misleading error:
{code}
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
We should look into making this error message less misleading (more about
hadoop daemons not running instead of metastore client not being
instantiable).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Assigned] (HIVE-3844) Unix timestamps don't seem to be read correctly from HDFS as Timestamp column
[
https://issues.apache.org/jira/browse/HIVE-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover reassigned HIVE-3844:
-
Assignee: Venki Korukanti (was: Mark Grover)
Venki,
I am not. Assigned it to you.
Unix timestamps don't seem to be read correctly from HDFS as Timestamp column
-
Key: HIVE-3844
URL: https://issues.apache.org/jira/browse/HIVE-3844
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.8.0
Reporter: Mark Grover
Assignee: Venki Korukanti
Serega Shepak pointed out that something like
{code}
select cast(date_occurrence as timestamp) from xvlr_data limit 10
{code}
where date_occurrence has BIGINT type (timestamp in milliseconds) works. But
it doesn't work if the declared type is TIMESTAMP on column. The data in the
date_occurence column in unix timestamp in millis.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-3819) Creating a table on Hive without Hadoop daemons running returns a misleading error
[
https://issues.apache.org/jira/browse/HIVE-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13800063#comment-13800063
]
Mark Grover commented on HIVE-3819:
---
Let merry to reproduce it.
Creating a table on Hive without Hadoop daemons running returns a misleading
error
--
Key: HIVE-3819
URL: https://issues.apache.org/jira/browse/HIVE-3819
Project: Hive
Issue Type: Bug
Components: CLI, Metastore
Reporter: Mark Grover
Assignee: Xuefu Zhang
I was running hive without running the underlying hadoop daemon's running.
Hadoop was configured to run in pseudo-distributed mode. However, when I
tried to create a hive table, I got this rather misleading error:
{code}
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
We should look into making this error message less misleading (more about
hadoop daemons not running instead of metastore client not being
instantiable).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-3819) Creating a table on Hive without Hadoop daemons running returns a misleading error
[
https://issues.apache.org/jira/browse/HIVE-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13800064#comment-13800064
]
Mark Grover commented on HIVE-3819:
---
Stupid autocorrect! I meant to say let me try to reproduce it:-)
Creating a table on Hive without Hadoop daemons running returns a misleading
error
--
Key: HIVE-3819
URL: https://issues.apache.org/jira/browse/HIVE-3819
Project: Hive
Issue Type: Bug
Components: CLI, Metastore
Reporter: Mark Grover
Assignee: Xuefu Zhang
I was running hive without running the underlying hadoop daemon's running.
Hadoop was configured to run in pseudo-distributed mode. However, when I
tried to create a hive table, I got this rather misleading error:
{code}
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
{code}
We should look into making this error message less misleading (more about
hadoop daemons not running instead of metastore client not being
instantiable).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (HIVE-5534) Webhcatalog shell scripts have non-executable permissions
Mark Grover created HIVE-5534:
-
Summary: Webhcatalog shell scripts have non-executable permissions
Key: HIVE-5534
URL: https://issues.apache.org/jira/browse/HIVE-5534
Project: Hive
Issue Type: Bug
Components: WebHCat
Affects Versions: 0.11.0, 0.12.0
Reporter: Mark Grover
Fix For: 0.12.0
While playing with the hive 0.12 rc1, I noticed that the script for starting
and stopping webhcatalog server don't have the correct permissions.
{code}
localhost:hive-0.12.0 mgrover$ find . -name *webhcat*.sh | grep -v './src/' |
xargs ls -lrt
-rw-r--r-- 1 mgrover staff 7141 Oct 9 18:50
./hcatalog/sbin/webhcat_server.sh
-rw-r--r-- 1 mgrover staff 4243 Oct 9 18:50
./hcatalog/sbin/webhcat_config.sh
{code}
This means that we can't start or stop webhcat server using the tarball out of
the box.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (HIVE-5534) HCatalog shell scripts have non-executable permissions
[
https://issues.apache.org/jira/browse/HIVE-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-5534:
--
Priority: Critical (was: Major)
Summary: HCatalog shell scripts have non-executable permissions (was:
Webhcatalog shell scripts have non-executable permissions)
HCatalog shell scripts have non-executable permissions
--
Key: HIVE-5534
URL: https://issues.apache.org/jira/browse/HIVE-5534
Project: Hive
Issue Type: Bug
Components: WebHCat
Affects Versions: 0.11.0, 0.12.0
Reporter: Mark Grover
Priority: Critical
Fix For: 0.12.0
While playing with the hive 0.12 rc1, I noticed that the script for starting
and stopping webhcatalog server don't have the correct permissions.
{code}
localhost:hive-0.12.0 mgrover$ find . -name *webhcat*.sh | grep -v './src/' |
xargs ls -lrt
-rw-r--r-- 1 mgrover staff 7141 Oct 9 18:50
./hcatalog/sbin/webhcat_server.sh
-rw-r--r-- 1 mgrover staff 4243 Oct 9 18:50
./hcatalog/sbin/webhcat_config.sh
{code}
This means that we can't start or stop webhcat server using the tarball out
of the box.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5534) Webhcatalog shell scripts have non-executable permissions
[
https://issues.apache.org/jira/browse/HIVE-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13794386#comment-13794386
]
Mark Grover commented on HIVE-5534:
---
Poking more seems like this a problem with all hcatalog scripts so starting
hcatalog server is affected as well:
{code}
-rw-r--r-- 1 mgrover staff 2048 Oct 9 18:50
./hcatalog/share/hcatalog/scripts/hcat_server_stop.sh
-rw-r--r-- 1 mgrover staff 3003 Oct 9 18:50
./hcatalog/share/hcatalog/scripts/hcat_server_start.sh
-rw-r--r-- 1 mgrover staff 4698 Oct 9 18:50
./hcatalog/share/hcatalog/scripts/hcat_server_install.sh
-rw-r--r-- 1 mgrover staff 7141 Oct 9 18:50
./hcatalog/sbin/webhcat_server.sh
-rw-r--r-- 1 mgrover staff 4243 Oct 9 18:50
./hcatalog/sbin/webhcat_config.sh
-rw-r--r-- 1 mgrover staff 10013 Oct 9 18:50
./hcatalog/sbin/update-hcatalog-env.sh
-rw-r--r-- 1 mgrover staff 4540 Oct 9 18:50 ./hcatalog/sbin/hcat_server.sh
-rw-r--r-- 1 mgrover staff 2543 Oct 9 18:50
./hcatalog/libexec/hcat-config.sh
{code}
I update the title of the JIRA to reflect the above.
Webhcatalog shell scripts have non-executable permissions
-
Key: HIVE-5534
URL: https://issues.apache.org/jira/browse/HIVE-5534
Project: Hive
Issue Type: Bug
Components: WebHCat
Affects Versions: 0.11.0, 0.12.0
Reporter: Mark Grover
Fix For: 0.12.0
While playing with the hive 0.12 rc1, I noticed that the script for starting
and stopping webhcatalog server don't have the correct permissions.
{code}
localhost:hive-0.12.0 mgrover$ find . -name *webhcat*.sh | grep -v './src/' |
xargs ls -lrt
-rw-r--r-- 1 mgrover staff 7141 Oct 9 18:50
./hcatalog/sbin/webhcat_server.sh
-rw-r--r-- 1 mgrover staff 4243 Oct 9 18:50
./hcatalog/sbin/webhcat_config.sh
{code}
This means that we can't start or stop webhcat server using the tarball out
of the box.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-3976) Support specifying scale and precision with Hive decimal type
[
https://issues.apache.org/jira/browse/HIVE-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765119#comment-13765119
]
Mark Grover commented on HIVE-3976:
---
Hi Xuefu,
Thanks for posting the document. Overall, looks good! It's great that you are
working on this.
One minor question, in the document you mention Specifically, for +/-, the
result scale is s1 + s2, and for multiplication, max(s1, s2). Did you mean the
other way around (max for +/-, s1+s2 for multiplication)?
In the error handling case, you mention rounding instead of using null. That
definitely seems like a plausible idea. However, I think it may be a slippery
slope. For example, we may be using a rounding policy here by default. This
policy may not work for all users and then we may have to expose
configuration/table specific options to specify such rounding policies. There
might be value in masking the values that don't conform as nulls and putting
the onus on the users to explicitly round them to conform to the column type.
However, if mysql or another popular database has set a precedent, what you are
suggesting may be ok.
Big +1 on changing arithmetic operation UDFs to GenericUDFs like you and Jason
already suggested.
Support specifying scale and precision with Hive decimal type
-
Key: HIVE-3976
URL: https://issues.apache.org/jira/browse/HIVE-3976
Project: Hive
Issue Type: Improvement
Components: Query Processor, Types
Reporter: Mark Grover
Assignee: Xuefu Zhang
Attachments: remove_prec_scale.diff
HIVE-2693 introduced support for Decimal datatype in Hive. However, the
current implementation has unlimited precision and provides no way to specify
precision and scale when creating the table.
For example, MySQL allows users to specify scale and precision of the decimal
datatype when creating the table:
{code}
CREATE TABLE numbers (a DECIMAL(20,2));
{code}
Hive should support something similar too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762017#comment-13762017 ] Mark Grover commented on HIVE-4003: --- [~appodictic] or [~brocknoland] would one of you mind committing this? NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch, HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4003) NullPointerException in exec.Utilities
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762057#comment-13762057 ] Mark Grover commented on HIVE-4003: --- Thank you! NullPointerException in exec.Utilities -- Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Priority: Blocker Fix For: 0.12.0, 0.13.0 Attachments: HIVE-4003.patch, HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13738355#comment-13738355 ] Mark Grover commented on HIVE-4003: --- Thanks, Brock, for taking a look. I will rebase this. NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-4003: -- Status: Patch Available (was: Open) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch, HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-4003: -- Attachment: HIVE-4003.patch I am guessing I have to keep the name of the patch same, so tests can be run. Correcting the name to be HIVE-4003.patch now. NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch, HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-4003: -- Attachment: (was: HIVE-4003.2.patch) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch, HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-4003: -- Attachment: HIVE-4003.2.patch Uploaded a new rebased patch. NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Attachments: HIVE-4003.patch, HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5046) Hcatalog's bin/hcat script doesn't respect HIVE_HOME
[
https://issues.apache.org/jira/browse/HIVE-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13737008#comment-13737008
]
Mark Grover commented on HIVE-5046:
---
Thanks Brock!
Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Key: HIVE-5046
URL: https://issues.apache.org/jira/browse/HIVE-5046
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.11.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.12.0
Attachments: HIVE-5046.1.patch
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the above code ends
up setting HIVE_HOME to the default value if it is actually set already,
overriding the set value. This condition needs to be negated.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4388) HBase tests fail against Hadoop 2
[
https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13737106#comment-13737106
]
Mark Grover commented on HIVE-4388:
---
Brock, thanks for looking into this.
I was reviewing the patch and saw that you have several references to
{{getFamilyMap()}}.
This method's return type was changed in newer version of HBase. Even though
HBASE-9142 introduces the original method back in 0.95.2, it's deprecated.
Do you think it makes more sense to use {{getFamilyCellMap()}} here instead?
HBase tests fail against Hadoop 2
-
Key: HIVE-4388
URL: https://issues.apache.org/jira/browse/HIVE-4388
Project: Hive
Issue Type: Bug
Components: HBase Handler
Reporter: Gunther Hagleitner
Assignee: Brock Noland
Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch,
HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388-wip.txt
Currently we're building by default against 0.92. When you run against hadoop
2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963.
HIVE-3861 upgrades the version of hbase used. This will get you past the
problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-5046) Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Mark Grover created HIVE-5046:
-
Summary: Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Key: HIVE-5046
URL: https://issues.apache.org/jira/browse/HIVE-5046
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.11.0
Reporter: Mark Grover
Assignee: Mark Grover
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the condition is
checking for the boolean inverse of what it's supposed to be checking for.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5046) Hcatalog's bin/hcat script doesn't respect HIVE_HOME
[
https://issues.apache.org/jira/browse/HIVE-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-5046:
--
Description:
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the above code ends
up setting HIVE_HOME to the default value if it is actually set already,
overriding the set value. This condition needs to be negated.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
was:
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the condition is
checking for the boolean inverse of what it's supposed to be checking for.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Key: HIVE-5046
URL: https://issues.apache.org/jira/browse/HIVE-5046
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.11.0
Reporter: Mark Grover
Assignee: Mark Grover
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the above code ends
up setting HIVE_HOME to the default value if it is actually set already,
overriding the set value. This condition needs to be negated.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5046) Hcatalog's bin/hcat script doesn't respect HIVE_HOME
[
https://issues.apache.org/jira/browse/HIVE-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-5046:
--
Attachment: HIVE-5046.1.patch
Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Key: HIVE-5046
URL: https://issues.apache.org/jira/browse/HIVE-5046
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.11.0
Reporter: Mark Grover
Assignee: Mark Grover
Attachments: HIVE-5046.1.patch
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the above code ends
up setting HIVE_HOME to the default value if it is actually set already,
overriding the set value. This condition needs to be negated.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5046) Hcatalog's bin/hcat script doesn't respect HIVE_HOME
[
https://issues.apache.org/jira/browse/HIVE-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-5046:
--
Status: Patch Available (was: Open)
Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Key: HIVE-5046
URL: https://issues.apache.org/jira/browse/HIVE-5046
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.11.0
Reporter: Mark Grover
Assignee: Mark Grover
Attachments: HIVE-5046.1.patch
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the above code ends
up setting HIVE_HOME to the default value if it is actually set already,
overriding the set value. This condition needs to be negated.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5046) Hcatalog's bin/hcat script doesn't respect HIVE_HOME
[
https://issues.apache.org/jira/browse/HIVE-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735566#comment-13735566
]
Mark Grover commented on HIVE-5046:
---
Eugene, thanks for posting. I don't think so. The intent of the snippet you
mentioned is to do something with the variable ({{HIVE_IN_PATH}}) if it is
defined. That something is to make use to {{HIVE_IN_PATH}} to popualate
{{HIVE_DIR}}.
On the other hand, the intent of the snippet 12 lines later (that this JIRA
addresses) is to do something if the variable is not defined. That something is
to set HIVE_HOME to some default value. This snippet however, checks if the
variable is not null instead of checking if the variable is null.
Hcatalog's bin/hcat script doesn't respect HIVE_HOME
Key: HIVE-5046
URL: https://issues.apache.org/jira/browse/HIVE-5046
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.11.0
Reporter: Mark Grover
Assignee: Mark Grover
Attachments: HIVE-5046.1.patch
https://github.com/apache/hive/blob/trunk/hcatalog/bin/hcat#L81
The quoted snippet (see below) intends to set HIVE_HOME if it's not set (i.e.
HIVE_HOME is currently null).
{code}
if [ -n ${HIVE_HOME} ]; then
{code}
However, {{-n}} checks if the variable is _not_ null. So, the above code ends
up setting HIVE_HOME to the default value if it is actually set already,
overriding the set value. This condition needs to be negated.
Moreover, {{-n}} checks requires the string being tested to be enclosed in
quotes.
Reference:
http://tldp.org/LDP/abs/html/comparison-ops.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4403) Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters
[
https://issues.apache.org/jira/browse/HIVE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13674027#comment-13674027
]
Mark Grover commented on HIVE-4403:
---
Chu, that's fine. Nothing required from your side.
Running Hive queries on Yarn (MR2) gives warnings related to overriding final
parameters
Key: HIVE-4403
URL: https://issues.apache.org/jira/browse/HIVE-4403
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.11.0
Reporter: Mark Grover
Assignee: Chu Tong
Fix For: 0.12.0
Attachments: HIVE-4403.patch, HIVE-4403.patch
While working on BIGTOP-885, I saw that Hive was giving a bunch of warnings
related to overriding final parameters in job.conf. This was on a pseudo
distributed cluster. FWIW, I didn't see this happen on a fully-distributed
cluster. Perhaps, Hive's job.conf is overriding some final parameters it
shouldn't.
Here is what the warnings looked like:
{code}
2013-04-19 14:20:32,304 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2013-04-19 14:20:32,367 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
{code}
To reproduce, run a query like:
{code}
CREATE TABLE u_data (
userid INT,
movieid INT,
rating INT,
unixtime STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;
{code}
Load some data into u_data, here is some sample data:
https://github.com/apache/bigtop/blob/master/bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/ml-data/u.data
Run a simple query on that data (on YARN/MR2)
{code}
INSERT OVERWRITE DIRECTORY '/tmp/count'
SELECT COUNT(1) FROM u_data
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4403) Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters
[
https://issues.apache.org/jira/browse/HIVE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672703#comment-13672703
]
Mark Grover commented on HIVE-4403:
---
Ashutosh +1'ed it. So, I guess there is no need for Phabricator now:-)
Thanks Chu and Ashutosh!
Running Hive queries on Yarn (MR2) gives warnings related to overriding final
parameters
Key: HIVE-4403
URL: https://issues.apache.org/jira/browse/HIVE-4403
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Chu Tong
Attachments: HIVE-4403.patch, HIVE-4403.patch
While working on BIGTOP-885, I saw that Hive was giving a bunch of warnings
related to overriding final parameters in job.conf. This was on a pseudo
distributed cluster. FWIW, I didn't see this happen on a fully-distributed
cluster. Perhaps, Hive's job.conf is overriding some final parameters it
shouldn't.
Here is what the warnings looked like:
{code}
2013-04-19 14:20:32,304 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2013-04-19 14:20:32,367 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
{code}
To reproduce, run a query like:
{code}
CREATE TABLE u_data (
userid INT,
movieid INT,
rating INT,
unixtime STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;
{code}
Load some data into u_data, here is some sample data:
https://github.com/apache/bigtop/blob/master/bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/ml-data/u.data
Run a simple query on that data (on YARN/MR2)
{code}
INSERT OVERWRITE DIRECTORY '/tmp/count'
SELECT COUNT(1) FROM u_data
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4403) Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters
[
https://issues.apache.org/jira/browse/HIVE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669521#comment-13669521
]
Mark Grover commented on HIVE-4403:
---
Chu, sorry about the delay. I can test patched Hive to see if this resolves my
original problem, please give me a few days.
In the meanwhile, even though this is a small patch, do you mind uploading it
on Reviewboard or Phabricator? Thanks for contributing!
Running Hive queries on Yarn (MR2) gives warnings related to overriding final
parameters
Key: HIVE-4403
URL: https://issues.apache.org/jira/browse/HIVE-4403
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Attachments: HIVE-4403.patch
While working on BIGTOP-885, I saw that Hive was giving a bunch of warnings
related to overriding final parameters in job.conf. This was on a pseudo
distributed cluster. FWIW, I didn't see this happen on a fully-distributed
cluster. Perhaps, Hive's job.conf is overriding some final parameters it
shouldn't.
Here is what the warnings looked like:
{code}
2013-04-19 14:20:32,304 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2013-04-19 14:20:32,367 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
{code}
To reproduce, run a query like:
{code}
CREATE TABLE u_data (
userid INT,
movieid INT,
rating INT,
unixtime STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;
{code}
Load some data into u_data, here is some sample data:
https://github.com/apache/bigtop/blob/master/bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/ml-data/u.data
Run a simple query on that data (on YARN/MR2)
{code}
INSERT OVERWRITE DIRECTORY '/tmp/count'
SELECT COUNT(1) FROM u_data
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4070) Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive
[ https://issues.apache.org/jira/browse/HIVE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13666785#comment-13666785 ] Mark Grover commented on HIVE-4070: --- I agree with Edward that having a property like that is risky. I also like John's idea of having two different like UDFs. We can only change the behavior (if at all) in a major release. I think in this case, we are just going to have to pick one kind of like and implement another one. Yeah, people coming from MySQL land may get confused but that's the best we can do is document it and ask them to use the other like. John, it would be great if you could create a JIRA for the case-insensitive like. Thanks! Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive Key: HIVE-4070 URL: https://issues.apache.org/jira/browse/HIVE-4070 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Trivial Hive's like operator seems to be case sensitive. See https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLike.java#L164 However, MySQL's like operator is case insensitive. I don't have other DB's (like PostgreSQL) installed and handy but I am guessing their LIKE is case insensitive as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649880#comment-13649880 ] Mark Grover commented on HIVE-3384: --- I verified that this was committed on trunk (Thanks!). Is there anything left to be done regarding this patch? If not, can we please update the JIRA to resolved status? HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification -- Key: HIVE-3384 URL: https://issues.apache.org/jira/browse/HIVE-3384 Project: Hive Issue Type: Bug Components: JDBC Affects Versions: 0.10.0 Reporter: Weidong Bian Assignee: Chris Drome Priority: Minor Fix For: 0.11.0 Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, HIVE-3384-0.10.patch, HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch jdbc module couldn't be compiled with jdk7 as it adds some abstract method in the JDBC specification some error info: error: HiveCallableStatement is not abstract and does not override abstract method TgetObject(String,ClassT) in CallableStatement . . . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4403) Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters
Mark Grover created HIVE-4403:
-
Summary: Running Hive queries on Yarn (MR2) gives warnings related
to overriding final parameters
Key: HIVE-4403
URL: https://issues.apache.org/jira/browse/HIVE-4403
Project: Hive
Issue Type: Bug
Reporter: Mark Grover
While working on BIGTOP-885, I saw that Hive was giving a bunch of warnings
related to overriding final parameters in job.conf. Perhaps, Hive's job.conf is
overriding some final parameters it shouldn't.
Here is what the warnings looked like:
{code}
2013-04-19 14:20:32,304 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2013-04-19 14:20:32,367 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
{code}
To reproduce, run a query like:
{code}
CREATE TABLE u_data (
userid INT,
movieid INT,
rating INT,
unixtime STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;
{code}
Load some data into u_data, here is some sample data:
https://github.com/apache/bigtop/blob/master/bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/ml-data/u.data
Run a simple query on that data (on YARN/MR2)
{code}
INSERT OVERWRITE DIRECTORY '/tmp/count'
SELECT COUNT(1) FROM u_data
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4403) Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters
[
https://issues.apache.org/jira/browse/HIVE-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-4403:
--
Affects Version/s: 0.10.0
Running Hive queries on Yarn (MR2) gives warnings related to overriding final
parameters
Key: HIVE-4403
URL: https://issues.apache.org/jira/browse/HIVE-4403
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
While working on BIGTOP-885, I saw that Hive was giving a bunch of warnings
related to overriding final parameters in job.conf. Perhaps, Hive's job.conf
is overriding some final parameters it shouldn't.
Here is what the warnings looked like:
{code}
2013-04-19 14:20:32,304 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
2013-04-19 14:20:32,367 WARN [main] conf.Configuration
(Configuration.java:loadProperty(2032)) -
file:/tmp/root/hive_2013-04-19_14-20-30_159_5701876916688815815/-local-10002/jobconf.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
{code}
To reproduce, run a query like:
{code}
CREATE TABLE u_data (
userid INT,
movieid INT,
rating INT,
unixtime STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE;
{code}
Load some data into u_data, here is some sample data:
https://github.com/apache/bigtop/blob/master/bigtop-tests/test-artifacts/hive/src/main/resources/seed_data_files/ml-data/u.data
Run a simple query on that data (on YARN/MR2)
{code}
INSERT OVERWRITE DIRECTORY '/tmp/count'
SELECT COUNT(1) FROM u_data
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4362) Allow Hive unit tests to run against fully-distributed cluster
Mark Grover created HIVE-4362: - Summary: Allow Hive unit tests to run against fully-distributed cluster Key: HIVE-4362 URL: https://issues.apache.org/jira/browse/HIVE-4362 Project: Hive Issue Type: Improvement Components: Testing Infrastructure Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 It seems like Hive unit tests can run in (Hadoop) local mode or miniMR mode. It would be nice (especially for projects like Apache Bigtop) to be able to run Hive tests in fully distributed mode. This JIRA tracks the introduction of such functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4174) Round UDF converts BigInts to double
[
https://issues.apache.org/jira/browse/HIVE-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621069#comment-13621069
]
Mark Grover commented on HIVE-4174:
---
Indeed, thanks Chen Chun. And, congratulations on your first Hive commit!
Round UDF converts BigInts to double
Key: HIVE-4174
URL: https://issues.apache.org/jira/browse/HIVE-4174
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: hive.4174.1.patch-nohcat, HIVE-4174.1.patch.txt,
HIVE-4174.D9687.1.patch
Chen Chun pointed out on the hive-user mailing list that round() in Hive 0.10
returns
{code}
select round(cast(1234560 as BIGINT)), round(cast(12345670 as BIGINT)) from
test limit 1;
//hive 0.10
1234560.0 1.234567E7
{code}
This is not consistent with
MySQL(http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html#function_round)
which quotes
{code}
The return type is the same type as that of the first argument (assuming that
it is integer, double, or decimal). This means that for an integer argument,
the result is an integer (no decimal places)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4174) Round UDF converts BigInts to double
[
https://issues.apache.org/jira/browse/HIVE-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621137#comment-13621137
]
Mark Grover commented on HIVE-4174:
---
It should be. [~namitjain] can you please add Chen Chun (chenchun dot feed at
gmail dot com) to the project contributors list and assign this JIRA to him?
Thanks!
Round UDF converts BigInts to double
Key: HIVE-4174
URL: https://issues.apache.org/jira/browse/HIVE-4174
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: hive.4174.1.patch-nohcat, HIVE-4174.1.patch.txt,
HIVE-4174.D9687.1.patch
Chen Chun pointed out on the hive-user mailing list that round() in Hive 0.10
returns
{code}
select round(cast(1234560 as BIGINT)), round(cast(12345670 as BIGINT)) from
test limit 1;
//hive 0.10
1234560.0 1.234567E7
{code}
This is not consistent with
MySQL(http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html#function_round)
which quotes
{code}
The return type is the same type as that of the first argument (assuming that
it is integer, double, or decimal). This means that for an integer argument,
the result is an integer (no decimal places)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4213) List bucketing error too restrictive
[
https://issues.apache.org/jira/browse/HIVE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621158#comment-13621158
]
Mark Grover commented on HIVE-4213:
---
Hi [~gangtimliu], really sorry for dropping the ball on this. I do see what you
mean by the legal cases but is it legal to do this:
{code}
set hive.mapred.supports.subdirectories=false;
set mapred.input.dir.recursive=true;
set hive.optimize.listbucketing=false;
{code}
List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Gang Tim Liu
Fix For: 0.11.0
With the introduction of List bucketing, we introduced a config validation
step where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3951) Allow Decimal type columns in Regex Serde
[ https://issues.apache.org/jira/browse/HIVE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3951: -- Attachment: HIVE-3951.2.patch Thanks for the comments, Ashutosh. Uploaded a new patch. Will update Review Board momentarily. Allow Decimal type columns in Regex Serde - Key: HIVE-3951 URL: https://issues.apache.org/jira/browse/HIVE-3951 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3951.1.patch, HIVE-3951.2.patch Decimal type in Hive was recently added by HIVE-2693. We should allow users to create tables with decimal type columns when using Regex Serde. HIVE-3004 did something similar for other primitive types. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3951) Allow Decimal type columns in Regex Serde
[ https://issues.apache.org/jira/browse/HIVE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618821#comment-13618821 ] Mark Grover commented on HIVE-3951: --- RB updated Allow Decimal type columns in Regex Serde - Key: HIVE-3951 URL: https://issues.apache.org/jira/browse/HIVE-3951 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3951.1.patch, HIVE-3951.2.patch Decimal type in Hive was recently added by HIVE-2693. We should allow users to create tables with decimal type columns when using Regex Serde. HIVE-3004 did something similar for other primitive types. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3951) Allow Decimal type columns in Regex Serde
[ https://issues.apache.org/jira/browse/HIVE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619518#comment-13619518 ] Mark Grover commented on HIVE-3951: --- Thanks, Ashutosh! Allow Decimal type columns in Regex Serde - Key: HIVE-3951 URL: https://issues.apache.org/jira/browse/HIVE-3951 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3951.1.patch, HIVE-3951.2.patch Decimal type in Hive was recently added by HIVE-2693. We should allow users to create tables with decimal type columns when using Regex Serde. HIVE-3004 did something similar for other primitive types. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories
[
https://issues.apache.org/jira/browse/HIVE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover reassigned HIVE-3451:
-
Assignee: Mark Grover (was: Gang Tim Liu)
map-reduce jobs does not work for a partition containing sub-directories
Key: HIVE-3451
URL: https://issues.apache.org/jira/browse/HIVE-3451
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Mark Grover
Fix For: 0.10.0
Attachments: HIVE-3451.patch
Consider the following test:
-- The test verifies that sub-directories are supported for versions of hadoop
-- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
-- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set mapred.input.dir.recursive=true;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
Say, the above file was named: recursive_dir.q
and we ran the test for hadoop 23:
by executing:
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true
-Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
-Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
The select * from the table works fine, but the last command does not work
since it requires a map-reduce job.
This will prevent other features which are creating sub-directories to add
any tests which requires a map-reduce job. The work-around is to issue
queries which do not require map-reduce jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories
[
https://issues.apache.org/jira/browse/HIVE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover reassigned HIVE-3451:
-
Assignee: Gang Tim Liu (was: Mark Grover)
map-reduce jobs does not work for a partition containing sub-directories
Key: HIVE-3451
URL: https://issues.apache.org/jira/browse/HIVE-3451
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Gang Tim Liu
Fix For: 0.10.0
Attachments: HIVE-3451.patch
Consider the following test:
-- The test verifies that sub-directories are supported for versions of hadoop
-- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
-- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set mapred.input.dir.recursive=true;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
Say, the above file was named: recursive_dir.q
and we ran the test for hadoop 23:
by executing:
ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true
-Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha
-Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
The select * from the table works fine, but the last command does not work
since it requires a map-reduce job.
This will prevent other features which are creating sub-directories to add
any tests which requires a map-reduce job. The work-around is to issue
queries which do not require map-reduce jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4213) List bucketing error too restrictive
Mark Grover created HIVE-4213:
-
Summary: List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Fix For: 0.11.0
With the introduction of List bucketing, we introduced a config validation step
where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4213) List bucketing error too restrictive
[
https://issues.apache.org/jira/browse/HIVE-4213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609262#comment-13609262
]
Mark Grover commented on HIVE-4213:
---
[~gangtimliu] I would love to hear your thoughts on this! Thanks in advance!
List bucketing error too restrictive
Key: HIVE-4213
URL: https://issues.apache.org/jira/browse/HIVE-4213
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Mark Grover
Fix For: 0.11.0
With the introduction of List bucketing, we introduced a config validation
step where we say:
{code}
SUPPORT_DIR_MUST_TRUE_FOR_LIST_BUCKETING(
10199,
hive.mapred.supports.subdirectories must be true
+ if any one of following is true:
hive.internal.ddl.list.bucketing.enable,
+ hive.optimize.listbucketing and mapred.input.dir.recursive),
{code}
This seems overly restrictive to because there are use cases where people may
want to use {{mapred.input.dir.recursive}} to {{true}} even when they don't
care about list bucketing.
Is that not true?
For example, here is the unit test code for {{clientpositive/recursive_dir.q}}
{code}
CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING)
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
SELECT key+11 FROM src WHERE key=484;
ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
ALTER TABLE fact_daily ADD PARTITION (ds='1')
LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
set hive.mapred.supports.subdirectories=true;
set mapred.input.dir.recursive=true;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
SELECT * FROM fact_daily WHERE ds='1';
SELECT count(1) FROM fact_daily WHERE ds='1';
{code}
The unit test doesn't seem to be concerned about list bucketing but wants to
set {{mapred.input.dir.recursive}} to {{true}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4070) Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive
[ https://issues.apache.org/jira/browse/HIVE-4070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609278#comment-13609278 ] Mark Grover commented on HIVE-4070: --- Thanks Gwen. In that case, I am ok with documenting it and resolving this JIRA as won't fix. Is that ok with you as well, [~mackrorysd]? Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case insensitive Key: HIVE-4070 URL: https://issues.apache.org/jira/browse/HIVE-4070 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Trivial Fix For: 0.11.0 Hive's like operator seems to be case sensitive. See https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLike.java#L164 However, MySQL's like operator is case insensitive. I don't have other DB's (like PostgreSQL) installed and handy but I am guessing their LIKE is case insensitive as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4208) Clientpositive test parenthesis_star_by is non-deteministic
Mark Grover created HIVE-4208:
-
Summary: Clientpositive test parenthesis_star_by is
non-deteministic
Key: HIVE-4208
URL: https://issues.apache.org/jira/browse/HIVE-4208
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
parenthesis_star_by is testing {{DISTRIBUTE BY}}; however, the order of rows
returned by {{DISTRIBUTE BY}} is not deterministic and results in failures
depending on Hadoop version.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4208) Clientpositive test parenthesis_star_by is non-deteministic
[
https://issues.apache.org/jira/browse/HIVE-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-4208:
--
Attachment: HIVE-4208.1.patch
Uploading patch, adding {{ORDER BY}} to make the order of rows in the result
deterministic.
Clientpositive test parenthesis_star_by is non-deteministic
---
Key: HIVE-4208
URL: https://issues.apache.org/jira/browse/HIVE-4208
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: HIVE-4208.1.patch
parenthesis_star_by is testing {{DISTRIBUTE BY}}; however, the order of rows
returned by {{DISTRIBUTE BY}} is not deterministic and results in failures
depending on Hadoop version.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4208) Clientpositive test parenthesis_star_by is non-deteministic
[
https://issues.apache.org/jira/browse/HIVE-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-4208:
--
Status: Patch Available (was: Open)
[~namitjain] since you were the original author of this test, would you mind
reviewing it please?
Clientpositive test parenthesis_star_by is non-deteministic
---
Key: HIVE-4208
URL: https://issues.apache.org/jira/browse/HIVE-4208
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: HIVE-4208.1.patch
parenthesis_star_by is testing {{DISTRIBUTE BY}}; however, the order of rows
returned by {{DISTRIBUTE BY}} is not deterministic and results in failures
depending on Hadoop version.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4174) Round UDF converts BigInts to double
Mark Grover created HIVE-4174:
-
Summary: Round UDF converts BigInts to double
Key: HIVE-4174
URL: https://issues.apache.org/jira/browse/HIVE-4174
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Chen Chun pointed out on the hive-user mailing list that round() in Hive 0.10
returns
{code}
select round(cast(1234560 as BIGINT)), round(cast(12345670 as BIGINT)) from
test limit 1;
//hive 0.10
1234560.0 1.234567E7
{code}
This is not consistent with
MySQL(http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html#function_round)
which quotes
{code}
The return type is the same type as that of the first argument (assuming that
it is integer, double, or decimal). This means that for an integer argument,
the result is an integer (no decimal places)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3963) Allow Hive to connect to RDBMS
[
https://issues.apache.org/jira/browse/HIVE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599022#comment-13599022
]
Mark Grover commented on HIVE-3963:
---
Thanks Maxime. Can you also please post the patch on reviewboard (or
Phabricator) as well?
Allow Hive to connect to RDBMS
--
Key: HIVE-3963
URL: https://issues.apache.org/jira/browse/HIVE-3963
Project: Hive
Issue Type: New Feature
Components: Import/Export, JDBC, SQL, StorageHandler
Affects Versions: 0.10.0, 0.9.1, 0.11.0
Reporter: Maxime LANCIAUX
Fix For: 0.10.1
Attachments: patchfile
I am thinking about something like :
SELECT jdbcload('driver','url','user','password','sql') FROM dual;
There is already a JIRA https://issues.apache.org/jira/browse/HIVE-1555 for
JDBCStorageHandler
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4144) Add select database() command to show the current database
Mark Grover created HIVE-4144:
-
Summary: Add select database() command to show the current
database
Key: HIVE-4144
URL: https://issues.apache.org/jira/browse/HIVE-4144
Project: Hive
Issue Type: Bug
Components: SQL
Reporter: Mark Grover
A recent hive-user mailing list conversation asked about having a command to
show the current database.
http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/%3CCAMGr+0i+CRY69m3id=DxthmUCWLf0NxpKMCtROb=uauh2va...@mail.gmail.com%3E
MySQL seems to have a command to do so:
{code}
select database();
{code}
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_database
We should look into having something similar in Hive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3850) hour() function returns 12 hour clock value when using timestamp datatype
[ https://issues.apache.org/jira/browse/HIVE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598049#comment-13598049 ] Mark Grover commented on HIVE-3850: --- +1 (non-committer) hour() function returns 12 hour clock value when using timestamp datatype - Key: HIVE-3850 URL: https://issues.apache.org/jira/browse/HIVE-3850 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.9.0, 0.10.0 Reporter: Pieterjan Vriends Fix For: 0.11.0 Attachments: hive-3850_1.patch, HIVE-3850.patch.txt Apparently UDFHour.java does have two evaluate() functions. One that does accept a Text object as parameter and one that does use a TimeStampWritable object as parameter. The first function does return the value of Calendar.HOUR_OF_DAY and the second one of Calendar.HOUR. In the documentation I couldn't find any information on the overload of the evaluation function. I did spent quite some time finding out why my statement didn't return a 24 hour clock value. Shouldn't both functions return the same? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3963) Allow Hive to connect to RDBMS
[
https://issues.apache.org/jira/browse/HIVE-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598050#comment-13598050
]
Mark Grover commented on HIVE-3963:
---
Maxime, can you upload a patch and post it for review.
Also, out of curiosity, when would a user use this instead of using something
like Apache Sqoop.
Allow Hive to connect to RDBMS
--
Key: HIVE-3963
URL: https://issues.apache.org/jira/browse/HIVE-3963
Project: Hive
Issue Type: New Feature
Components: Import/Export, JDBC, SQL, StorageHandler
Affects Versions: 0.9.0, 0.10.0, 0.9.1, 0.11.0
Reporter: Maxime LANCIAUX
I am thinking about something like :
SELECT jdbcload('driver','url','user','password','sql') FROM dual;
There is already a JIRA https://issues.apache.org/jira/browse/HIVE-1555 for
JDBCStorageHandler
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4053) Add support for phonetic algorithms in Hive
[ https://issues.apache.org/jira/browse/HIVE-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13591449#comment-13591449 ] Mark Grover commented on HIVE-4053: --- Krishna, thanks for doing this. I don't have a whole lot of insight into these particular algorithms but do they always take the same parameters? What's the possibility of a new phonetic algorithm using a different set or number of parameters? If these functions always take same parameters, it may make sense to do (2). However, if not, (1) would be a good idea. Of course, you can still refactor the code and share amongst all different UDFs even when they are separate. To post a review on reviewboard, go to reviews.apache.org. Generate a diff file of your changes on top of hive trunk (using svn diff or git diff) and upload that diff (use hive repository when using svn diff output and hive-git repository when using git diff output). Please let me know if you have any further questions. Add support for phonetic algorithms in Hive --- Key: HIVE-4053 URL: https://issues.apache.org/jira/browse/HIVE-4053 Project: Hive Issue Type: New Feature Components: UDF Affects Versions: 0.10.0 Reporter: Krishna Labels: patch Fix For: 0.10.0 Attachments: FunctionRegistry.java, GenericUDFRefinedSoundex.java, HIVE-4053.1.patch.txt Following phonetic algorithms should be considered, which are very useful in search: Soundex: http://en.wikipedia.org/wiki/Soundex Refined Soundex: Refer to the comment on 22/Feb/13 23:51 Daitch–Mokotoff Soundex: http://en.wikipedia.org/wiki/Daitch%E2%80%93Mokotoff_Soundex Metaphone and Double Metaphone: http://en.wikipedia.org/wiki/Metaphone New York State Identification and Intelligence System (NYSIIS): http://en.wikipedia.org/wiki/New_York_State_Identification_and_Intelligence_System Caverphone: http://en.wikipedia.org/wiki/Caverphone -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4100) Improve regex_replace UDF to allow non-ascii characters
Mark Grover created HIVE-4100: - Summary: Improve regex_replace UDF to allow non-ascii characters Key: HIVE-4100 URL: https://issues.apache.org/jira/browse/HIVE-4100 Project: Hive Issue Type: Improvement Components: UDF Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 There have a been a few email threads on the user mailing list regarding regex_replace UDF not supporting non-ASCII characters. We should validate that and improve the UDF to allow it. Translate UDF will be a good reference since it does that by using code points instead of characters -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4070) Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case sensitive
Mark Grover created HIVE-4070: - Summary: Like operator in Hive is case sensitive while in MySQL (and most likely other DBs) it's case sensitive Key: HIVE-4070 URL: https://issues.apache.org/jira/browse/HIVE-4070 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Hive's like operator seems to be case sensitive. See https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFLike.java#L164 However, MySQL's like operator is case insensitive. I don't have other DB's (like PostgreSQL) installed and handy but I am guessing their LIKE is case insensitive as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4053) Add support for phonetic algorithms in Hive
[ https://issues.apache.org/jira/browse/HIVE-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585226#comment-13585226 ] Mark Grover commented on HIVE-4053: --- Can you please post a patch on the JIRA and post a review on reviewboard? You should also consider adding some unit tests. If you need help with any of this, please let us know. Add support for phonetic algorithms in Hive --- Key: HIVE-4053 URL: https://issues.apache.org/jira/browse/HIVE-4053 Project: Hive Issue Type: New Feature Components: UDF Reporter: Krishna Attachments: FunctionRegistry.java, GenericUDFRefinedSoundex.java Following phonetic algorithms should be considered, which are very useful in search: Soundex Refined Soundex Daitch–Mokotoff Soundex Metaphone and Double Metaphone New York State Identification and Intelligence System (NYSIIS) Caverphone -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3885) CLI command SHOW PARTITIONS could be extended to provide LOCATION information
[ https://issues.apache.org/jira/browse/HIVE-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579844#comment-13579844 ] Mark Grover commented on HIVE-3885: --- Sanjay: I am replying to your question on the IRC channel. I think this will be a good starting point, https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/ShowPartitionsDesc.java See who uses this class and how. CLI command SHOW PARTITIONS could be extended to provide LOCATION information --- Key: HIVE-3885 URL: https://issues.apache.org/jira/browse/HIVE-3885 Project: Hive Issue Type: New Feature Reporter: Sanjay Subramanian Priority: Minor SHOW PARTITIONS does not provide information on the HDFS location of the data. The workaround is to query the metadata DB. The following command will give you the HDFS file locations as stored in the metadata tables. echo select t.TBL_NAME, p.PART_NAME, s.LOCATION from PARTITIONS p, SDS s, TBLS t where t.TBL_ID=p.TBL_ID and p.SD_ID=s.SD_ID |mysql -uusername –ppassword hive_metastore_DB |grep HIVETABLENAME|less If this could be encapsulated in a CLI command SHOW LOCATIONS that displays PARTITION_NAMELOCATION -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3951) Allow Decimal type columns in Regex Serde
[ https://issues.apache.org/jira/browse/HIVE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13577324#comment-13577324 ] Mark Grover commented on HIVE-3951: --- This patch is ready for review. Would anyone be willing to please review? Allow Decimal type columns in Regex Serde - Key: HIVE-3951 URL: https://issues.apache.org/jira/browse/HIVE-3951 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3951.1.patch Decimal type in Hive was recently added by HIVE-2693. We should allow users to create tables with decimal type columns when using Regex Serde. HIVE-3004 did something similar for other primitive types. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover reassigned HIVE-4003: - Assignee: Mark Grover NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575185#comment-13575185 ] Mark Grover commented on HIVE-4003: --- Ok, in that case, I can take it on from here. Thanks for your contribution, Thomas! NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Fix For: 0.11.0 Attachments: HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3995) PostgreSQL upgrade scripts are not valid
[ https://issues.apache.org/jira/browse/HIVE-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575186#comment-13575186 ] Mark Grover commented on HIVE-3995: --- Thanks, Namit! PostgreSQL upgrade scripts are not valid Key: HIVE-3995 URL: https://issues.apache.org/jira/browse/HIVE-3995 Project: Hive Issue Type: Bug Components: Metastore Reporter: Jarek Jarcec Cecho Fix For: 0.11.0 Attachments: postgre_update_issue.patch, postgre_update_issue.patch I've noticed that scripts for upgrading metastore backed up on PostgreSQL are not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3179) HBase Handler doesn't handle NULLs properly
[
https://issues.apache.org/jira/browse/HIVE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575246#comment-13575246
]
Mark Grover commented on HIVE-3179:
---
They timed out on my pseudo-distributed laptop but that most likely is an
environment issue local to me. But looks like Lars mentioned that he had run
the tests, so that should be ok.
I will try to fix the environment, but don't wait on me.
HBase Handler doesn't handle NULLs properly
---
Key: HIVE-3179
URL: https://issues.apache.org/jira/browse/HIVE-3179
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.9.0, 0.10.0
Reporter: Lars Francke
Priority: Critical
Attachments: HIVE-3179.1.patch
We found a quite severe issue in the HBase Handler which actually means that
Hive potentially returns incorrect data if a column has NULL values in HBase
(which means the cell doesn't even exist)
In HBase Shell:
{noformat}
create 'hive_hbase_test', 'test'
put 'hive_hbase_test', '1', 'test:c1', 'c1-1'
put 'hive_hbase_test', '1', 'test:c2', 'c2-1'
put 'hive_hbase_test', '1', 'test:c3', 'c3-1'
put 'hive_hbase_test', '2', 'test:c1', 'c1-2'
{noformat}
In Hive:
{noformat}
DROP TABLE IF EXISTS hive_hbase_test;
CREATE EXTERNAL TABLE hive_hbase_test (
id int,
c1 string,
c2 string,
c3 string
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping =
:key#s,test:c1#s,test:c2#s,test:c3#s)
TBLPROPERTIES(hbase.table.name = hive_hbase_test);
hive select * from hive_hbase_test;
OK
1 c1-1c2-1c3-1
2 c1-2NULLNULL
hive select c1 from hive_hbase_test;
c1-1
c1-2
hive select c1, c2 from hive_hbase_test;
c1-1 c2-1
c1-2 NULL
{noformat}
So far everything is correct but now:
{noformat}
hive select c1, c2, c2 from hive_hbase_test;
c1-1 c2-1c2-1
c1-2 NULLc2-1
{noformat}
Selecting c2 twice works the first time but the second time we
actually get the value from the previous row.
{noformat}
hive select c1, c3, c2, c2, c3, c3, c1 from hive_hbase_test;
c1-1 c3-1c2-1c2-1c3-1c3-1c1-1
c1-2 NULLNULLc2-1c3-1c3-1c1-2
{noformat}
We've narrowed this down to an early initialization of
{{fieldsInited\[fieldID] = true}} in {{LazyHBaseRow#uncheckedGetField}} and
we'll try to provide a patch which surely needs review.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3999) Mysql metastore upgrade script will end up with different schema than the full schema load
[
https://issues.apache.org/jira/browse/HIVE-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574505#comment-13574505
]
Mark Grover commented on HIVE-3999:
---
Thanks Namit!
Mysql metastore upgrade script will end up with different schema than the
full schema load
--
Key: HIVE-3999
URL: https://issues.apache.org/jira/browse/HIVE-3999
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Jarek Jarcec Cecho
Fix For: 0.11.0
Attachments: mysql_upgrade_issue.patch
I've noticed that the file {{hive-schema-0.10.0.mysql.sql}} is creating table
SDS with following column:
{code}
`IS_STOREDASSUBDIRECTORIES` bit(1) NOT NULL,
{code}
However the upgrade script {{011-HIVE-3649.mysql.sql}} will create the column
differently:
{code}
ALTER TABLE `SDS` ADD `IS_STOREDASSUBDIRECTORIES` bit(1) ;
{code}
Thus user will get slightly different schema each time - once with NOT NULL
and secondly with NULL definition.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3998) Oracle metastore update script will fail when upgrading from 0.9.0 to 0.10.0
[
https://issues.apache.org/jira/browse/HIVE-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574504#comment-13574504
]
Mark Grover commented on HIVE-3998:
---
Thanks Namit! Could you look at HIVE-3995 as well when you get a chance? It
does the same change to PostgreSQL scripts.
Oracle metastore update script will fail when upgrading from 0.9.0 to 0.10.0
Key: HIVE-3998
URL: https://issues.apache.org/jira/browse/HIVE-3998
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Jarek Jarcec Cecho
Fix For: 0.11.0
Attachments: oracle_update_issue.patch
The problem is in following file {{011-HIVE-3649.oracle.sql}} that contains
following line:
{code}
alter table SDS add IS_STOREDASSUBDIRECTORIES NUMBER(1) NOT NULL;
{code}
This query will however fail if the table SDS have at least one row.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3179) HBase Handler doesn't handle NULLs properly
[
https://issues.apache.org/jira/browse/HIVE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574553#comment-13574553
]
Mark Grover commented on HIVE-3179:
---
Running TestHBaseCliDriver tests...
HBase Handler doesn't handle NULLs properly
---
Key: HIVE-3179
URL: https://issues.apache.org/jira/browse/HIVE-3179
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.9.0, 0.10.0
Reporter: Lars Francke
Priority: Critical
Attachments: HIVE-3179.1.patch
We found a quite severe issue in the HBase Handler which actually means that
Hive potentially returns incorrect data if a column has NULL values in HBase
(which means the cell doesn't even exist)
In HBase Shell:
{noformat}
create 'hive_hbase_test', 'test'
put 'hive_hbase_test', '1', 'test:c1', 'c1-1'
put 'hive_hbase_test', '1', 'test:c2', 'c2-1'
put 'hive_hbase_test', '1', 'test:c3', 'c3-1'
put 'hive_hbase_test', '2', 'test:c1', 'c1-2'
{noformat}
In Hive:
{noformat}
DROP TABLE IF EXISTS hive_hbase_test;
CREATE EXTERNAL TABLE hive_hbase_test (
id int,
c1 string,
c2 string,
c3 string
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (hbase.columns.mapping =
:key#s,test:c1#s,test:c2#s,test:c3#s)
TBLPROPERTIES(hbase.table.name = hive_hbase_test);
hive select * from hive_hbase_test;
OK
1 c1-1c2-1c3-1
2 c1-2NULLNULL
hive select c1 from hive_hbase_test;
c1-1
c1-2
hive select c1, c2 from hive_hbase_test;
c1-1 c2-1
c1-2 NULL
{noformat}
So far everything is correct but now:
{noformat}
hive select c1, c2, c2 from hive_hbase_test;
c1-1 c2-1c2-1
c1-2 NULLc2-1
{noformat}
Selecting c2 twice works the first time but the second time we
actually get the value from the previous row.
{noformat}
hive select c1, c3, c2, c2, c3, c3, c1 from hive_hbase_test;
c1-1 c3-1c2-1c2-1c3-1c3-1c1-1
c1-2 NULLNULLc2-1c3-1c3-1c1-2
{noformat}
We've narrowed this down to an early initialization of
{{fieldsInited\[fieldID] = true}} in {{LazyHBaseRow#uncheckedGetField}} and
we'll try to provide a patch which surely needs review.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4003) NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
[ https://issues.apache.org/jira/browse/HIVE-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-4003: -- Description: Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 Affects Version/s: 0.10.0 Fix Version/s: 0.11.0 NullPointerException in ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java - Key: HIVE-4003 URL: https://issues.apache.org/jira/browse/HIVE-4003 Project: Hive Issue Type: Bug Affects Versions: 0.10.0 Reporter: Thomas Adam Fix For: 0.11.0 Attachments: HIVE-4003.patch Utilities.java seems to be throwing a NPE. Change contributed by Thomas Adam. Reference: https://github.com/tecbot/hive/commit/1e29d88837e4101a76e870a716aadb729437355b#commitcomment-2588350 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3995) PostgreSQL upgrade scripts are not valid
[ https://issues.apache.org/jira/browse/HIVE-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574249#comment-13574249 ] Mark Grover commented on HIVE-3995: --- +1 (non-committer) Thanks Jarcec! I left a comment (no action required from your side) on Reviewboard that false seems to be the right value to put for existing rows. Perhaps, [~gangtimliu] or [~namitjain], who were involved with HIVE-3649 can confirm. I like the new approach of adding the new column as Nullable, updating the existing null values and then setting the column to not-null. The only other approach I can think of is to use a default value (of false) but that seems more risky in the long run. PostgreSQL upgrade scripts are not valid Key: HIVE-3995 URL: https://issues.apache.org/jira/browse/HIVE-3995 Project: Hive Issue Type: Bug Components: Metastore Reporter: Jarek Jarcec Cecho Attachments: postgre_update_issue.patch, postgre_update_issue.patch I've noticed that scripts for upgrading metastore backed up on PostgreSQL are not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3998) Oracle metastore update script will fail when upgrading from 0.9.0 to 0.10.0
[
https://issues.apache.org/jira/browse/HIVE-3998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574251#comment-13574251
]
Mark Grover commented on HIVE-3998:
---
+1 (non-committer)
Oracle metastore update script will fail when upgrading from 0.9.0 to 0.10.0
Key: HIVE-3998
URL: https://issues.apache.org/jira/browse/HIVE-3998
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Jarek Jarcec Cecho
Attachments: oracle_update_issue.patch
The problem is in following file {{011-HIVE-3649.oracle.sql}} that contains
following line:
{code}
alter table SDS add IS_STOREDASSUBDIRECTORIES NUMBER(1) NOT NULL;
{code}
This query will however fail if the table SDS have at least one row.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3999) Mysql metastore upgrade script will end up with different schema than the full schema load
[
https://issues.apache.org/jira/browse/HIVE-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13574253#comment-13574253
]
Mark Grover commented on HIVE-3999:
---
+1 (non-committer)
Mysql metastore upgrade script will end up with different schema than the
full schema load
--
Key: HIVE-3999
URL: https://issues.apache.org/jira/browse/HIVE-3999
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Jarek Jarcec Cecho
Attachments: mysql_upgrade_issue.patch
I've noticed that the file {{hive-schema-0.10.0.mysql.sql}} is creating table
SDS with following column:
{code}
`IS_STOREDASSUBDIRECTORIES` bit(1) NOT NULL,
{code}
However the upgrade script {{011-HIVE-3649.mysql.sql}} will create the column
differently:
{code}
ALTER TABLE `SDS` ADD `IS_STOREDASSUBDIRECTORIES` bit(1) ;
{code}
Thus user will get slightly different schema each time - once with NOT NULL
and secondly with NULL definition.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3995) PostgreSQL upgrade scripts are not valid
[ https://issues.apache.org/jira/browse/HIVE-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573110#comment-13573110 ] Mark Grover commented on HIVE-3995: --- Jarcec, thanks for this! The PostgreSQL scripts were not supported until recently, so it's possible the syntax in existing scripts is incorrect. I left some comments on the review. PostgreSQL upgrade scripts are not valid Key: HIVE-3995 URL: https://issues.apache.org/jira/browse/HIVE-3995 Project: Hive Issue Type: Bug Components: Metastore Reporter: Jarek Jarcec Cecho Attachments: postgre_update_issue.patch I've noticed that scripts for upgrading metastore backed up on PostgreSQL are not valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3725) Add support for pulling HBase columns with prefixes
[ https://issues.apache.org/jira/browse/HIVE-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13570925#comment-13570925 ] Mark Grover commented on HIVE-3725: --- Comments on reviewboard Add support for pulling HBase columns with prefixes --- Key: HIVE-3725 URL: https://issues.apache.org/jira/browse/HIVE-3725 Project: Hive Issue Type: Improvement Components: HBase Handler Affects Versions: 0.9.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-3725.1.patch.txt Current HBase Hive integration supports reading many values from the same row by specifying a column family. And specifying just the column family can pull in all qualifiers within the family. We should add in support to be able to specify a prefix for the qualifier and all columns that start with the prefix would automatically get pulled in. A wildcard support would be ideal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3918) Normalize more CRLF line endings
[ https://issues.apache.org/jira/browse/HIVE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569933#comment-13569933 ] Mark Grover commented on HIVE-3918: --- Thanks, Ashutosh, for committing this! Normalize more CRLF line endings Key: HIVE-3918 URL: https://issues.apache.org/jira/browse/HIVE-3918 Project: Hive Issue Type: Bug Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3918.1.patch While I tried to fix files that had incompatible line endings in HIVE-3858, I missed some more files. It was most likely because HIVE-3858 was tested by me on Mac OS X but when I started using the github repository on a ubuntu box, I saw there are some more files in the Hive repo that have incorrect line endings. These files get added to the index automatically when a change is made to the local git repo. Once these cosmetic changes get committed, we would have gotten rid of all windows/old Mac OS style line endings in our Hive repository. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3932) Hive release tarballs don't contain PostgreSQL metastore scripts
[ https://issues.apache.org/jira/browse/HIVE-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569934#comment-13569934 ] Mark Grover commented on HIVE-3932: --- Thanks, Ashutosh, for committing this! Hive release tarballs don't contain PostgreSQL metastore scripts Key: HIVE-3932 URL: https://issues.apache.org/jira/browse/HIVE-3932 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3932.1.patch By means of HIVE-2552, we decided to not include Hive metastore upgrade scripts for PostgreSQL in Hive release tarballs. This was done primarily because the scripts were incomplete at that time. However, since then, starting at least 0.10.0 (maybe 0.9?) PostgreSQL scripts are now being maintained and any changes to the metastore are reflected to them (HIVE-2529). Consequently, there is no reason to not release PostgreSQL scripts. This JIRA plans to disable/remove/ ignore the include.postgres property in build.xml that enables the exclusion of PostgreSQL scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3810) HiveHistory.log need to replace '\r' with space before writing Entry.value to historyfile
[
https://issues.apache.org/jira/browse/HIVE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569937#comment-13569937
]
Mark Grover commented on HIVE-3810:
---
Thanks, Edward for the review!
Your comment made me think about using {{System.getProperty(line.separator)}}
instead.
Let me post a new patch with that. I am not entirely sure how to test it (apart
from running the unit tests). If you have any ideas, please let me know.
HiveHistory.log need to replace '\r' with space before writing Entry.value to
historyfile
-
Key: HIVE-3810
URL: https://issues.apache.org/jira/browse/HIVE-3810
Project: Hive
Issue Type: Bug
Components: Logging
Reporter: qiangwang
Assignee: Mark Grover
Priority: Minor
Attachments: HIVE-3810.1.patch, HIVE-3810.2.patch, HIVE-3810.3.patch
HiveHistory.log will replace '\n' with space before writing Entry.value to
history file:
val = val.replace('\n', ' ');
but HiveHistory.parseHiveHistory use BufferedReader.readLine which takes
'\n', '\r', '\r\n' as line delimiter to parse history file
if val contains '\r', there is a high possibility that HiveHistory.parseLine
will fail, in which case usually RecordTypes.valueOf(recType) will throw
exception 'java.lang.IllegalArgumentException'
HiveHistory.log need to replace '\r' with space as well:
val = val.replace('\n', ' ');
changed to
val = val.replaceAll(\r|\n, );
or
val = val.replace('\r', ' ').replace('\n', ' ');
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3810) HiveHistory.log need to replace '\r' with space before writing Entry.value to historyfile
[
https://issues.apache.org/jira/browse/HIVE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-3810:
--
Attachment: HIVE-3810.3.patch
HiveHistory.log need to replace '\r' with space before writing Entry.value to
historyfile
-
Key: HIVE-3810
URL: https://issues.apache.org/jira/browse/HIVE-3810
Project: Hive
Issue Type: Bug
Components: Logging
Reporter: qiangwang
Assignee: Mark Grover
Priority: Minor
Attachments: HIVE-3810.1.patch, HIVE-3810.2.patch, HIVE-3810.3.patch
HiveHistory.log will replace '\n' with space before writing Entry.value to
history file:
val = val.replace('\n', ' ');
but HiveHistory.parseHiveHistory use BufferedReader.readLine which takes
'\n', '\r', '\r\n' as line delimiter to parse history file
if val contains '\r', there is a high possibility that HiveHistory.parseLine
will fail, in which case usually RecordTypes.valueOf(recType) will throw
exception 'java.lang.IllegalArgumentException'
HiveHistory.log need to replace '\r' with space as well:
val = val.replace('\n', ' ');
changed to
val = val.replaceAll(\r|\n, );
or
val = val.replace('\r', ' ').replace('\n', ' ');
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3975) Hive JIRA still shows 0.10 as unreleased in Affects Version/s dropdown
Mark Grover created HIVE-3975: - Summary: Hive JIRA still shows 0.10 as unreleased in Affects Version/s dropdown Key: HIVE-3975 URL: https://issues.apache.org/jira/browse/HIVE-3975 Project: Hive Issue Type: Bug Components: Website Affects Versions: 0.10.0 Reporter: Mark Grover Fix For: 0.10.0 The JIRA still shows 0.10.0 as unreleased when selecting Affects Version/s and Fix Version/s. Now that Hive 0.10.0 has been released, can we please change that? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3976) Support specifying scale and precision with Hive decimal type
Mark Grover created HIVE-3976:
-
Summary: Support specifying scale and precision with Hive decimal
type
Key: HIVE-3976
URL: https://issues.apache.org/jira/browse/HIVE-3976
Project: Hive
Issue Type: Improvement
Components: Query Processor, Types
Reporter: Mark Grover
HIVE-2693 introduced support for Decimal datatype in Hive. However, the current
implementation has unlimited precision and provides no way to specify precision
and scale when creating the table.
For example, MySQL allows users to specify scale and precision of the decimal
datatype when creating the table:
{code}
CREATE TABLE numbers (a DECIMAL(20,2));
{code}
Hive should support something similar too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2693) Add DECIMAL data type
[ https://issues.apache.org/jira/browse/HIVE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569234#comment-13569234 ] Mark Grover commented on HIVE-2693: --- Thanks for committing this, Carl! FYI, I have updated the wiki pages with info about Decimal type. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL Add DECIMAL data type - Key: HIVE-2693 URL: https://issues.apache.org/jira/browse/HIVE-2693 Project: Hive Issue Type: New Feature Components: Query Processor, Types Affects Versions: 0.10.0 Reporter: Carl Steinbach Assignee: Prasad Mujumdar Fix For: 0.11.0 Attachments: 2693_7.patch, 2693_8.patch, 2693_fix_all_tests1.patch, HIVE-2693-10.patch, HIVE-2693-11.patch, HIVE-2693-12-SortableSerDe.patch, HIVE-2693-13.patch, HIVE-2693-14.patch, HIVE-2693-15.patch, HIVE-2693-16.patch, HIVE-2693-17.patch, HIVE-2693-18.patch, HIVE-2693-19.patch, HIVE-2693-1.patch.txt, HIVE-2693-20.patch, HIVE-2693-21.patch, HIVE-2693-22.patch, HIVE-2693-23.patch, HIVE-2693-all.patch, HIVE-2693.D7683.1.patch, HIVE-2693-fix.patch, HIVE-2693.patch, HIVE-2693-take3.patch, HIVE-2693-take4.patch Add support for the DECIMAL data type. HIVE-2272 (TIMESTAMP) provides a nice template for how to do this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3951) Allow Decimal type columns in Regex Serde
[ https://issues.apache.org/jira/browse/HIVE-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3951: -- Status: Patch Available (was: Open) Allow Decimal type columns in Regex Serde - Key: HIVE-3951 URL: https://issues.apache.org/jira/browse/HIVE-3951 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3951.1.patch Decimal type in Hive was recently added by HIVE-2693. We should allow users to create tables with decimal type columns when using Regex Serde. HIVE-3004 did something similar for other primitive types. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3850) hour() function returns 12 hour clock value when using timestamp datatype
[
https://issues.apache.org/jira/browse/HIVE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567763#comment-13567763
]
Mark Grover commented on HIVE-3850:
---
It's all good, we were all new at some point:-)
Here is what I would do for the tests:
Developer Guide is a really good link to read on how unit tests work in Hive:
https://cwiki.apache.org/confluence/display/Hive/DeveloperGuide#DeveloperGuide-Unittestsanddebugging
Practically, unit tests take a long time to run on especially if you run it on
1 node (like me), so for a minor change like this, I run only the tests that
are impacted.
To figure that out what's impacted, I would do something like:
{code}
cd ql/src/test/queries
grep -ri hour *
{code}
For results under clientpositive, I would run something like:
{code}
ant test -Dtestcase=TestCliDriver -Dqfile=comma separated list of .q files
{code}
For results under clientnegative, I would run something like:
{code}
ant test TestNegativeCliDriver -Dqfile=comma separated list of .q files
{code}
If any of the above tests fail, it may legitimately be because of the bug fix,
in which case, you would want to update the test output. You can do that by
using {{-Doverwrite=true}} at the end of your commands. If the outputs, do need
to get updated, please include those diffs in the patch as well.
The committers would usually run the whole suite of tests (some times on faster
clusters), so don't be paranoid about testing all tests but of course, do you
due-diligence about finding all tests that may be impacted and making sure they
run fine.
Hope this makes sense. If you have any further questions, please don't hesitate
to ask. And, thanks for working on this patch!
hour() function returns 12 hour clock value when using timestamp datatype
-
Key: HIVE-3850
URL: https://issues.apache.org/jira/browse/HIVE-3850
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.9.0, 0.10.0
Reporter: Pieterjan Vriends
Fix For: 0.11.0
Attachments: HIVE-3850.patch.txt
Apparently UDFHour.java does have two evaluate() functions. One that does
accept a Text object as parameter and one that does use a TimeStampWritable
object as parameter. The first function does return the value of
Calendar.HOUR_OF_DAY and the second one of Calendar.HOUR. In the
documentation I couldn't find any information on the overload of the
evaluation function. I did spent quite some time finding out why my statement
didn't return a 24 hour clock value.
Shouldn't both functions return the same?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3850) hour() function returns 12 hour clock value when using timestamp datatype
[ https://issues.apache.org/jira/browse/HIVE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567355#comment-13567355 ] Mark Grover commented on HIVE-3850: --- For completeness, review is at: https://reviews.apache.org/r/9171/ hour() function returns 12 hour clock value when using timestamp datatype - Key: HIVE-3850 URL: https://issues.apache.org/jira/browse/HIVE-3850 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.9.0 Reporter: Pieterjan Vriends Attachments: HIVE-3850.patch.txt Apparently UDFHour.java does have two evaluate() functions. One that does accept a Text object as parameter and one that does use a TimeStampWritable object as parameter. The first function does return the value of Calendar.HOUR_OF_DAY and the second one of Calendar.HOUR. In the documentation I couldn't find any information on the overload of the evaluation function. I did spent quite some time finding out why my statement didn't return a 24 hour clock value. Shouldn't both functions return the same? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3850) hour() function returns 12 hour clock value when using timestamp datatype
[ https://issues.apache.org/jira/browse/HIVE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567356#comment-13567356 ] Mark Grover commented on HIVE-3850: --- Patch looks good to me. Usually, I would ask for unit tests to be added with any change but given that it's a trivial change, I would be ok without new tests. We should, however, make sure we update the existing unit tests if needed. Did you get a chance to run the unit tests (atleast the ones that use hour UDF) and make sure no changes are required in their output? hour() function returns 12 hour clock value when using timestamp datatype - Key: HIVE-3850 URL: https://issues.apache.org/jira/browse/HIVE-3850 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.9.0 Reporter: Pieterjan Vriends Attachments: HIVE-3850.patch.txt Apparently UDFHour.java does have two evaluate() functions. One that does accept a Text object as parameter and one that does use a TimeStampWritable object as parameter. The first function does return the value of Calendar.HOUR_OF_DAY and the second one of Calendar.HOUR. In the documentation I couldn't find any information on the overload of the evaluation function. I did spent quite some time finding out why my statement didn't return a 24 hour clock value. Shouldn't both functions return the same? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-3850) hour() function returns 12 hour clock value when using timestamp datatype
[ https://issues.apache.org/jira/browse/HIVE-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover reopened HIVE-3850: --- The change wasn't committed, re-opening the JIRA. hour() function returns 12 hour clock value when using timestamp datatype - Key: HIVE-3850 URL: https://issues.apache.org/jira/browse/HIVE-3850 Project: Hive Issue Type: Bug Components: UDF Affects Versions: 0.9.0, 0.10.0 Reporter: Pieterjan Vriends Fix For: 0.11.0 Attachments: HIVE-3850.patch.txt Apparently UDFHour.java does have two evaluate() functions. One that does accept a Text object as parameter and one that does use a TimeStampWritable object as parameter. The first function does return the value of Calendar.HOUR_OF_DAY and the second one of Calendar.HOUR. In the documentation I couldn't find any information on the overload of the evaluation function. I did spent quite some time finding out why my statement didn't return a 24 hour clock value. Shouldn't both functions return the same? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3810) HiveHistory.log need to replace '\r' with space before writing Entry.value to historyfile
[
https://issues.apache.org/jira/browse/HIVE-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564975#comment-13564975
]
Mark Grover commented on HIVE-3810:
---
Review at https://reviews.apache.org/r/9129/
HiveHistory.log need to replace '\r' with space before writing Entry.value to
historyfile
-
Key: HIVE-3810
URL: https://issues.apache.org/jira/browse/HIVE-3810
Project: Hive
Issue Type: Bug
Components: Logging
Reporter: qiangwang
Assignee: Mark Grover
Priority: Minor
Attachments: HIVE-3810.1.patch, HIVE-3810.2.patch
HiveHistory.log will replace '\n' with space before writing Entry.value to
history file:
val = val.replace('\n', ' ');
but HiveHistory.parseHiveHistory use BufferedReader.readLine which takes
'\n', '\r', '\r\n' as line delimiter to parse history file
if val contains '\r', there is a high possibility that HiveHistory.parseLine
will fail, in which case usually RecordTypes.valueOf(recType) will throw
exception 'java.lang.IllegalArgumentException'
HiveHistory.log need to replace '\r' with space as well:
val = val.replace('\n', ' ');
changed to
val = val.replaceAll(\r|\n, );
or
val = val.replace('\r', ' ').replace('\n', ' ');
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3957) Add pseudo-BNF grammar for RCFile to Javadoc
[ https://issues.apache.org/jira/browse/HIVE-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565156#comment-13565156 ] Mark Grover commented on HIVE-3957: --- Some of the comments are being garbled in Javadoc. New patch coming soon. Add pseudo-BNF grammar for RCFile to Javadoc Key: HIVE-3957 URL: https://issues.apache.org/jira/browse/HIVE-3957 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Minor Fix For: 0.11.0 Attachments: HIVE-3957.1.patch Carl Steinbach did wrote some nice pseudo-BNF grammar for RCFile. I think it would rad to put it in the Javadoc to benefit other developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3957) Add pseudo-BNF grammar for RCFile to Javadoc
[ https://issues.apache.org/jira/browse/HIVE-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3957: -- Attachment: HIVE-3957.2.patch Rendered javadoc looks much better now. Add pseudo-BNF grammar for RCFile to Javadoc Key: HIVE-3957 URL: https://issues.apache.org/jira/browse/HIVE-3957 Project: Hive Issue Type: Bug Components: Documentation Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Priority: Minor Fix For: 0.11.0 Attachments: HIVE-3957.1.patch, HIVE-3957.2.patch Carl Steinbach did wrote some nice pseudo-BNF grammar for RCFile. I think it would rad to put it in the Javadoc to benefit other developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3951) Allow Decimal type columns in Regex Serde
Mark Grover created HIVE-3951: - Summary: Allow Decimal type columns in Regex Serde Key: HIVE-3951 URL: https://issues.apache.org/jira/browse/HIVE-3951 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Decimal type in Hive was recently added by HIVE-2693. We should allow users to create tables with decimal type columns when using Regex Serde. HIVE-3004 did something similar for other primitive types. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3932) Hive release tarballs don't contain PostGreSQL metastore scripts
[
https://issues.apache.org/jira/browse/HIVE-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Grover updated HIVE-3932:
--
Attachment: HIVE-3932.1.patch
Removes the includes.postgres property so PostGreSQL metastore scripts are
included by default (like mysql, derby and oracle). This is essentially a
revert of HIVE-2552.
Testing done:
1. Verified PostGreSQL scripts were infact being included under
{{build/dist/scripts/metastore/upgrade}}
2. Verified that the {{vcs.excludes}} property is still being honored.
Hive release tarballs don't contain PostGreSQL metastore scripts
Key: HIVE-3932
URL: https://issues.apache.org/jira/browse/HIVE-3932
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: HIVE-3932.1.patch
By means of HIVE-2552, we decided to not include Hive metastore upgrade
scripts for PostGreSQL in Hive release tarballs. This was done primarily
because the scripts were incomplete at that time. However, since then,
starting at least 0.10.0 (maybe 0.9?) PostGreSQL scripts are now being
maintained and any changes to the metastore are reflected to them
(HIVE-2529). Consequently, there is no reason to not release PostGreSQL
scripts. This JIRA plans to disable/remove/ ignore the include.postgres
property in build.xml that enables the exclusion of PostGreSQL scripts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3932) Hive release tarballs don't contain PostgreSQL metastore scripts
[ https://issues.apache.org/jira/browse/HIVE-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3932: -- Summary: Hive release tarballs don't contain PostgreSQL metastore scripts (was: Hive release tarballs don't contain PostGreSQL metastore scripts) Hive release tarballs don't contain PostgreSQL metastore scripts Key: HIVE-3932 URL: https://issues.apache.org/jira/browse/HIVE-3932 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3932.1.patch By means of HIVE-2552, we decided to not include Hive metastore upgrade scripts for PostGreSQL in Hive release tarballs. This was done primarily because the scripts were incomplete at that time. However, since then, starting at least 0.10.0 (maybe 0.9?) PostGreSQL scripts are now being maintained and any changes to the metastore are reflected to them (HIVE-2529). Consequently, there is no reason to not release PostGreSQL scripts. This JIRA plans to disable/remove/ ignore the include.postgres property in build.xml that enables the exclusion of PostGreSQL scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3932) Hive release tarballs don't contain PostgreSQL metastore scripts
[ https://issues.apache.org/jira/browse/HIVE-3932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3932: -- Description: By means of HIVE-2552, we decided to not include Hive metastore upgrade scripts for PostgreSQL in Hive release tarballs. This was done primarily because the scripts were incomplete at that time. However, since then, starting at least 0.10.0 (maybe 0.9?) PostgreSQL scripts are now being maintained and any changes to the metastore are reflected to them (HIVE-2529). Consequently, there is no reason to not release PostgreSQL scripts. This JIRA plans to disable/remove/ ignore the include.postgres property in build.xml that enables the exclusion of PostgreSQL scripts. (was: By means of HIVE-2552, we decided to not include Hive metastore upgrade scripts for PostGreSQL in Hive release tarballs. This was done primarily because the scripts were incomplete at that time. However, since then, starting at least 0.10.0 (maybe 0.9?) PostGreSQL scripts are now being maintained and any changes to the metastore are reflected to them (HIVE-2529). Consequently, there is no reason to not release PostGreSQL scripts. This JIRA plans to disable/remove/ ignore the include.postgres property in build.xml that enables the exclusion of PostGreSQL scripts.) Hive release tarballs don't contain PostgreSQL metastore scripts Key: HIVE-3932 URL: https://issues.apache.org/jira/browse/HIVE-3932 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.8.0, 0.8.1, 0.9.0, 0.10.0 Reporter: Mark Grover Assignee: Mark Grover Fix For: 0.11.0 Attachments: HIVE-3932.1.patch By means of HIVE-2552, we decided to not include Hive metastore upgrade scripts for PostgreSQL in Hive release tarballs. This was done primarily because the scripts were incomplete at that time. However, since then, starting at least 0.10.0 (maybe 0.9?) PostgreSQL scripts are now being maintained and any changes to the metastore are reflected to them (HIVE-2529). Consequently, there is no reason to not release PostgreSQL scripts. This JIRA plans to disable/remove/ ignore the include.postgres property in build.xml that enables the exclusion of PostgreSQL scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2868) Insert into table wipes out table content
[ https://issues.apache.org/jira/browse/HIVE-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561152#comment-13561152 ] Mark Grover commented on HIVE-2868: --- You are most likely running into HIVE-3062. [~mcazzari] and [~deanwampler], if you get a chance, can you please try with the table name being in all lower case, it should work as expected. FWIW, HIVE-3062 got fixed in recently released hive 0.10 Insert into table wipes out table content - Key: HIVE-2868 URL: https://issues.apache.org/jira/browse/HIVE-2868 Project: Hive Issue Type: Bug Affects Versions: 0.8.1 Reporter: Mauro Cazzari Fix For: 0.8.1 The INSERT INTO table statement still wipes out the target table, even though I read in more than once place that the statement should behave in append mode. Is this true? If not, is there an available or upcoming fix for this? Thanks! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3918) Normalize more CRLF line endings
[ https://issues.apache.org/jira/browse/HIVE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3918: -- Attachment: HIVE-3918.1.patch Normalize more CRLF line endings Key: HIVE-3918 URL: https://issues.apache.org/jira/browse/HIVE-3918 Project: Hive Issue Type: Bug Reporter: Mark Grover Assignee: Mark Grover Attachments: HIVE-3918.1.patch While I tried to fix files that had incompatible line endings in HIVE-3858, I missed some more files. It was most likely because HIVE-3858 was tested by me on Mac OS X but when I started using the github repository on a ubuntu box, I saw there are some more files in the Hive repo that have incorrect line endings. These files get added to the index automatically when a change is made to the local git repo. Once these cosmetic changes get committed, we would have gotten rid of all windows/old Mac OS style line endings in our Hive repository. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3918) Normalize more CRLF line endings
[ https://issues.apache.org/jira/browse/HIVE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Grover updated HIVE-3918: -- Status: Patch Available (was: Open) Normalize more CRLF line endings Key: HIVE-3918 URL: https://issues.apache.org/jira/browse/HIVE-3918 Project: Hive Issue Type: Bug Reporter: Mark Grover Assignee: Mark Grover Attachments: HIVE-3918.1.patch While I tried to fix files that had incompatible line endings in HIVE-3858, I missed some more files. It was most likely because HIVE-3858 was tested by me on Mac OS X but when I started using the github repository on a ubuntu box, I saw there are some more files in the Hive repo that have incorrect line endings. These files get added to the index automatically when a change is made to the local git repo. Once these cosmetic changes get committed, we would have gotten rid of all windows/old Mac OS style line endings in our Hive repository. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3918) Normalize more CRLF line endings
[ https://issues.apache.org/jira/browse/HIVE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561461#comment-13561461 ] Mark Grover commented on HIVE-3918: --- Review request: https://reviews.apache.org/r/9083/ Normalize more CRLF line endings Key: HIVE-3918 URL: https://issues.apache.org/jira/browse/HIVE-3918 Project: Hive Issue Type: Bug Reporter: Mark Grover Assignee: Mark Grover Attachments: HIVE-3918.1.patch While I tried to fix files that had incompatible line endings in HIVE-3858, I missed some more files. It was most likely because HIVE-3858 was tested by me on Mac OS X but when I started using the github repository on a ubuntu box, I saw there are some more files in the Hive repo that have incorrect line endings. These files get added to the index automatically when a change is made to the local git repo. Once these cosmetic changes get committed, we would have gotten rid of all windows/old Mac OS style line endings in our Hive repository. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3918) Normalize more CRLF line endings
[ https://issues.apache.org/jira/browse/HIVE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561472#comment-13561472 ] Mark Grover commented on HIVE-3918: --- Scratch that, new review request at https://reviews.apache.org/r/9084 Normalize more CRLF line endings Key: HIVE-3918 URL: https://issues.apache.org/jira/browse/HIVE-3918 Project: Hive Issue Type: Bug Reporter: Mark Grover Assignee: Mark Grover Attachments: HIVE-3918.1.patch While I tried to fix files that had incompatible line endings in HIVE-3858, I missed some more files. It was most likely because HIVE-3858 was tested by me on Mac OS X but when I started using the github repository on a ubuntu box, I saw there are some more files in the Hive repo that have incorrect line endings. These files get added to the index automatically when a change is made to the local git repo. Once these cosmetic changes get committed, we would have gotten rid of all windows/old Mac OS style line endings in our Hive repository. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3918) Normalize more CRLF line endings
[ https://issues.apache.org/jira/browse/HIVE-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561473#comment-13561473 ] Mark Grover commented on HIVE-3918: --- Some more context into this change: If you are using Ubuntu (like me) and use github, touching any file would add a bunch of files in your git index because they have windows style line endings. This started happening due to recent introduction of .gitattributes file (HIVE-3839). This change fixes those Windows style line endings to be Unix style line endings. Normalize more CRLF line endings Key: HIVE-3918 URL: https://issues.apache.org/jira/browse/HIVE-3918 Project: Hive Issue Type: Bug Reporter: Mark Grover Assignee: Mark Grover Attachments: HIVE-3918.1.patch While I tried to fix files that had incompatible line endings in HIVE-3858, I missed some more files. It was most likely because HIVE-3858 was tested by me on Mac OS X but when I started using the github repository on a ubuntu box, I saw there are some more files in the Hive repo that have incorrect line endings. These files get added to the index automatically when a change is made to the local git repo. Once these cosmetic changes get committed, we would have gotten rid of all windows/old Mac OS style line endings in our Hive repository. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
