[jira] [Commented] (HIVE-3152) Disallow certain character patterns in partition names
[
https://issues.apache.org/jira/browse/HIVE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476821#comment-13476821
]
Ashutosh Chauhan commented on HIVE-3152:
In ListenerEvent.java
{code}
- public ListenerEvent(boolean status, HMSHandler handler) {
+ public ListenerEvent(boolean status) {
{code}
This is backward incompatible change. In particular, this will break HCatalog.
Please hold off for commit.
Disallow certain character patterns in partition names
--
Key: HIVE-3152
URL: https://issues.apache.org/jira/browse/HIVE-3152
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: Andrew Poland
Assignee: Ivan Gorbachev
Priority: Minor
Labels: api-addition, configuration-addition
Attachments: jira-3152.0.patch
New event listener to allow metastore to reject a partition name if it
contains undesired character patterns such as unicode and commas.
Match pattern is implemented as a regular expression
Modifies append_partition to call a new MetaStorePreventListener
implementation, PreAppendPartitionEvent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3585) Integrate Trevni as another columnar oriented file format
alex gemini created HIVE-3585: - Summary: Integrate Trevni as another columnar oriented file format Key: HIVE-3585 URL: https://issues.apache.org/jira/browse/HIVE-3585 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.10.0 Reporter: alex gemini Priority: Minor add new avro module trevni as another columnar format.New columnar format need a columnar SerDe,seems fastutil is a good choice.the shark project use fastutil library as columnar serde library but it seems too large (almost 15m) for just a few primitive array collection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3581) get_json_object and json_tuple return null in the presence of new line characters
[ https://issues.apache.org/jira/browse/HIVE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476965#comment-13476965 ] Hudson commented on HIVE-3581: -- Integrated in Hive-trunk-h0.21 #1740 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1740/]) HIVE-3581 get_json_object and json_tuple return null in the presence of new line characters (Kevin Wilfong via namit) (Revision 1398654) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1398654 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFJson.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFJSONTuple.java * /hive/trunk/ql/src/test/queries/clientpositive/udf_get_json_object.q * /hive/trunk/ql/src/test/queries/clientpositive/udtf_json_tuple.q * /hive/trunk/ql/src/test/results/clientpositive/udf_get_json_object.q.out * /hive/trunk/ql/src/test/results/clientpositive/udtf_json_tuple.q.out get_json_object and json_tuple return null in the presence of new line characters - Key: HIVE-3581 URL: https://issues.apache.org/jira/browse/HIVE-3581 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3581.1.patch.txt This was introduced when these functions were updated to use Jackson. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: HIVE-3528
On Oct. 9, 2012, 1:06 a.m., Jakob Homan wrote: /trunk/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerializer.java, line 97 https://reviews.apache.org/r/7431/diff/3/?file=174393#file174393line97 Good optimization on the null check. Any reason not to put it as the first check in the method and potentially save the trip to the AvroSerde methods? As an aside, it wasn't an optimization. Since we obscure nullable fields, the null case gets handled by the same path as actual values. Without the check, the tests that make sure nullable records work explode with NPEs. - Sean --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7431/#review12257 --- On Oct. 16, 2012, 1:08 p.m., Sean Busbey wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7431/ --- (Updated Oct. 16, 2012, 1:08 p.m.) Review request for hive. Description --- Changes AvroSerDe to properly give the non-null schema to serialization routines when using Nullable complex types Diffs - /trunk/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerializer.java 1398763 /trunk/serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerializer.java 1398763 Diff: https://reviews.apache.org/r/7431/diff/ Testing --- Adds tests that check each of the Avro types that Serialization needs to use a user-provided schema, both as top level fields and as nested members of a complex type. Thanks, Sean Busbey
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477113#comment-13477113 ] Alan Gates commented on HIVE-2935: -- bq. I think this patch is in pretty good shape right now. Since this work is almsot completely decoupled from the rest of Hive I think the best option would be to commit it directly to trunk and address any bugs in separate tickets. However, in an earlier email to the PMC mailing list you argued that the size of this patch precludes the possibility of reviewing it, so I am in the process of breaking it into smaller pieces (as detailed above) and will start posting those pieces for review over the next couple of days. I don't think committing this to a separate branch will make the code any easier to review, and there are no established conventions within this project concerning development work on feature branches. I would prefer that we not make this patch the testcase for a new policy. Carl: I'm fine with checking directly into trunk and working from there. My goal isn't to create a branch but to figure out how to collaborate. It seems like people are starting to reviewing it. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #169
See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/169/ -- [...truncated 5456 lines...] [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hbase-handler/test/resources [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/hbase-handler/src/test/resources does not exist. init: [echo] Project: hbase-handler create-dirs: [echo] Project: pdk [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/pdk [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/pdk/classes [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/pdk/test [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/pdk/test/src [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/pdk/test/classes [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/pdk/test/resources [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/pdk/src/test/resources does not exist. init: [echo] Project: pdk create-dirs: [echo] Project: builtins [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/builtins [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/builtins/classes [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/builtins/test [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/builtins/test/src [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/builtins/test/classes [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/builtins/test/resources [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/builtins/src/test/resources does not exist. init: [echo] Project: builtins jar: [echo] Project: hive create-dirs: [echo] Project: shims [copy] Warning: /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/test/resources does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml [ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/3.4.3/zookeeper-3.4.3.jar ... [ivy:resolve] . (749kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.apache.zookeeper#zookeeper;3.4.3!zookeeper.jar (166ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/thrift/libthrift/0.7.0/libthrift-0.7.0.jar ... [ivy:resolve] ... (294kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.apache.thrift#libthrift;0.7.0!libthrift.jar (103ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar ... [ivy:resolve] (37kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-logging#commons-logging;1.0.4!commons-logging.jar (31ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/commons-logging/commons-logging-api/1.0.4/commons-logging-api-1.0.4.jar ... [ivy:resolve] (25kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] commons-logging#commons-logging-api;1.0.4!commons-logging-api.jar (31ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/com/google/guava/guava/r09/guava-r09.jar ... [ivy:resolve] (1117kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] com.google.guava#guava;r09!guava.jar (66ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar ... [ivy:resolve] .. (222kB) [ivy:resolve] .. (0kB) [ivy:resolve] [SUCCESSFUL ] org.codehaus.jackson#jackson-core-asl;1.8.8!jackson-core-asl.jar (46ms) [ivy:resolve] downloading http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar ... [ivy:resolve]
[jira] [Updated] (HIVE-3213) ODBC API enhancements
[ https://issues.apache.org/jira/browse/HIVE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-3213: -- Status: Patch Available (was: Open) ODBC API enhancements - Key: HIVE-3213 URL: https://issues.apache.org/jira/browse/HIVE-3213 Project: Hive Issue Type: Sub-task Components: ODBC Affects Versions: 0.10.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.10.0 Attachments: HIVE-1101-source.patch, HIVE-1101-source-with-thrift.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3213) ODBC API enhancements
[ https://issues.apache.org/jira/browse/HIVE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477151#comment-13477151 ] Prasad Mujumdar commented on HIVE-3213: --- Review request on https://reviews.apache.org/r/5685/ ODBC API enhancements - Key: HIVE-3213 URL: https://issues.apache.org/jira/browse/HIVE-3213 Project: Hive Issue Type: Sub-task Components: ODBC Affects Versions: 0.10.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.10.0 Attachments: HIVE-1101-source.patch, HIVE-1101-source-with-thrift.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3527) Allow CREATE TABLE LIKE command to take TBLPROPERTIES
[ https://issues.apache.org/jira/browse/HIVE-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477188#comment-13477188 ] Namit Jain commented on HIVE-3527: -- That works +1 Allow CREATE TABLE LIKE command to take TBLPROPERTIES - Key: HIVE-3527 URL: https://issues.apache.org/jira/browse/HIVE-3527 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3527.1.patch.txt, HIVE-3527.D5883.1.patch CREATE TABLE ... LIKE ... commands currently don't take TBLPROPERTIES. I think it would be a useful feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3527) Allow CREATE TABLE LIKE command to take TBLPROPERTIES
[ https://issues.apache.org/jira/browse/HIVE-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3527: - Attachment: hive.3527.2.patch Allow CREATE TABLE LIKE command to take TBLPROPERTIES - Key: HIVE-3527 URL: https://issues.apache.org/jira/browse/HIVE-3527 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3527.1.patch.txt, hive.3527.2.patch, HIVE-3527.D5883.1.patch CREATE TABLE ... LIKE ... commands currently don't take TBLPROPERTIES. I think it would be a useful feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-967) Implement show create table
[ https://issues.apache.org/jira/browse/HIVE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-967: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed, thanks Feng. Implement show create table - Key: HIVE-967 URL: https://issues.apache.org/jira/browse/HIVE-967 Project: Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Adam Kramer Assignee: Feng Lu Attachments: HIVE-967.2.patch.txt, HIVE-967.3.patch.txt, HIVE-967.4.patch.txt, HIVE-967.5.patch.txt, HIVE-967.6.patch.txt, HIVE-967.patch.txt, HiveShowCreateTable.jar, show_create.txt SHOW CREATE TABLE would be very useful in cases where you are trying to figure out the partitioning and/or bucketing scheme for a table. Perhaps this could be implemented by having new tables automatically SET PROPERTIES (create_command='raw text of the create statement')? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3551) Add more tests where output of sort merge join is sorted
[ https://issues.apache.org/jira/browse/HIVE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3551: Status: Open (was: Patch Available) Add more tests where output of sort merge join is sorted Key: HIVE-3551 URL: https://issues.apache.org/jira/browse/HIVE-3551 Project: Hive Issue Type: Test Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3551.1.patch As a follow-up to HIVE-3536, it would be good to have tests where a join is being performed between a single partition of the big table with multiple partitions of the small tables. The output should still be sorted -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-967) Implement show create table
[ https://issues.apache.org/jira/browse/HIVE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Lu updated HIVE-967: - Fix Version/s: 0.10.0 Implement show create table - Key: HIVE-967 URL: https://issues.apache.org/jira/browse/HIVE-967 Project: Hive Issue Type: New Feature Components: Metastore, Query Processor Reporter: Adam Kramer Assignee: Feng Lu Fix For: 0.10.0 Attachments: HIVE-967.2.patch.txt, HIVE-967.3.patch.txt, HIVE-967.4.patch.txt, HIVE-967.5.patch.txt, HIVE-967.6.patch.txt, HIVE-967.patch.txt, HiveShowCreateTable.jar, show_create.txt SHOW CREATE TABLE would be very useful in cases where you are trying to figure out the partitioning and/or bucketing scheme for a table. Perhaps this could be implemented by having new tables automatically SET PROPERTIES (create_command='raw text of the create statement')? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3560) Hive always prints a warning message when using remote metastore
[
https://issues.apache.org/jira/browse/HIVE-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford updated HIVE-3560:
--
Attachment: HIVE-3560_logging_tweaks.1.patch
Hive always prints a warning message when using remote metastore
Key: HIVE-3560
URL: https://issues.apache.org/jira/browse/HIVE-3560
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3560_logging_tweaks.1.patch
This issue was discovered in HIVE-2585 and more details about why this issue
was filed are available there.
Currently if one sets {{hive.metastore.uris}} the following error will always
be displayed:
{code}
2012-07-24 15:23:58,647 [main] WARN org.apache.hadoop.hive.conf.HiveConf -
DEPRECATED: Configuration property hive.metastore.local no longer has any
effect. Make sure to provide a valid value for hive.metastore.uris if you are
connecting to a remote metastore.
{code}
The reason is {{javax.jdo.option.ConnectionURL}} has a default value and will
never be null. I set this property in {{hive-site.xml}} and walked through
the configuration loading in a debugger. If the value is not empty it takes
effect, and is ignored if empty.
Since {{javax.jdo.option.ConnectionURL}} has a default and cannot be unset,
this warning will always be printed if someone sets {{hive.metastore.uris}}.
Per the review comments, the error message was added to reduce user
confusion, and prevent surprises by using the wrong metastore (either
embedded or remote). In {{HiveMetaStoreClient.java}} we see a very clear info
message printed saying that a remote metastore is used.
{code}
LOG.info(Trying to connect to metastore with URI + store);
...
LOG.info(Connected to metastore.);
{code}
Since we clearly communicate to the user that a remote metastore at the given
URI is being used we'll remove that message. Additionally, to further clarify
a remote metastore is used I'll make the following HiveMetaStoreClient
logging change:
{code}
LOG.debug(Trying to connect to remote HiveMetaStore: + store);
...
LOG.info(Connected to remote HiveMetaStore: + store);
{code}
The change is at debug level we print connection attempts, and always print
which remote HiveMetaStore we actually connected to.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3560) Hive always prints a warning message when using remote metastore
[
https://issues.apache.org/jira/browse/HIVE-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford updated HIVE-3560:
--
Status: Patch Available (was: Open)
Hive always prints a warning message when using remote metastore
Key: HIVE-3560
URL: https://issues.apache.org/jira/browse/HIVE-3560
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3560_logging_tweaks.1.patch
This issue was discovered in HIVE-2585 and more details about why this issue
was filed are available there.
Currently if one sets {{hive.metastore.uris}} the following error will always
be displayed:
{code}
2012-07-24 15:23:58,647 [main] WARN org.apache.hadoop.hive.conf.HiveConf -
DEPRECATED: Configuration property hive.metastore.local no longer has any
effect. Make sure to provide a valid value for hive.metastore.uris if you are
connecting to a remote metastore.
{code}
The reason is {{javax.jdo.option.ConnectionURL}} has a default value and will
never be null. I set this property in {{hive-site.xml}} and walked through
the configuration loading in a debugger. If the value is not empty it takes
effect, and is ignored if empty.
Since {{javax.jdo.option.ConnectionURL}} has a default and cannot be unset,
this warning will always be printed if someone sets {{hive.metastore.uris}}.
Per the review comments, the error message was added to reduce user
confusion, and prevent surprises by using the wrong metastore (either
embedded or remote). In {{HiveMetaStoreClient.java}} we see a very clear info
message printed saying that a remote metastore is used.
{code}
LOG.info(Trying to connect to metastore with URI + store);
...
LOG.info(Connected to metastore.);
{code}
Since we clearly communicate to the user that a remote metastore at the given
URI is being used we'll remove that message. Additionally, to further clarify
a remote metastore is used I'll make the following HiveMetaStoreClient
logging change:
{code}
LOG.debug(Trying to connect to remote HiveMetaStore: + store);
...
LOG.info(Connected to remote HiveMetaStore: + store);
{code}
The change is at debug level we print connection attempts, and always print
which remote HiveMetaStore we actually connected to.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3560) Hive always prints a warning message when using remote metastore
[
https://issues.apache.org/jira/browse/HIVE-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Travis Crawford updated HIVE-3560:
--
Attachment: HIVE-3560_logging_tweaks.2.patch
Looking at this patch with [~pengfeng] he had the good suggestion we should log
the Trying to connect line at the same level as the Connected line. The
reason is for making calls out to another service we should always have the
start/end log messages at the same level or higher, so they can be matched up
in the logs.
Hive always prints a warning message when using remote metastore
Key: HIVE-3560
URL: https://issues.apache.org/jira/browse/HIVE-3560
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.10.0
Reporter: Travis Crawford
Assignee: Travis Crawford
Attachments: HIVE-3560_logging_tweaks.1.patch,
HIVE-3560_logging_tweaks.2.patch
This issue was discovered in HIVE-2585 and more details about why this issue
was filed are available there.
Currently if one sets {{hive.metastore.uris}} the following error will always
be displayed:
{code}
2012-07-24 15:23:58,647 [main] WARN org.apache.hadoop.hive.conf.HiveConf -
DEPRECATED: Configuration property hive.metastore.local no longer has any
effect. Make sure to provide a valid value for hive.metastore.uris if you are
connecting to a remote metastore.
{code}
The reason is {{javax.jdo.option.ConnectionURL}} has a default value and will
never be null. I set this property in {{hive-site.xml}} and walked through
the configuration loading in a debugger. If the value is not empty it takes
effect, and is ignored if empty.
Since {{javax.jdo.option.ConnectionURL}} has a default and cannot be unset,
this warning will always be printed if someone sets {{hive.metastore.uris}}.
Per the review comments, the error message was added to reduce user
confusion, and prevent surprises by using the wrong metastore (either
embedded or remote). In {{HiveMetaStoreClient.java}} we see a very clear info
message printed saying that a remote metastore is used.
{code}
LOG.info(Trying to connect to metastore with URI + store);
...
LOG.info(Connected to metastore.);
{code}
Since we clearly communicate to the user that a remote metastore at the given
URI is being used we'll remove that message. Additionally, to further clarify
a remote metastore is used I'll make the following HiveMetaStoreClient
logging change:
{code}
LOG.debug(Trying to connect to remote HiveMetaStore: + store);
...
LOG.info(Connected to remote HiveMetaStore: + store);
{code}
The change is at debug level we print connection attempts, and always print
which remote HiveMetaStore we actually connected to.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477371#comment-13477371 ] Thejas M Nair commented on HIVE-2935: - Carl, The existing HiveServer also supports the MetaStore api's (the HiveServer thrift service inherits MetaStore thrift service). This means that we need to only one server for both. In case of this patch, the server does not include MetaStore service . I think we should have the HiveServer2 thrift service inherit metastore service as well. This will avoid increasing the complexity of managing clusters where you want both interfaces to be supported. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3499) add tests to use bucketing metadata for partitions
[ https://issues.apache.org/jira/browse/HIVE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477463#comment-13477463 ] Kevin Wilfong commented on HIVE-3499: - Sorry it took me so long to get this one in. Could you update the test, it seems to have gotten out of date. add tests to use bucketing metadata for partitions -- Key: HIVE-3499 URL: https://issues.apache.org/jira/browse/HIVE-3499 Project: Hive Issue Type: Test Components: Tests Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3499.1.patch What happens if the bucketing metadata is different for different partitions ? I dont mean, different number of buckets, but what happens if the partitions are bucketed on different keys. Do we get wrong results ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3499) add tests to use bucketing metadata for partitions
[ https://issues.apache.org/jira/browse/HIVE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3499: Status: Open (was: Patch Available) add tests to use bucketing metadata for partitions -- Key: HIVE-3499 URL: https://issues.apache.org/jira/browse/HIVE-3499 Project: Hive Issue Type: Test Components: Tests Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3499.1.patch What happens if the bucketing metadata is different for different partitions ? I dont mean, different number of buckets, but what happens if the partitions are bucketed on different keys. Do we get wrong results ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3587) Lost data during INSERT query
Jim Krehl created HIVE-3587: --- Summary: Lost data during INSERT query Key: HIVE-3587 URL: https://issues.apache.org/jira/browse/HIVE-3587 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Ubuntu 10.04 Hadoop MapReduce 0.20.2 Cloudera 4.1.0 3 data/task nodes Reporter: Jim Krehl Priority: Critical I'm trying to load a table using an INSERT query [1]. Not all the data is making it from the original table into the new table. The query generates 2 jobs. The first job takes about 45 minutes with mapred.mapper.class = org.apache.hadoop.hive.ql.io.rcfile.merge.RCFileMergeMapper and the second takes ~10 seconds with mapred.mapper.class = org.apache.hadoop.hive.ql.exec.ExecMapper. Toward the end ( 2 minutes) of the first job a number of IOExceptions are raised [2]. The exceptions are only raised in the last mapper task to complete, the other mapper tasks complete successfully. The exceptions indicate that an expected temporary file is missing. The second jobs completes entirely successfully. According to the task tracker web interface the jobs are run sequentially with no overlap. However, the second job spawns a number of tasks which rename the very temporary files that are the cause of the failures in the first job [3]. [1] https://cwiki.apache.org/Hive/languagemanual-dml.html#LanguageManualDML-InsertingdataintoHiveTablesfromqueries [2] Example: ERROR org.apache.hadoop.hdfs.DFSClient: Failed to close file /tmp/hive-hive/hive_2012-10-15_13-45-21_245_1936216192130095423/_task_tmp.-ext-10002/month=2012-01/_tmp.00_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /tmp/hive-hive/hive_2012-10-15_13-45-21_245_1936216192130095423/_task_tmp.-ext-10002/month=2012-01/_tmp.00_1 File does not exist. Holder DFSClient_NONMAPREDUCE_-672101740_1 does not have any open files. [3] Example: 2012-10-16 15:36:57,605 INFO RCFileMergeMapper: renamed path hdfs://analysis-hadoop-master/tmp/hive-hive/hive_2012-10-16_14-48-47_633_7033175453889409541/_task_tmp.-ext-1/month=2012-01/_tmp.11_0 to hdfs://analysis-hadoop-master/tmp/hive-hive/hive_2012-10-16_14-48-47_633_7033175453889409541/_tmp.-ext-1/month=2012-01/11_0 . File size is 3482 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3587) Lost data during INSERT query
[ https://issues.apache.org/jira/browse/HIVE-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Krehl updated HIVE-3587: Description: I'm trying to load a table using an INSERT query (1). Not all the data is making it from the original table into the new table. The query generates 2 jobs. The first job takes about 45 minutes with mapred.mapper.class = org.apache.hadoop.hive.ql.io.rcfile.merge.RCFileMergeMapper and the second takes ~10 seconds with mapred.mapper.class = org.apache.hadoop.hive.ql.exec.ExecMapper. Toward the end ( 2 minutes) of the first job a number of IOExceptions are raised (2). The exceptions are only raised in the last mapper task to complete, the other mapper tasks complete successfully. The exceptions indicate that an expected temporary file is missing. The second jobs completes entirely successfully. According to the task tracker web interface the jobs are run sequentially with no overlap. However, the second job spawns a number of tasks which rename the very temporary files that are the cause of the failures in the first job (3). (1) https://cwiki.apache.org/Hive/languagemanual-dml.html#LanguageManualDML-InsertingdataintoHiveTablesfromqueries (2) Example: ERROR org.apache.hadoop.hdfs.DFSClient: Failed to close file /tmp/hive-hive/hive_2012-10-15_13-45-21_245_1936216192130095423/_task_tmp.-ext-10002/month=2012-01/_tmp.00_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /tmp/hive-hive/hive_2012-10-15_13-45-21_245_1936216192130095423/_task_tmp.-ext-10002/month=2012-01/_tmp.00_1 File does not exist. Holder DFSClient_NONMAPREDUCE_-672101740_1 does not have any open files. (3) Example: 2012-10-16 15:36:57,605 INFO RCFileMergeMapper: renamed path hdfs://analysis-hadoop-master/tmp/hive-hive/hive_2012-10-16_14-48-47_633_7033175453889409541/_task_tmp.-ext-1/month=2012-01/_tmp.11_0 to hdfs://analysis-hadoop-master/tmp/hive-hive/hive_2012-10-16_14-48-47_633_7033175453889409541/_tmp.-ext-1/month=2012-01/11_0 . File size is 3482 was: I'm trying to load a table using an INSERT query [1]. Not all the data is making it from the original table into the new table. The query generates 2 jobs. The first job takes about 45 minutes with mapred.mapper.class = org.apache.hadoop.hive.ql.io.rcfile.merge.RCFileMergeMapper and the second takes ~10 seconds with mapred.mapper.class = org.apache.hadoop.hive.ql.exec.ExecMapper. Toward the end ( 2 minutes) of the first job a number of IOExceptions are raised [2]. The exceptions are only raised in the last mapper task to complete, the other mapper tasks complete successfully. The exceptions indicate that an expected temporary file is missing. The second jobs completes entirely successfully. According to the task tracker web interface the jobs are run sequentially with no overlap. However, the second job spawns a number of tasks which rename the very temporary files that are the cause of the failures in the first job [3]. [1] https://cwiki.apache.org/Hive/languagemanual-dml.html#LanguageManualDML-InsertingdataintoHiveTablesfromqueries [2] Example: ERROR org.apache.hadoop.hdfs.DFSClient: Failed to close file /tmp/hive-hive/hive_2012-10-15_13-45-21_245_1936216192130095423/_task_tmp.-ext-10002/month=2012-01/_tmp.00_1 org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /tmp/hive-hive/hive_2012-10-15_13-45-21_245_1936216192130095423/_task_tmp.-ext-10002/month=2012-01/_tmp.00_1 File does not exist. Holder DFSClient_NONMAPREDUCE_-672101740_1 does not have any open files. [3] Example: 2012-10-16 15:36:57,605 INFO RCFileMergeMapper: renamed path hdfs://analysis-hadoop-master/tmp/hive-hive/hive_2012-10-16_14-48-47_633_7033175453889409541/_task_tmp.-ext-1/month=2012-01/_tmp.11_0 to hdfs://analysis-hadoop-master/tmp/hive-hive/hive_2012-10-16_14-48-47_633_7033175453889409541/_tmp.-ext-1/month=2012-01/11_0 . File size is 3482 Lost data during INSERT query - Key: HIVE-3587 URL: https://issues.apache.org/jira/browse/HIVE-3587 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Ubuntu 10.04 Hadoop MapReduce 0.20.2 Cloudera 4.1.0 3 data/task nodes Reporter: Jim Krehl Priority: Critical I'm trying to load a table using an INSERT query (1). Not all the data is making it from the original table into the new table. The query generates 2 jobs. The first job takes about 45 minutes with mapred.mapper.class = org.apache.hadoop.hive.ql.io.rcfile.merge.RCFileMergeMapper and the second takes ~10 seconds with mapred.mapper.class = org.apache.hadoop.hive.ql.exec.ExecMapper. Toward the end ( 2 minutes) of the first job a number of IOExceptions are raised (2). The
Hive-trunk-h0.21 - Build # 1741 - Still Failing
Changes for Build #1739 Changes for Build #1740 [namit] HIVE-3581 get_json_object and json_tuple return null in the presence of new line characters (Kevin Wilfong via namit) Changes for Build #1741 1 tests failed. REGRESSION: org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Error Message: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. Stack Trace: junit.framework.AssertionFailedError: Unexpected exception See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. at junit.framework.Assert.fail(Assert.java:47) at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1(TestNegativeCliDriver.java:11317) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:232) at junit.framework.TestSuite.run(TestSuite.java:227) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1741) Status: Still Failing Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1741/ to view the results.
[jira] [Assigned] (HIVE-3557) Access to external URLs in hivetest.py
[ https://issues.apache.org/jira/browse/HIVE-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gorbachev reassigned HIVE-3557: Assignee: Ivan Gorbachev Access to external URLs in hivetest.py -- Key: HIVE-3557 URL: https://issues.apache.org/jira/browse/HIVE-3557 Project: Hive Issue Type: Improvement Reporter: Ivan Gorbachev Assignee: Ivan Gorbachev Migrate all non-HTTP urls to HTTP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3588) Get Hive to work with hbase 94
Vikram Dixit K created HIVE-3588: Summary: Get Hive to work with hbase 94 Key: HIVE-3588 URL: https://issues.apache.org/jira/browse/HIVE-3588 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.9.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-3588.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3588) Get Hive to work with hbase 94
[ https://issues.apache.org/jira/browse/HIVE-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3588: - Attachment: HIVE-3588.patch Get Hive to work with hbase 94 -- Key: HIVE-3588 URL: https://issues.apache.org/jira/browse/HIVE-3588 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.9.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-3588.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3588) Get Hive to work with hbase 94
[ https://issues.apache.org/jira/browse/HIVE-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-3588: - Status: Patch Available (was: Open) Get Hive to work with hbase 94 -- Key: HIVE-3588 URL: https://issues.apache.org/jira/browse/HIVE-3588 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.9.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-3588.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3537) release locks at the end of move tasks
[ https://issues.apache.org/jira/browse/HIVE-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3537: Status: Open (was: Patch Available) release locks at the end of move tasks -- Key: HIVE-3537 URL: https://issues.apache.org/jira/browse/HIVE-3537 Project: Hive Issue Type: Bug Components: Locking, Query Processor Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3537.1.patch Look at HIVE-3106 for details. In order to make sure that concurrency is not an issue for multi-table inserts, the current option is to introduce a dependency task, which thereby delays the creation of all partitions. It would be desirable to release the locks for the outputs as soon as the move task is completed. That way, for multi-table inserts, the concurrency can be enabled without delaying any table. Currently, the movetask contains a input/output, but they do not seem to be populated correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2539) Enable passing username/password via JDBC
[ https://issues.apache.org/jira/browse/HIVE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2539: - Status: Open (was: Patch Available) Please post a review request on reviewboard or phabricator. Thanks. Enable passing username/password via JDBC - Key: HIVE-2539 URL: https://issues.apache.org/jira/browse/HIVE-2539 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.7.1 Reporter: Sriram Krishnan Assignee: chunqing xie Labels: patch Attachments: HIVE-2539.PATCH Changing the username and/or the password seems to have no effect (also confirmed here: https://cwiki.apache.org/Hive/hivejdbcinterface.html). Connection con = DriverManager.getConnection(jdbc:hive://localhost:1/default, , ); Would be beneficial to pass the username/password via JDBC - and also for the server to honor the username password being passed (may be dependent of that being fixed first). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3576) Regression: ALTER TABLE DROP IF EXISTS PARTITION throws a SemanticException if Partition is not found
[
https://issues.apache.org/jira/browse/HIVE-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3576:
-
Component/s: Metastore
Regression: ALTER TABLE DROP IF EXISTS PARTITION throws a SemanticException
if Partition is not found
-
Key: HIVE-3576
URL: https://issues.apache.org/jira/browse/HIVE-3576
Project: Hive
Issue Type: Bug
Components: Metastore, Query Processor
Affects Versions: 0.9.0
Reporter: Harsh J
Doing a simple {{ALTER TABLE testtable DROP IF EXISTS
PARTITION(dt=NONEXISTENTPARTITION)}} fails with a SemanticException of the
10006 kind (INVALID_PARTITION).
This does not respect the {{hive.exec.drop.ignorenonexistent}} condition
either, since there are no if-check-wraps around this area, when fetching
partitions from the store.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
