[jira] [Created] (HIVE-11439) PERCENTILE_APPROX is inconsistent with PERCENTILE even for a small number of data points
Mikhail Bautin created HIVE-11439: - Summary: PERCENTILE_APPROX is inconsistent with PERCENTILE even for a small number of data points Key: HIVE-11439 URL: https://issues.apache.org/jira/browse/HIVE-11439 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-4525: - Status: Patch Available (was: Open) > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin > Attachments: D10755.1.patch, D10755.2.patch > > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13661870#comment-13661870 ] Mikhail Bautin commented on HIVE-4525: -- Test results with and without this patch differ only by a spurious failure of a ZK-related test that is not affected by the changes here. *** 3838,3843 --- 3838,3845 [junit] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 [junit] Running org.apache.hadoop.hive.serde2.dynamic_type.TestDynamicSerDe [junit] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0 + [junit] Running org.apache.hadoop.hive.serde2.io.TestTimestampWritable + [junit] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0 [junit] Running org.apache.hadoop.hive.serde2.lazy.TestLazyArrayMapStruct [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0 [junit] Running org.apache.hadoop.hive.serde2.lazy.TestLazyPrimitive *** *** 3901,3906 [junit] Running org.apache.hcatalog.hbase.snapshot.TestZNodeSetUp [junit] Tests run: 0, Failures: 0, Errors: 2, Skipped: 0 [junit] Running org.apache.hcatalog.hbase.snapshot.lock.WriteLockTest ! [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [junit] Running org.apache.hcatalog.hbase.snapshot.lock.ZNodeNameTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 --- 3903,3908 [junit] Running org.apache.hcatalog.hbase.snapshot.TestZNodeSetUp [junit] Tests run: 0, Failures: 0, Errors: 2, Skipped: 0 [junit] Running org.apache.hcatalog.hbase.snapshot.lock.WriteLockTest ! [junit] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 [junit] Running org.apache.hcatalog.hbase.snapshot.lock.ZNodeNameTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 + set +x Committers: could you please take a look and consider committing this? Cc [~ashutoshc], [~owen.omalley], [~cwsteinbach]. Thanks! > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin > Attachments: D10755.1.patch, D10755.2.patch > > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659128#comment-13659128 ] Mikhail Bautin commented on HIVE-4525: -- I am not quite sure how to solve the backward compatibility issue in the "writable" part of {{TimestampWritable}} code ({{write}}/{{readFields}}) by switching to a unified nanosecond-timestamp-as-long format. If {{readFields}} is presented with eight bytes, would it interpret them as a four-byte int followed by a VInt or as a long nanosecond timestamp? Would it attempt to do the former and revert to the latter if there are inconsistencies? What if the bytes of a long nanosecond timestamp also happen to represent a valid legacy (int/VInt) timestamp? In my patch, I try to maintain backward compatibility as much as possible. If a timestamp is in the range that can be represented by the old format, it is serialized using the old format. The extended format I've proposed and implemented for the full timestamp range builds on top of the existing one and can be unambiguously distinguished from the old format by examining serialized bytes. In addition, the included test, {{TestTimestampWritable}}, tests both the old and the new (extended format), as well as double/BigDecimal conversion, getters/setters/constructors and everything else I could test in {{TimestampWritable}}. I am sure there is a way to handle vector optimizations for timestamps in a backward-compatible way, and I don't think this patch would make it much more complicated than it already is. However, vectorized computations are a performance optimization, while this issue is a correctness fix. Currently, timestamps outside of the ~1970-2038 range would be silently corrupted in some queries, and this patch successfully fixes that. It is also pretty small and immediately available. > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: D10755.1.patch > > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13656271#comment-13656271 ] Mikhail Bautin commented on HIVE-4525: -- [~ehans]: switching to long nanosecond timestamps would definitely be a much nicer solution, but don't you think it would break backward-compatibility for timestamps serialized using the old format? > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: D10755.1.patch > > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13656141#comment-13656141 ] Mikhail Bautin commented on HIVE-4525: -- Correction to the design of this feature (I can't edit comments because of permissions, so adding another comment). In case the seconds field needs more than 31 bit, the first VInt is {{-1-reversedDecimal}} regardless of whether {{reversedDecimal}} is zero or not. > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: D10755.1.patch > > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
cannot edit comments on Hive JIRA
Hello, I cannot edit my comments on JIRA. In some cases it is desirable to edit comments, e.g. to be able to correct a feature's design description. Could one of Hive JIRA admins please enable comment editing for my account? Thanks in advance! Mikhail
Re: support for timestamps before 1970 or after 2038
Since this does seem to be a problem in Hive, I went ahead and created a JIRA with a design proposal of a backward-compatible solution. https://issues.apache.org/jira/browse/HIVE-4525 Years earlier than 1970 in particular are an important case for supporting historical data in many business applications. Thanks, Mikhail On Wed, May 8, 2013 at 3:15 PM, Mikhail Bautin < bautin.mailing.li...@gmail.com> wrote: > Hello, > > Are there plans to support timestamps that cannot be represented by a > signed 32-bit integer number of seconds since the UNIX epoch? (i.e. those > before 1970 or after a certain point in 2038). Currently Hive's behavior > regarding these timestamps is inconsistent, because it is possible to > insert them into a table, but Hive does not handle them properly. Trying to > serialize and deserialize the "1969-12-31 23:59:59" timestamp using > TimestampWritable results in a "2038-01-19 03:14:07" timestamp. > > Thanks, > Mikhail > >
[jira] [Commented] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13652709#comment-13652709 ] Mikhail Bautin commented on HIVE-4525: -- Also, the binary-sortable representation of timestamps would have to change to accommodate additional high-order bits. If a 4-byte second-precision timestamp covers 68 years (or 136 if signed), by adding one most-significant byte we can cover 17408 (or 34816) years, which is good enough for all practical purposes. > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
[ https://issues.apache.org/jira/browse/HIVE-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13652676#comment-13652676 ] Mikhail Bautin commented on HIVE-4525: -- h4. Design proposal We have to be able to read the current {{TimestampWritable}}-serializable format for backward-compatibility, and write the format recognizable by the current {{TimestampWritable}} implementation for timestamps within the currently supported range. We can use the negative range of the {{VInt}} in the binary representation of the timestamp that normally represents the reversed decimal part to indicate the presence of an additional {{VInt}} field that stores the remaining bits of the {{seconds}} number (i.e. {{seconds >> 31}}). The meaning of the 7th bit of the first byte then changes from "has decimal" to "has decimal or >31 bits of seconds". The following table summarizes the four logical cases of timestamp serialization. The first two are backward-compatible. The second two cases are unsupported by the current format, so they will not be recognized by the current version. || Seconds need >31 bits || Has decimal || 7th bit of the first byte || First VInt || Second VInt || | No | No | {{0}} | N/A | N/A | | No | Yes | {{1}} | {{reversedDecimal}} | N/A | | Yes | No | {{1}} | {{-1}} | {{seconds >> 31}} | | Yes | Yes | {{1}} | {{-2 - reversedDecimal}} | {{seconds >> 31}} | > Support timestamps earlier than 1970 and later than 2038 > > > Key: HIVE-4525 > URL: https://issues.apache.org/jira/browse/HIVE-4525 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > > TimestampWritable currently serializes timestamps using the lower 31 bits of > an int. This does not allow to store timestamps earlier than 1970 or later > than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4525) Support timestamps earlier than 1970 and later than 2038
Mikhail Bautin created HIVE-4525: Summary: Support timestamps earlier than 1970 and later than 2038 Key: HIVE-4525 URL: https://issues.apache.org/jira/browse/HIVE-4525 Project: Hive Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin TimestampWritable currently serializes timestamps using the lower 31 bits of an int. This does not allow to store timestamps earlier than 1970 or later than a certain point in 2038. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
support for timestamps before 1970 or after 2038
Hello, Are there plans to support timestamps that cannot be represented by a signed 32-bit integer number of seconds since the UNIX epoch? (i.e. those before 1970 or after a certain point in 2038). Currently Hive's behavior regarding these timestamps is inconsistent, because it is possible to insert them into a table, but Hive does not handle them properly. Trying to serialize and deserialize the "1969-12-31 23:59:59" timestamp using TimestampWritable results in a "2038-01-19 03:14:07" timestamp. Thanks, Mikhail
[jira] [Commented] (HIVE-4220) TimestampWritable.toString throws array index exception sometimes
[ https://issues.apache.org/jira/browse/HIVE-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13652133#comment-13652133 ] Mikhail Bautin commented on HIVE-4220: -- [~navis]: I think HIVE-4516 solves the same problem but in somewhat simpler way (using ThreadLocal). Could you please take a look at the patch there and let us know what you think? [~ashutoshc]: I think it is reasonable to assume at this point that Hive primitives, especially as low-level as TimestampWritable, have to be thread-safe. This is not only required by third-party low-latency query processing systems such as AmpLab's Shark, but also by the effort in the Hive community itself to speed up query processing (e.g. http://hortonworks.com/blog/introducing-tez-faster-hadoop-processing/) that I believe will inevitably require keeping pre-existing multi-threaded "executor" JVMs around. > TimestampWritable.toString throws array index exception sometimes > - > > Key: HIVE-4220 > URL: https://issues.apache.org/jira/browse/HIVE-4220 > Project: Hive > Issue Type: Bug >Reporter: Navis >Assignee: Navis > Attachments: HIVE-4220.D9669.1.patch > > > {noformat} > org.apache.hive.service.cli.HiveSQLException: java.io.IOException: > java.lang.ArrayIndexOutOfBoundsException: 45 > at > org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:215) > at > org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:170) > at > org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:288) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:348) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 45 > at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:194) > at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1449) > at > org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:193) > ... 11 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: 45 > at > sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:436) > at > java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2081) > at > java.util.GregorianCalendar.computeFields(GregorianCalendar.java:1996) > at java.util.Calendar.setTimeInMillis(Calendar.java:1110) > at java.util.Calendar.setTime(Calendar.java:1076) > at java.text.SimpleDateFormat.format(SimpleDateFormat.java:875) > at java.text.SimpleDateFormat.format(SimpleDateFormat.java:868) > at java.text.DateFormat.format(DateFormat.java:316) > at > org.apache.hadoop.hive.serde2.io.TimestampWritable.toString(TimestampWritable.java:327) > at > org.apache.hadoop.hive.serde2.lazy.LazyTimestamp.writeUTF8(LazyTimestamp.java:95) > at > org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:234) > at > org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:427) > at > org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:381) > at > org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:365) > at > org.apache.hadoop.hive.ql.exec.ListSinkOperator.processOp(ListSinkOperator.java:96) > at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:487) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:821) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84) > at org.apache.hadoop.hive.
[jira] [Commented] (HIVE-3930) Generate and publish source jars
[ https://issues.apache.org/jira/browse/HIVE-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601576#comment-13601576 ] Mikhail Bautin commented on HIVE-3930: -- There is a patch for a the AMP Lab version of Hive 0.9: https://github.com/amplab/hive/pull/10/files > Generate and publish source jars > > > Key: HIVE-3930 > URL: https://issues.apache.org/jira/browse/HIVE-3930 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin > > Hive should generate and publish source jars to Maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3974) RCFile needs to export a method to flush buffered records and provide a flush callback
Mikhail Bautin created HIVE-3974: Summary: RCFile needs to export a method to flush buffered records and provide a flush callback Key: HIVE-3974 URL: https://issues.apache.org/jira/browse/HIVE-3974 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Sometimes it is desirable to make sure all buffered records have been persisted, and take some action when that happens. The easiest way to achieve this is to make the flushRecords() method public and add a callback for when a flush happens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561822#comment-13561822 ] Mikhail Bautin commented on HIVE-3809: -- [~ashutoshc]: any chance this fix could be committed to branch 0.9 as well, if this is not too difficult? Thanks! Patch for 0.9: https://issues.apache.org/jira/secure/attachment/12562623/0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Critical > Fix For: 0.11.0 > > Attachments: > 0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch, > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3930) Generate and publish source jars
[ https://issues.apache.org/jira/browse/HIVE-3930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561386#comment-13561386 ] Mikhail Bautin commented on HIVE-3930: -- [~ashutoshc]: HIVE-3830 seems to talk about generating a source-only release (a tarball, as far as I understand), while here I am talking about generating source jars that could be distributed through Maven to make it easier to develop projects that depend on Hive. > Generate and publish source jars > > > Key: HIVE-3930 > URL: https://issues.apache.org/jira/browse/HIVE-3930 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin >Priority: Minor > > Hive should generate and publish source jars to Maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3930) Generate and publish source jars
Mikhail Bautin created HIVE-3930: Summary: Generate and publish source jars Key: HIVE-3930 URL: https://issues.apache.org/jira/browse/HIVE-3930 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Hive should generate and publish source jars to Maven. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3913) Possible deadlock in ZK lock manager
[ https://issues.apache.org/jira/browse/HIVE-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13559830#comment-13559830 ] Mikhail Bautin commented on HIVE-3913: -- [~ashutoshc]: thanks for pointing this out. I took a look at HIVE-3537, and I think this is a much simpler fix for a situation that I observed when the client attempts to acquire a lock, but the ZK connection is closed for some reason and no reconnection is attempted. > Possible deadlock in ZK lock manager > > > Key: HIVE-3913 > URL: https://issues.apache.org/jira/browse/HIVE-3913 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Critical > Attachments: D8097.1.patch > > > ZK Hive lock manager can get into a state when the connection is closed, but > no reconnection is attempted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3036) hive should support BigDecimal datatype
[ https://issues.apache.org/jira/browse/HIVE-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13557476#comment-13557476 ] Mikhail Bautin commented on HIVE-3036: -- Actually this seems to be a duplicate of HIVE-2693. > hive should support BigDecimal datatype > --- > > Key: HIVE-3036 > URL: https://issues.apache.org/jira/browse/HIVE-3036 > Project: Hive > Issue Type: New Feature > Components: Query Processor, Types >Affects Versions: 0.7.1, 0.8.0, 0.8.1 >Reporter: Anurag Tangri > > hive has support for big int but people have use cases where they need > decimal precision to a big value. > Values in question are like decimal(x,y). > for eg. decimal of form (17,6) which cannot be represented by float/double. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3913) Possible deadlock in ZK lock manager
Mikhail Bautin created HIVE-3913: Summary: Possible deadlock in ZK lock manager Key: HIVE-3913 URL: https://issues.apache.org/jira/browse/HIVE-3913 Project: Hive Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Critical ZK Hive lock manager can get into a state when the connection is closed, but no reconnection is attempted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3855) Fix tests with non-deterministic output, e.g. limit without order by (patch by Reynold Xin)
[ https://issues.apache.org/jira/browse/HIVE-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3855: - Attachment: Fixed-hive-tests-with-non-deterministic-output-e.g.-.patch Patch for branch-0.9. > Fix tests with non-deterministic output, e.g. limit without order by (patch > by Reynold Xin) > --- > > Key: HIVE-3855 > URL: https://issues.apache.org/jira/browse/HIVE-3855 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin >Priority: Minor > Attachments: > Fixed-hive-tests-with-non-deterministic-output-e.g.-.patch > > > Fix tests with non-deterministic output, e.g. limit without order by (patch > by Reynold Xin). Needed by the Shark project (http://shark.cs.berkeley.edu/). > https://github.com/amplab/hive/commit/31be6d8a61f454293f13e0d8bb2d88c3dce19c44 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3855) Fix tests with non-deterministic output, e.g. limit without order by (patch by Reynold Xin)
Mikhail Bautin created HIVE-3855: Summary: Fix tests with non-deterministic output, e.g. limit without order by (patch by Reynold Xin) Key: HIVE-3855 URL: https://issues.apache.org/jira/browse/HIVE-3855 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Fix tests with non-deterministic output, e.g. limit without order by (patch by Reynold Xin). Needed by the Shark project (http://shark.cs.berkeley.edu/). https://github.com/amplab/hive/commit/31be6d8a61f454293f13e0d8bb2d88c3dce19c44 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3854) Add a condition in input_part8 to make the result deterministic (patch by Reynold Xin)
Mikhail Bautin created HIVE-3854: Summary: Add a condition in input_part8 to make the result deterministic (patch by Reynold Xin) Key: HIVE-3854 URL: https://issues.apache.org/jira/browse/HIVE-3854 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Attachments: 0001-Added-a-condition-in-input_part8-to-make-the-result-.patch Add a condition in input_part8 to make the result deterministic (patch by Reynold Xin). Needed for the Shark project. https://github.com/amplab/hive/commit/3e7606bb4793e000e7a0e51b8565bf1bec2dbf33 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3854) Add a condition in input_part8 to make the result deterministic (patch by Reynold Xin)
[ https://issues.apache.org/jira/browse/HIVE-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3854: - Attachment: 0001-Added-a-condition-in-input_part8-to-make-the-result-.patch > Add a condition in input_part8 to make the result deterministic (patch by > Reynold Xin) > -- > > Key: HIVE-3854 > URL: https://issues.apache.org/jira/browse/HIVE-3854 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin >Priority: Minor > Attachments: > 0001-Added-a-condition-in-input_part8-to-make-the-result-.patch > > > Add a condition in input_part8 to make the result deterministic (patch by > Reynold Xin). Needed for the Shark project. > https://github.com/amplab/hive/commit/3e7606bb4793e000e7a0e51b8565bf1bec2dbf33 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3809: - Attachment: 0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Critical > Fix For: 0.11.0 > > Attachments: > 0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch, > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540660#comment-13540660 ] Mikhail Bautin commented on HIVE-3809: -- [~ashutoshc]: thank you for committing this to trunk. Could you please also commit this to branch 0.9? Patch for branch 0.9: https://issues.apache.org/jira/secure/attachment/12562623/0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Critical > Fix For: 0.11.0 > > Attachments: > 0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch, > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3813) Allow publishing artifacts to an arbitrary remote repository
[ https://issues.apache.org/jira/browse/HIVE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3813: - Attachment: 0001-HIVE-3813-Allow-publishing-artifacts-to-an-arbitrary.patch Attaching a manually generated patch. > Allow publishing artifacts to an arbitrary remote repository > > > Key: HIVE-3813 > URL: https://issues.apache.org/jira/browse/HIVE-3813 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin > Attachments: > 0001-HIVE-3813-Allow-publishing-artifacts-to-an-arbitrary.patch, D7455.1.patch > > > Allow publishing artifacts to an arbitrary remote repository by specifying > -Dmvn.publish.repoUrl on the command line (patch by Thomas Dudziak). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3813) Allow publishing artifacts to an arbitrary remote repository
[ https://issues.apache.org/jira/browse/HIVE-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3813: - Description: Allow publishing artifacts to an arbitrary remote repository by specifying -Dmvn.publish.repoUrl on the command line (patch by Thomas Dudziak). (was: Allow publishing artifacts to an arbitrary remote repository by specifying -Dmvn.publish.repoUrl on the command line.) > Allow publishing artifacts to an arbitrary remote repository > > > Key: HIVE-3813 > URL: https://issues.apache.org/jira/browse/HIVE-3813 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin > > Allow publishing artifacts to an arbitrary remote repository by specifying > -Dmvn.publish.repoUrl on the command line (patch by Thomas Dudziak). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3813) Allow publishing artifacts to an arbitrary remote repository
Mikhail Bautin created HIVE-3813: Summary: Allow publishing artifacts to an arbitrary remote repository Key: HIVE-3813 URL: https://issues.apache.org/jira/browse/HIVE-3813 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Allow publishing artifacts to an arbitrary remote repository by specifying -Dmvn.publish.repoUrl on the command line. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3645) RCFileWriter does not implement the right function to support Federation
[ https://issues.apache.org/jira/browse/HIVE-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534210#comment-13534210 ] Mikhail Bautin commented on HIVE-3645: -- I am getting the following compilation errors with this patch on branch-0.9: {code} ivy-retrieve-hadoop-shim: [echo] Project: shims [javac] Compiling 1 source file to /wd/hive/build/shims/classes [javac] /wd/hive/shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:118: error: method getDefaultBlockSize in class FileSystem cannot be applied to given types; [javac] return fs.getDefaultBlockSize(path); [javac] ^ [javac] required: no arguments [javac] found: Path [javac] reason: actual and formal argument lists differ in length [javac] /wd/hive/shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:123: error: method getDefaultReplication in class FileSystem cannot be applied to given types; [javac] return fs.getDefaultReplication(path); [javac] ^ [javac] required: no arguments [javac] found: Path [javac] reason: actual and formal argument lists differ in length [javac] 2 errors BUILD FAILED /wd/hive/build.xml:319: The following error occurred while executing this line: /wd/hive/build.xml:169: The following error occurred while executing this line: /wd/hive/shims/build.xml:90: The following error occurred while executing this line: /wd/hive/shims/build.xml:93: The following error occurred while executing this line: /wd/hive/shims/build.xml:82: Compile failed; see the compiler error output for details. {code} I am building with the default Hadoop version. > RCFileWriter does not implement the right function to support Federation > > > Key: HIVE-3645 > URL: https://issues.apache.org/jira/browse/HIVE-3645 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Affects Versions: 0.9.0, 0.10.0 > Environment: Hadoop 0.23.3 federation, Hive 0.9 and Pig 0.10 >Reporter: Viraj Bhat >Assignee: Arup Malakar > Fix For: 0.11 > > Attachments: HIVE_3645_branch_0.patch, HIVE_3645_trunk_0.patch > > > Create a table using Hive DDL > {code} > CREATE TABLE tmp_hcat_federated_numbers_part_1 ( > id int, > intnum int, > floatnum float > )partitioned by ( > part1string, > part2string > ) > STORED AS rcfile > LOCATION 'viewfs:///database/tmp_hcat_federated_numbers_part_1'; > {code} > Populate it using Pig: > {code} > A = load 'default.numbers_pig' using org.apache.hcatalog.pig.HCatLoader(); > B = filter A by id <= 500; > C = foreach B generate (int)id, (int)intnum, (float)floatnum; > store C into > 'default.tmp_hcat_federated_numbers_part_1' > using org.apache.hcatalog.pig.HCatStorer >('part1=pig, part2=hcat_pig_insert', > 'id: int,intnum: int,floatnum: float'); > {code} > Generates the following error when running on a Federated Cluster: > {quote} > 2012-10-29 20:40:25,011 [main] ERROR > org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate > exception from backed error: AttemptID:attempt_1348522594824_0846_m_00_3 > Info:Error: org.apache.hadoop.fs.viewfs.NotInMountpointException: > getDefaultReplication on empty path is invalid > at > org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:479) > at org.apache.hadoop.hive.ql.io.RCFile$Writer.(RCFile.java:723) > at org.apache.hadoop.hive.ql.io.RCFile$Writer.(RCFile.java:705) > at > org.apache.hadoop.hive.ql.io.RCFileOutputFormat.getRecordWriter(RCFileOutputFormat.java:86) > at > org.apache.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:100) > at > org.apache.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:228) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84) > at > org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.(MapTask.java:587) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:706) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) &g
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3809: - Attachment: 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch Attaching a manually generated patch. > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin >Priority: Critical > Attachments: > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
Mikhail Bautin created HIVE-3809: Summary: Concurrency issue in RCFile: multiple threads can use the same decompressor Key: HIVE-3809 URL: https://issues.apache.org/jira/browse/HIVE-3809 Project: Hive Issue Type: Bug Reporter: Mikhail Bautin Priority: Critical RCFile is not thread-safe, even if each reader is only used by one thread as intended, because it is possible to return decompressors to the pool multiple times by calling close on the reader multiple times. Then, different threads can pick up the same decompressor twice from the pool, resulting in decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13526840#comment-13526840 ] Mikhail Bautin commented on HIVE-3384: -- [~cdrome]: I think this patch has already been committed to 0.9 (thanks [~ashutoshc]!), and we need to deal with remaining test failures separately. > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Assignee: Mikhail Bautin >Priority: Minor > Fix For: 0.11 > > Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, > D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, > HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, > HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review request: JIRAs useful for the Shark project
Thanks a lot to Ashutosh for reviewing and committing HIVE-3384 to trunk. We still have the following patches outstanding that would help the Shark project a lot -- your feedback / reviewing / committing would be much appreciated! - https://reviews.facebook.net/D6879 (HIVE-3731<https://issues.apache.org/jira/browse/HIVE-3731>): adding an Ant target to create a Debian package, which allows deploying the patched version of Hive alongside Shark on Debian systems. - https://reviews.facebook.net/D7155 (HIVE-3772<https://issues.apache.org/jira/browse/HIVE-3772>): a one-line concurrency bug fix in LazyBinaryUtils (patch by Reynold Xin). Thanks, Mikhail On Tue, Nov 27, 2012 at 7:08 PM, Ashutosh Chauhan wrote: > Hi Mikhail, > > I will take a look into those jiras. > > Thanks, > Ashutosh > > On Tue, Nov 27, 2012 at 11:43 AM, Mikhail Bautin < > bautin.mailing.li...@gmail.com> wrote: > > > Hello, > > > > There are the following review requests pending that are very useful for > > the Shark project (http://shark.cs.berkeley.edu/). It would be great if > > someone could take a look and help us get these JIRAs committed. > > > >- https://reviews.facebook.net/D6879 > > (HIVE-3731<https://issues.apache.org/jira/browse/HIVE-3731>): > >adding an Ant target to create a Debian package, which allows > deploying > > the > >patched version of Hive alongside Shark on Debian systems. > >- https://reviews.facebook.net/D7005 > > (HIVE-3748<https://issues.apache.org/jira/browse/HIVE-3748>): > >making QTestUtil work correctly when running the test suite, which > helps > >with running Hive/Shark unit tests from using Maven. > > > > In addition, the following JIRA would make a lot easier to work with Hive > > for anyone who is using JDK 1.7: > > > >- https://reviews.facebook.net/D6873 > > (HIVE-3384<https://issues.apache.org/jira/browse/HIVE-3384>): > >HIVE JDBC module won't compile under JDK1.7 as new methods added in > JDBC > >specification > > > > Your help in reviewing/committing these patches is greatly appreciated! > > > > Thanks, > > Mikhail > > >
[jira] [Updated] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin)
[ https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3772: - Attachment: HIVE-3772-2012-12-04.patch Attaching trunk patch. > Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by > Reynold Xin) > - > > Key: HIVE-3772 > URL: https://issues.apache.org/jira/browse/HIVE-3772 > Project: Hive > Issue Type: Bug > Reporter: Mikhail Bautin > Attachments: D7155.1.patch, HIVE-3772-2012-12-04.patch > > > Creating a JIRA for [~rxin]'s patch needed by the Shark project. > https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765 > writeVLong should not use a static field due to concurrency concerns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin)
Mikhail Bautin created HIVE-3772: Summary: Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin) Key: HIVE-3772 URL: https://issues.apache.org/jira/browse/HIVE-3772 Project: Hive Issue Type: Bug Reporter: Mikhail Bautin Creating a JIRA for [~rxin]'s patch needed by the Shark project. https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765 writeVLong should not use a static field due to concurrency concerns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3384: - Attachment: HIVE-3384-branch-0.9.patch Attaching a version of the patch for the 0.9 branch. [~ashutoshc]: Thank you for taking care of this JIRA. Could you please commit this one to the 0.9 branch as well? > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Assignee: Mikhail Bautin >Priority: Minor > Fix For: 0.11 > > Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, > D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, > HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, > HIVE-3384-branch-0.9.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509587#comment-13509587 ] Mikhail Bautin commented on HIVE-3384: -- [~ashutoshc]: Thank you for testing the patch. I have addressed your comment and PluginTest now passes when running from both {{builtin}} and {{pdk}} modules. {code:title=build/pdk/test-plugin/build/TEST-org.apache.hive.pdk.PluginTest.txt} Testsuite: org.apache.hive.pdk.PluginTest Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 42.592 sec - Standard Error - GLOBAL SETUP: Copying file: file:/wd/hive-trunk/build/pdk/test-plugin/test/onerow.txt Deleted file:/wd/hive-trunk/build/pdk/test-plugin/build/warehouse/onerow org.apache.hive.pdktest.Rot13 SETUP: Deleted file:/wd/hive-trunk/build/pdk/test-plugin/build/warehouse/rot13_data org.apache.hive.pdktest.Rot13 TEARDOWN: Hive history file=/tmp/mbautin/hive_job_log_mbautin_201212040011_1557635606.txt OK Time taken: 6.16 seconds GLOBAL TEARDOWN: Hive history file=/tmp/mbautin/hive_job_log_mbautin_201212040012_1607147323.txt OK Time taken: 0.287 seconds - --- Testcase: SELECT tp_rot13('Mixed Up!') FROM onerow; took 2.532 sec Testcase: SELECT tp_rot13(s) FROM rot13_data; took 2.495 sec {code} > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, > D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, > HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, > HIVE-3384.patch, HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3384: - Attachment: HIVE-3384-2012-12-04.patch Manually generated patch corresponding to https://reviews.facebook.net/D6873?vs=on&id=22815&whitespace=ignore-all#differential-review-toc addressing the problem with PluginTest running from the pdk module. > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, > D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, D6873.7.patch, > HIVE-3384-2012-12-02.patch, HIVE-3384-2012-12-04.patch, HIVE-3384.2.patch, > HIVE-3384.patch, HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508518#comment-13508518 ] Mikhail Bautin commented on HIVE-3384: -- [~ashutoshc]: PluginTest succeeds for me with my most recent patch. {quote} Testsuite: org.apache.hive.pdk.PluginTest Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 33.596 sec - Standard Error - GLOBAL SETUP: Copying file: file:/wd/hive-trunk/builtins/test/onerow.txt Deleted file:/wd/hive-trunk/build/builtins/warehouse/onerow Copying file: file:/wd/hive-trunk/builtins/test/iris.txt Deleted file:/wd/hive-trunk/build/builtins/warehouse/iris org.apache.hive.builtins.UDAFUnionMap TEARDOWN: Hive history file=/tmp/mbautin/hive_job_log_mbautin_201212022240_971726350.txt GLOBAL TEARDOWN: Hive history file=/tmp/mbautin/hive_job_log_mbautin_201212022240_805568604.txt OK Time taken: 8.473 seconds OK Time taken: 0.531 seconds - --- Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris took 3.454 sec {quote} > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, > D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, > HIVE-3384-2012-12-02.patch, HIVE-3384.2.patch, HIVE-3384.patch, > HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3384: - Attachment: HIVE-3384-2012-12-02.patch Uploading a patch corresponding to https://reviews.facebook.net/D6873?vs=on&id=22815&whitespace=ignore-all#differential-review-toc but generated manually so that it applies cleanly using the patch command. > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873-0.9.1.patch, D6873.1.patch, D6873.2.patch, > D6873.3.patch, D6873.4.patch, D6873.5.patch, D6873.6.patch, > HIVE-3384-2012-12-02.patch, HIVE-3384.2.patch, HIVE-3384.patch, > HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3748) QTestUtil should correctly find data files when running in the build directory
[ https://issues.apache.org/jira/browse/HIVE-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin resolved HIVE-3748. -- Resolution: Not A Problem Release Note: I found a way to achieve the same result without changing the code. In fact, Hive does run unit tests from a directory one level above the source root (e.g. ${hive.root}/ql, etc.) but data/conf/hive-site.xml specifies the property to look for test data files one directory above. > QTestUtil should correctly find data files when running in the build directory > -- > > Key: HIVE-3748 > URL: https://issues.apache.org/jira/browse/HIVE-3748 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Minor > Attachments: D7005.1.patch > > > Some parts of the the TestCliDriver test suite (i.e. some jar lookups) > require that the current directory is set to the build directory. This makes > QTestUtil correctly find data files when running either in the Hive source > root or in the build directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3748) QTestUtil should correctly find data files when running in the build directory
Mikhail Bautin created HIVE-3748: Summary: QTestUtil should correctly find data files when running in the build directory Key: HIVE-3748 URL: https://issues.apache.org/jira/browse/HIVE-3748 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Some parts of the the TestCliDriver test suite (i.e. some jar lookups) require that the current directory is set to the build directory. This makes QTestUtil correctly find data files when running either in the Hive source root or in the build directory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502582#comment-13502582 ] Mikhail Bautin commented on HIVE-3384: -- [~cdrome]: This type of error sometimes happens when loading class files compiled with Java 6 into a Java 7 JVM. The -XX:-UseSplitVerifier JVM option helps even though some people say it is unsafe. > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873-0.9.1.patch, D6873.1.patch, HIVE-3384.2.patch, > HIVE-3384.patch, HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3731) Ant target to create a Debian package
[ https://issues.apache.org/jira/browse/HIVE-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3731: - Status: Patch Available (was: Open) > Ant target to create a Debian package > - > > Key: HIVE-3731 > URL: https://issues.apache.org/jira/browse/HIVE-3731 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Minor > Attachments: D6879.1.patch > > > We need an Ant target to generate a Debian package with Hive binary > distribution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3731) Ant target to create a Debian package
[ https://issues.apache.org/jira/browse/HIVE-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3731: - Assignee: Mikhail Bautin > Ant target to create a Debian package > - > > Key: HIVE-3731 > URL: https://issues.apache.org/jira/browse/HIVE-3731 > Project: Hive > Issue Type: Improvement > Reporter: Mikhail Bautin > Assignee: Mikhail Bautin >Priority: Minor > Attachments: D6879.1.patch > > > We need an Ant target to generate a Debian package with Hive binary > distribution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3731) Ant target to create a Debian package
Mikhail Bautin created HIVE-3731: Summary: Ant target to create a Debian package Key: HIVE-3731 URL: https://issues.apache.org/jira/browse/HIVE-3731 Project: Hive Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor We need an Ant target to generate a Debian package with Hive binary distribution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
JIRA permissions for Hive
Hello, For some reason I don't seem to have permissions to assign issues, change issue priority, change issue status, cancel patch, etc. on Hive issues. Can someone add me to the relevant group on JIRA to allow these actions? My JIRA profile is: https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mikhail Thanks a lot! Mikhail
[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3384: - Status: Patch Available (was: Open) Submitting patch for Jenkins testing. > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, > HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification
[ https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501747#comment-13501747 ] Mikhail Bautin commented on HIVE-3384: -- [~shane.huang], [~cdrome]: thanks a lot for working on this! I am interested in this patch being committed soon because it is annoying to keep applying these changes locally just to build Hive with JDK 7. [~cdrome]: to help move this patch forward, I have combined the two patches in and posted them on Phabricator on https://reviews.facebook.net/D6873 (trunk patch). The patch should show up on this JIRA within the next hour, but if it does not, I will upload it manually. I think it is better to go through a review loop on the trunk patch first and then do a backport for branch-0.9. I am happy to do the backport once the trunk patch is committed. > HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC > specification > -- > > Key: HIVE-3384 > URL: https://issues.apache.org/jira/browse/HIVE-3384 > Project: Hive > Issue Type: Bug > Components: JDBC >Reporter: Weidong Bian >Priority: Minor > Attachments: D6873.1.patch, HIVE-3384.2.patch, HIVE-3384.patch, > HIVE-JDK7-JDBC.patch > > > jdbc module couldn't be compiled with jdk7 as it adds some abstract method in > the JDBC specification > some error info: > error: HiveCallableStatement is not abstract and does not override abstract > method getObject(String,Class) in CallableStatement > . > . > . -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3694) Generate test jars and publish them to Maven
[ https://issues.apache.org/jira/browse/HIVE-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501586#comment-13501586 ] Mikhail Bautin commented on HIVE-3694: -- Added a patch for review (preliminary version) at https://reviews.facebook.net/D6843 > Generate test jars and publish them to Maven > > > Key: HIVE-3694 > URL: https://issues.apache.org/jira/browse/HIVE-3694 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure > Reporter: Mikhail Bautin >Priority: Minor > Attachments: D6843.1.patch > > > It should be possible to generate Hive test jars and publish them to Maven so > that other projects that rely on Hive or extend it could reuse its test > library. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3694) Generate test jars and publish them to Maven
Mikhail Bautin created HIVE-3694: Summary: Generate test jars and publish them to Maven Key: HIVE-3694 URL: https://issues.apache.org/jira/browse/HIVE-3694 Project: Hive Issue Type: Improvement Components: Build Infrastructure Reporter: Mikhail Bautin Priority: Minor It should be possible to generate Hive test jars and publish them to Maven so that other projects that rely on Hive or extend it could reuse its test library. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira