[jira] [Created] (HIVE-10532) SpecificMutableRow doesn't handle Date Type correctly
Cheng Hao created HIVE-10532:
Summary: SpecificMutableRow doesn't handle Date Type correctly
Key: HIVE-10532
URL: https://issues.apache.org/jira/browse/HIVE-10532
Project: Hive
Issue Type: Bug
Reporter: Cheng Hao
{code}
test(test DATE types in cache) {
val rows = TestSQLContext.jdbc(urlWithUserAndPass,
TEST.TIMETYPES).collect()
TestSQLContext.jdbc(urlWithUserAndPass,
TEST.TIMETYPES).cache().registerTempTable(mycached_date)
val cachedRows = sql(select * from mycached_date).collect()
assert(rows(0).getAs[java.sql.Date](1) ===
java.sql.Date.valueOf(1996-01-01))
assert(cachedRows(0).getAs[java.sql.Date](1) ===
java.sql.Date.valueOf(1996-01-01))
}
{code}
{panel}
java.lang.ClassCastException:
org.apache.spark.sql.catalyst.expressions.MutableAny cannot be cast to
org.apache.spark.sql.catalyst.expressions.MutableInt
at
org.apache.spark.sql.catalyst.expressions.SpecificMutableRow.getInt(SpecificMutableRow.scala:252)
at
org.apache.spark.sql.columnar.IntColumnStats.gatherStats(ColumnStats.scala:208)
at
org.apache.spark.sql.columnar.NullableColumnBuilder$class.appendFrom(NullableColumnBuilder.scala:56)
at
org.apache.spark.sql.columnar.NativeColumnBuilder.org$apache$spark$sql$columnar$compression$CompressibleColumnBuilder$$super$appendFrom(ColumnBuilder.scala:87)
at
org.apache.spark.sql.columnar.compression.CompressibleColumnBuilder$class.appendFrom(CompressibleColumnBuilder.scala:78)
at
org.apache.spark.sql.columnar.NativeColumnBuilder.appendFrom(ColumnBuilder.scala:87)
at
org.apache.spark.sql.columnar.InMemoryRelation$$anonfun$3$$anon$1.next(InMemoryColumnarTableScan.scala:148)
at
org.apache.spark.sql.columnar.InMemoryRelation$$anonfun$3$$anon$1.next(InMemoryColumnarTableScan.scala:124)
at
org.apache.spark.storage.MemoryStore.unrollSafely(MemoryStore.scala:277)
at
org.apache.spark.CacheManager.putInBlockManager(CacheManager.scala:171)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:78)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:242)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
at org.apache.spark.scheduler.Task.run(Task.scala:64)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:209)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-9004: Attachment: (was: reset.patch) Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao Fix For: spark-branch, 0.15.0, 0.14.1 Attachments: HIVE-9004.patch To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-9004: Attachment: HIVE-9004.patch Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao Fix For: spark-branch, 0.15.0, 0.14.1 Attachments: HIVE-9004.patch To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14251138#comment-14251138 ] Cheng Hao commented on HIVE-9004: - Thank you [~szehon], updated. Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao Fix For: spark-branch, 0.15.0, 0.14.1 Attachments: HIVE-9004.patch To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231670#comment-14231670 ] Cheng Hao commented on HIVE-9004: - [~namit] sorry, I am not sure the review process, can you review this for me please? Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao Fix For: spark-branch, 0.15.0, 0.14.1 Attachments: reset.patch To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-9004) Reset doesn't work for the default empty value entry
Cheng Hao created HIVE-9004: --- Summary: Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao reassigned HIVE-9004: --- Assignee: Cheng Hao Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-9004: Attachment: reset.patch Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao Attachments: reset.patch To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9004) Reset doesn't work for the default empty value entry
[ https://issues.apache.org/jira/browse/HIVE-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-9004: Fix Version/s: 0.14.1 0.15.0 spark-branch Status: Patch Available (was: Open) Reset doesn't work for the default empty value entry Key: HIVE-9004 URL: https://issues.apache.org/jira/browse/HIVE-9004 Project: Hive Issue Type: Bug Components: Configuration Reporter: Cheng Hao Assignee: Cheng Hao Fix For: spark-branch, 0.15.0, 0.14.1 Attachments: reset.patch To illustrate that: In hive cli: hive set hive.table.parameters.default; hive.table.parameters.default is undefined hive set hive.table.parameters.default=key1=value1; hive reset; hive set hive.table.parameters.default; hive.table.parameters.default=key1=value1 I think we expect the last output as hive.table.parameters.default is undefined -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-4864) Code Comments seems confused between GenericUDFCase GenericUDFWhen
[ https://issues.apache.org/jira/browse/HIVE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-4864: Affects Version/s: 0.9.0 Status: Patch Available (was: Open) Code Comments seems confused between GenericUDFCase GenericUDFWhen Key: HIVE-4864 URL: https://issues.apache.org/jira/browse/HIVE-4864 Project: Hive Issue Type: Task Affects Versions: 0.9.0 Reporter: Cheng Hao Priority: Trivial Attachments: 1.patch Code Comment in GenericUDFCase: /** * GenericUDF Class for SQL construct * CASE WHEN a THEN b WHEN c THEN d [ELSE f] END. * * NOTES: 1. a and c should be boolean, or an exception will be thrown. 2. b, d * and f should have the same TypeInfo, or an exception will be thrown. */ And the code comment in GenericUDFWhen: /** * GenericUDF Class for SQL construct CASE a WHEN b THEN c [ELSE f] END. * * NOTES: 1. a and b should have the same TypeInfo, or an exception will be thrown. 2. c and f should have the same TypeInfo, or an exception will be * thrown. */ From the code itself, seems the comments should be exchanged. We'd better amend that to avoid confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4864) Code Comments seems confused between GenericUDFCase GenericUDFWhen
[ https://issues.apache.org/jira/browse/HIVE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-4864: Attachment: 1.patch Please check the attachment. Code Comments seems confused between GenericUDFCase GenericUDFWhen Key: HIVE-4864 URL: https://issues.apache.org/jira/browse/HIVE-4864 Project: Hive Issue Type: Task Reporter: Cheng Hao Priority: Trivial Attachments: 1.patch Code Comment in GenericUDFCase: /** * GenericUDF Class for SQL construct * CASE WHEN a THEN b WHEN c THEN d [ELSE f] END. * * NOTES: 1. a and c should be boolean, or an exception will be thrown. 2. b, d * and f should have the same TypeInfo, or an exception will be thrown. */ And the code comment in GenericUDFWhen: /** * GenericUDF Class for SQL construct CASE a WHEN b THEN c [ELSE f] END. * * NOTES: 1. a and b should have the same TypeInfo, or an exception will be thrown. 2. c and f should have the same TypeInfo, or an exception will be * thrown. */ From the code itself, seems the comments should be exchanged. We'd better amend that to avoid confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4864) Code Comments seems confused between GenericUDFCase GenericUDFWhen
[ https://issues.apache.org/jira/browse/HIVE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-4864: Attachment: 2.patch Code Comments seems confused between GenericUDFCase GenericUDFWhen Key: HIVE-4864 URL: https://issues.apache.org/jira/browse/HIVE-4864 Project: Hive Issue Type: Task Affects Versions: 0.9.0, 0.10.0, 0.11.0 Reporter: Cheng Hao Assignee: Cheng Hao Priority: Trivial Attachments: 1.patch, 2.patch Code Comment in GenericUDFCase: /** * GenericUDF Class for SQL construct * CASE WHEN a THEN b WHEN c THEN d [ELSE f] END. * * NOTES: 1. a and c should be boolean, or an exception will be thrown. 2. b, d * and f should have the same TypeInfo, or an exception will be thrown. */ And the code comment in GenericUDFWhen: /** * GenericUDF Class for SQL construct CASE a WHEN b THEN c [ELSE f] END. * * NOTES: 1. a and b should have the same TypeInfo, or an exception will be thrown. 2. c and f should have the same TypeInfo, or an exception will be * thrown. */ From the code itself, seems the comments should be exchanged. We'd better amend that to avoid confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4864) Code Comments seems confused between GenericUDFCase GenericUDFWhen
[ https://issues.apache.org/jira/browse/HIVE-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717873#comment-13717873 ] Cheng Hao commented on HIVE-4864: - Thanks [~ashutoshc], please check the [^2.patch] Code Comments seems confused between GenericUDFCase GenericUDFWhen Key: HIVE-4864 URL: https://issues.apache.org/jira/browse/HIVE-4864 Project: Hive Issue Type: Task Affects Versions: 0.9.0, 0.10.0, 0.11.0 Reporter: Cheng Hao Assignee: Cheng Hao Priority: Trivial Attachments: 1.patch, 2.patch Code Comment in GenericUDFCase: /** * GenericUDF Class for SQL construct * CASE WHEN a THEN b WHEN c THEN d [ELSE f] END. * * NOTES: 1. a and c should be boolean, or an exception will be thrown. 2. b, d * and f should have the same TypeInfo, or an exception will be thrown. */ And the code comment in GenericUDFWhen: /** * GenericUDF Class for SQL construct CASE a WHEN b THEN c [ELSE f] END. * * NOTES: 1. a and b should have the same TypeInfo, or an exception will be thrown. 2. c and f should have the same TypeInfo, or an exception will be * thrown. */ From the code itself, seems the comments should be exchanged. We'd better amend that to avoid confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4864) Code Comments seems confused between GenericUDFCase GenericUDFWhen
Cheng Hao created HIVE-4864: --- Summary: Code Comments seems confused between GenericUDFCase GenericUDFWhen Key: HIVE-4864 URL: https://issues.apache.org/jira/browse/HIVE-4864 Project: Hive Issue Type: Task Reporter: Cheng Hao Priority: Trivial Code Comment in GenericUDFCase: /** * GenericUDF Class for SQL construct * CASE WHEN a THEN b WHEN c THEN d [ELSE f] END. * * NOTES: 1. a and c should be boolean, or an exception will be thrown. 2. b, d * and f should have the same TypeInfo, or an exception will be thrown. */ And the code comment in GenericUDFWhen: /** * GenericUDF Class for SQL construct CASE a WHEN b THEN c [ELSE f] END. * * NOTES: 1. a and b should have the same TypeInfo, or an exception will be thrown. 2. c and f should have the same TypeInfo, or an exception will be * thrown. */ From the code itself, seems the comments should be exchanged. We'd better amend that to avoid confusing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4855) Failed in equality check of PrimitiveTypeInfo after ser/de
Cheng Hao created HIVE-4855:
---
Summary: Failed in equality check of PrimitiveTypeInfo after ser/de
Key: HIVE-4855
URL: https://issues.apache.org/jira/browse/HIVE-4855
Project: Hive
Issue Type: Bug
Reporter: Cheng Hao
Priority: Minor
The equals method in PrimitiveTypeInfo.java as shown below
/**
* Compare if 2 TypeInfos are the same. We use TypeInfoFactory to cache
* TypeInfos, so we only need to compare the Object pointer.
*/
@Override
public boolean equals(Object other) {
return this == other;
}
But, it may still fails the equality checking of PrimitiveTypeInfo instances,
before and after its de-serialization.
I met that bug as I was trying call the method of
ExprNodeGenericFuncDesc.isSame(Object obj), which are actually 2 de-serialized
instances from the exactly same source, and I always got false.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4777) Null value Versus RuntimeException in failed data type converting
Cheng Hao created HIVE-4777: --- Summary: Null value Versus RuntimeException in failed data type converting Key: HIVE-4777 URL: https://issues.apache.org/jira/browse/HIVE-4777 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Reporter: Cheng Hao Priority: Minor Fix For: 0.12.0 Usually null value will returns if the data can not be converted to the other type. (e.g. null == doubleConverter.convert(abc)). But it also may also throws RuntimeException other than NumberFormatException in the DoubleConverter.convert() method, as the type BINARY can not be converted into DOUBLE. And there are some others. (Check PrimitiveObjectInspectorUtils.java for details.) It's acceptable for throwing exception, but may not in runtime, as the data type convertibility can be decided before the data arrives. The earlier stage the better. Or just like the normal case, return null constantly for the in-convertible data types converting. What do you think? Thanks, Hao -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4410) Dummy Storage handler for the select performance benchmarking
Cheng Hao created HIVE-4410: --- Summary: Dummy Storage handler for the select performance benchmarking Key: HIVE-4410 URL: https://issues.apache.org/jira/browse/HIVE-4410 Project: Hive Issue Type: New Feature Components: StorageHandler Reporter: Cheng Hao Priority: Minor Create a Dummy Storage handler for the select performance benchmarking purpose, without real data output(write to the console or disk). It's would be great helpful if the result is big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4410) Dummy Storage handler for the select performance benchmarking
[ https://issues.apache.org/jira/browse/HIVE-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-4410: Fix Version/s: 0.9.0 Labels: storage-handler (was: ) Status: Patch Available (was: Open) Dummy Storage handler for the select performance benchmarking --- Key: HIVE-4410 URL: https://issues.apache.org/jira/browse/HIVE-4410 Project: Hive Issue Type: New Feature Components: StorageHandler Reporter: Cheng Hao Priority: Minor Labels: storage-handler Fix For: 0.9.0 Attachments: dummy.patch Create a Dummy Storage handler for the select performance benchmarking purpose, without real data output(write to the console or disk). It's would be great helpful if the result is big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4410) Dummy Storage handler for the select performance benchmarking
[ https://issues.apache.org/jira/browse/HIVE-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao updated HIVE-4410: Attachment: dummy.patch Dummy Storage handler for the select performance benchmarking --- Key: HIVE-4410 URL: https://issues.apache.org/jira/browse/HIVE-4410 Project: Hive Issue Type: New Feature Components: StorageHandler Reporter: Cheng Hao Priority: Minor Labels: storage-handler Fix For: 0.9.0 Attachments: dummy.patch Create a Dummy Storage handler for the select performance benchmarking purpose, without real data output(write to the console or disk). It's would be great helpful if the result is big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4410) Dummy Storage handler for the select performance benchmarking
[ https://issues.apache.org/jira/browse/HIVE-4410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13640144#comment-13640144 ] Cheng Hao commented on HIVE-4410: - for example: CREATE TABLE dummy_uservisits (sourceIP STRING,destURL STRING,visitDate STRING,adRevenue DOUBLE,userAgent STRING,countryCode STRING,languageCode STRING,searchWord STRING,duration INT) STORED BY 'org.apache.hadoop.hive.ql.metadata.DummyStorageHandler'; insert overwrite table dummy_uservisits select * from uservisits where (xxx); No real data be written into the dummy table, and great helpful in benchmarking the Select performance. Dummy Storage handler for the select performance benchmarking --- Key: HIVE-4410 URL: https://issues.apache.org/jira/browse/HIVE-4410 Project: Hive Issue Type: New Feature Components: StorageHandler Reporter: Cheng Hao Priority: Minor Labels: storage-handler Fix For: 0.9.0 Attachments: dummy.patch Create a Dummy Storage handler for the select performance benchmarking purpose, without real data output(write to the console or disk). It's would be great helpful if the result is big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3823) Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat
Cheng Hao created HIVE-3823:
---
Summary: Performance issue while retrieving the Result objects in
HiveHBaseTableInputFormat
Key: HIVE-3823
URL: https://issues.apache.org/jira/browse/HIVE-3823
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.9.0, 0.10.0, 0.9.1
Reporter: Cheng Hao
Priority: Trivial
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title:HiveHBaseTableInputFormat}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3823) Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat
[
https://issues.apache.org/jira/browse/HIVE-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Hao updated HIVE-3823:
Description:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title:HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
was:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title:HiveHBaseTableInputFormat}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
Performance issue while retrieving the Result objects in
HiveHBaseTableInputFormat
--
Key: HIVE-3823
URL: https://issues.apache.org/jira/browse/HIVE-3823
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.9.0, 0.10.0, 0.9.1
Reporter: Cheng Hao
Priority: Trivial
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title:HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3823) Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat
[
https://issues.apache.org/jira/browse/HIVE-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Hao updated HIVE-3823:
Description:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title=HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
was:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title:HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
Performance issue while retrieving the Result objects in
HiveHBaseTableInputFormat
--
Key: HIVE-3823
URL: https://issues.apache.org/jira/browse/HIVE-3823
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.9.0, 0.10.0, 0.9.1
Reporter: Cheng Hao
Priority: Trivial
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title=HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3823) Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat
[
https://issues.apache.org/jira/browse/HIVE-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Hao updated HIVE-3823:
Description:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title=HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom(Result
from), would solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
was:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title=HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom, would
solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
Performance issue while retrieving the Result objects in
HiveHBaseTableInputFormat
--
Key: HIVE-3823
URL: https://issues.apache.org/jira/browse/HIVE-3823
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.9.0, 0.10.0, 0.9.1
Reporter: Cheng Hao
Priority: Trivial
In HiveHBaseTableInputFormat.java, the Result objects retrieving has
performance issue.
{code:title=HiveHBaseTableInputFormat.java}
@Override
public boolean next(ImmutableBytesWritable rowKey, Result value) throws
IOException {
boolean next = false;
try {
next = recordReader.nextKeyValue();
if (next) {
rowKey.set(recordReader.getCurrentValue().getRow());
// performance issue here, as the copyWritable
// is Serialization - Bytes Copying - Deserialization.
Writables.copyWritable(recordReader.getCurrentValue(), value);
}
} catch (InterruptedException e) {
throw new IOException(e);
}
return next;
}
{code}
In HBASE 0.94.4 0.96.0, the Result provides a new method copyFrom(Result
from), would solve the issue.
See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
