[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-5279:
---
Status: Open (was: Patch Available)
Quite a few tests failed. 35 in TestCliDriver 2 in TestNegativeCliDriver,
including
database_drop.q,index*.q,ql_rewrite_gbtoidx.q,show_indexes_edge_cases.q,show_indexes_syntax.q,udaf_collect_set.q,union_view.q,virtual_column.q.
In -ve testcases, index_compact_entry_limit.q,index_compact_size_limit.q
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch, D12963.2.patch,
D12963.3.patch, D12963.4.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-5279:
--
Attachment: D12963.5.patch
navis updated the revision HIVE-5279 [jira] Kryo cannot instantiate
GenericUDAFEvaluator in GroupByDesc.
Fixed tests
Reviewers: ashutoshc, JIRA
REVISION DETAIL
https://reviews.facebook.net/D12963
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D12963?vs=40299id=40365#toc
BRANCH
HIVE-5279
ARCANIST PROJECT
hive
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java
ql/src/java/org/apache/hadoop/hive/ql/plan/AggregationDesc.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCollectList.java
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFMkCollectionEvaluator.java
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSumList.java
ql/src/test/queries/clientpositive/udaf_sum_list.q
ql/src/test/results/clientpositive/udaf_sum_list.q.out
ql/src/test/results/compiler/plan/groupby1.q.xml
ql/src/test/results/compiler/plan/groupby2.q.xml
ql/src/test/results/compiler/plan/groupby3.q.xml
ql/src/test/results/compiler/plan/groupby5.q.xml
To: JIRA, ashutoshc, navis
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch, D12963.2.patch,
D12963.3.patch, D12963.4.patch, D12963.5.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-5279:
Status: Patch Available (was: Open)
Missed those tests and sorry for delay.
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch, D12963.2.patch,
D12963.3.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-5279:
--
Attachment: D12963.4.patch
navis updated the revision HIVE-5279 [jira] Kryo cannot instantiate
GenericUDAFEvaluator in GroupByDesc.
Fixed test fails
Reviewers: ashutoshc, JIRA
REVISION DETAIL
https://reviews.facebook.net/D12963
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D12963?vs=40065id=40299#toc
BRANCH
HIVE-5279
ARCANIST PROJECT
hive
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java
ql/src/java/org/apache/hadoop/hive/ql/plan/AggregationDesc.java
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSumList.java
ql/src/test/queries/clientpositive/udaf_sum_list.q
ql/src/test/results/clientpositive/udaf_sum_list.q.out
ql/src/test/results/compiler/plan/groupby1.q.xml
ql/src/test/results/compiler/plan/groupby2.q.xml
ql/src/test/results/compiler/plan/groupby3.q.xml
ql/src/test/results/compiler/plan/groupby5.q.xml
To: JIRA, ashutoshc, navis
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch, D12963.2.patch,
D12963.3.patch, D12963.4.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-5279:
---
Assignee: Navis
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-5279:
---
Status: Open (was: Patch Available)
Following tests failed:
* TestParse
* TestCliDriver_autogen_colalias.q
* TestCliDriver_create_udaf.q
* TestCliDriver_create_view.q
* TestCliDriver_limit_pushdown.q
* TestCliDriver_show_functions.q
* TestCliDriver_udaf_sum_list.q
* TestCliDriver_udf_percentile.q
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-5279:
--
Attachment: D12963.2.patch
navis updated the revision HIVE-5279 [jira] Kryo cannot instantiate
GenericUDAFEvaluator in GroupByDesc.
Fixed test fails
Reviewers: ashutoshc, JIRA
REVISION DETAIL
https://reviews.facebook.net/D12963
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D12963?vs=40017id=40053#toc
BRANCH
HIVE-5279
ARCANIST PROJECT
hive
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java
ql/src/java/org/apache/hadoop/hive/ql/plan/AggregationDesc.java
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSumList.java
ql/src/test/queries/clientpositive/udaf_sum_list.q
ql/src/test/results/clientpositive/udaf_sum_list.q.out
To: JIRA, ashutoshc, navis
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch, D12963.2.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-5279:
--
Attachment: D12963.3.patch
navis updated the revision HIVE-5279 [jira] Kryo cannot instantiate
GenericUDAFEvaluator in GroupByDesc.
use class name for UDAFEvaluator in AggregationDesc when the class is not
Serializable or Externalizable
Reviewers: ashutoshc, JIRA
REVISION DETAIL
https://reviews.facebook.net/D12963
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D12963?vs=40053id=40065#toc
BRANCH
HIVE-5279
ARCANIST PROJECT
hive
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/UDAF.java
ql/src/java/org/apache/hadoop/hive/ql/plan/AggregationDesc.java
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSumList.java
ql/src/test/queries/clientpositive/udaf_sum_list.q
ql/src/test/results/clientpositive/udaf_sum_list.q.out
To: JIRA, ashutoshc, navis
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch, D12963.2.patch,
D12963.3.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-5279:
Status: Patch Available (was: Open)
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Priority: Critical
Attachments: 5279.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-5279:
--
Attachment: D12963.1.patch
navis requested code review of HIVE-5279 [jira] Kryo cannot instantiate
GenericUDAFEvaluator in GroupByDesc.
Reviewers: JIRA
HIVE-5279 Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's not
Serializable and fails the query.
The log below is the example,
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
If this cannot be fixed in somehow, some UDAFs should be modified to be run on
hive-0.13.0
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D12963
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/plan/AggregationDesc.java
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSumList.java
ql/src/test/queries/clientpositive/udaf_sum_list.q
ql/src/test/results/clientpositive/udaf_sum_list.q.out
MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/30963/
To: JIRA, navis
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Priority: Critical
Attachments: 5279.patch, D12963.1.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-5279) Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
[
https://issues.apache.org/jira/browse/HIVE-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-5279:
---
Attachment: 5279.patch
Here is an illustrative patch which should fix the issue. [~navis] Can you give
it a try?
Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc
---
Key: HIVE-5279
URL: https://issues.apache.org/jira/browse/HIVE-5279
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Priority: Critical
Attachments: 5279.patch
We didn't forced GenericUDAFEvaluator to be Serializable. I don't know how
previous serialization mechanism solved this but, kryo complaints that it's
not Serializable and fails the query.
The log below is the example,
{noformat}
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: Class
cannot be created (missing no-arg constructor):
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
Serialization trace:
inputOI
(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFGroupOn$VersionedFloatGroupOnEval)
genericUDAFEvaluator (org.apache.hadoop.hive.ql.plan.AggregationDesc)
aggregators (org.apache.hadoop.hive.ql.plan.GroupByDesc)
conf (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:312)
at
org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:261)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:256)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:383)
at org.apache.h
{noformat}
If this cannot be fixed in somehow, some UDAFs should be modified to be run
on hive-0.13.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
