[jira] [Updated] (HIVE-4078) Remove the serialize-deserialize pair in CommonJoinResolver
[
https://issues.apache.org/jira/browse/HIVE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-4078:
--
Status: Open (was: Patch Available)
Updating patch to match review comments.
Remove the serialize-deserialize pair in CommonJoinResolver
---
Key: HIVE-4078
URL: https://issues.apache.org/jira/browse/HIVE-4078
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Gopal V
Assignee: Gopal V
Attachments: HIVE-4078.patch
CommonJoinProcessor tries to clone a MapredWork while attempting a conversion
to a map-join
{code}
// deep copy a new mapred work from xml
InputStream in = new ByteArrayInputStream(xml.getBytes(UTF-8));
MapredWork newWork = Utilities.deserializeMapRedWork(in,
physicalContext.getConf());
{code}
which is a very heavy operation memory wise cpu-wise.
Instead of cloning via XMLEncoder, it is faster to use BeanUtils.cloneBean()
which is following same data paths (get/set bean methods) instead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4078) Remove the serialize-deserialize pair in CommonJoinResolver
[
https://issues.apache.org/jira/browse/HIVE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-4078:
--
Attachment: HIVE-4078-20130227.patch
Updated to throw an exception if the cloner throws a SemanticException wrapped
around the IllegalAccess or Invocation exception that are possible (but
unlikely).
Remove the serialize-deserialize pair in CommonJoinResolver
---
Key: HIVE-4078
URL: https://issues.apache.org/jira/browse/HIVE-4078
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Gopal V
Assignee: Gopal V
Attachments: HIVE-4078-20130227.patch, HIVE-4078.patch
CommonJoinProcessor tries to clone a MapredWork while attempting a conversion
to a map-join
{code}
// deep copy a new mapred work from xml
InputStream in = new ByteArrayInputStream(xml.getBytes(UTF-8));
MapredWork newWork = Utilities.deserializeMapRedWork(in,
physicalContext.getConf());
{code}
which is a very heavy operation memory wise cpu-wise.
Instead of cloning via XMLEncoder, it is faster to use BeanUtils.cloneBean()
which is following same data paths (get/set bean methods) instead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4078) Remove the serialize-deserialize pair in CommonJoinResolver
[
https://issues.apache.org/jira/browse/HIVE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-4078:
--
Status: Patch Available (was: Open)
Remove the serialize-deserialize pair in CommonJoinResolver
---
Key: HIVE-4078
URL: https://issues.apache.org/jira/browse/HIVE-4078
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Gopal V
Assignee: Gopal V
Attachments: HIVE-4078-20130227.patch, HIVE-4078.patch
CommonJoinProcessor tries to clone a MapredWork while attempting a conversion
to a map-join
{code}
// deep copy a new mapred work from xml
InputStream in = new ByteArrayInputStream(xml.getBytes(UTF-8));
MapredWork newWork = Utilities.deserializeMapRedWork(in,
physicalContext.getConf());
{code}
which is a very heavy operation memory wise cpu-wise.
Instead of cloning via XMLEncoder, it is faster to use BeanUtils.cloneBean()
which is following same data paths (get/set bean methods) instead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4078) Remove the serialize-deserialize pair in CommonJoinResolver
[
https://issues.apache.org/jira/browse/HIVE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-4078:
--
Status: Open (was: Patch Available)
cloneBean() only clones part of the data, does not do a true deep-copy.
Remove the serialize-deserialize pair in CommonJoinResolver
---
Key: HIVE-4078
URL: https://issues.apache.org/jira/browse/HIVE-4078
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Gopal V
Assignee: Gopal V
Attachments: HIVE-4078-20130227.patch, HIVE-4078.patch
CommonJoinProcessor tries to clone a MapredWork while attempting a conversion
to a map-join
{code}
// deep copy a new mapred work from xml
InputStream in = new ByteArrayInputStream(xml.getBytes(UTF-8));
MapredWork newWork = Utilities.deserializeMapRedWork(in,
physicalContext.getConf());
{code}
which is a very heavy operation memory wise cpu-wise.
Instead of cloning via XMLEncoder, it is faster to use BeanUtils.cloneBean()
which is following same data paths (get/set bean methods) instead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4078) Remove the serialize-deserialize pair in CommonJoinResolver
[
https://issues.apache.org/jira/browse/HIVE-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-4078:
--
Attachment: HIVE-4078.patch
Over six runs of an explain query with 3 joins
|| pre-patch || post-patch ||
|7.795 seconds| 3.374 seconds|
|3.228 seconds| 0.432 seconds|
|3.031 seconds| 0.341 seconds|
|2.95 seconds|0.31 seconds|
|2.909 seconds| 0.266 seconds|
|2.89 seconds|0.237 seconds|
Remove the serialize-deserialize pair in CommonJoinResolver
---
Key: HIVE-4078
URL: https://issues.apache.org/jira/browse/HIVE-4078
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Gopal V
Assignee: Gopal V
Attachments: HIVE-4078.patch
CommonJoinProcessor tries to clone a MapredWork while attempting a conversion
to a map-join
{code}
// deep copy a new mapred work from xml
InputStream in = new ByteArrayInputStream(xml.getBytes(UTF-8));
MapredWork newWork = Utilities.deserializeMapRedWork(in,
physicalContext.getConf());
{code}
which is a very heavy operation memory wise cpu-wise.
Instead of cloning via XMLEncoder, it is faster to use BeanUtils.cloneBean()
which is following same data paths (get/set bean methods) instead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
