[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13648683#comment-13648683 ] Samuel Yuan commented on HIVE-4433: --- I'm thinking it's possible to work around this by defining '' since it's present in the auto-generated header file. Given that other language bindings might also have been broken by HIVE-4322 though it's probably better to change the map key to a primitive type instead. I have filed HIVE-4492 to revert the original change. Fix C++ Thrift bindings broken in HIVE-4322 --- Key: HIVE-4433 URL: https://issues.apache.org/jira/browse/HIVE-4433 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Affects Versions: 0.12.0 Reporter: Carl Steinbach Assignee: Samuel Yuan Priority: Blocker Fix For: 0.12.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13644131#comment-13644131 ] Carl Steinbach commented on HIVE-4433: -- Good point. I changed the fixversion to 0.12.0 Fix C++ Thrift bindings broken in HIVE-4322 --- Key: HIVE-4433 URL: https://issues.apache.org/jira/browse/HIVE-4433 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Affects Versions: 0.12.0 Reporter: Carl Steinbach Assignee: Samuel Yuan Priority: Blocker Fix For: 0.12.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[
https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643572#comment-13643572
]
Carl Steinbach commented on HIVE-4433:
--
You can reproduce the error by running the following command:
% ant compile-cpp -Dthrift.home=$THRIFT_HOME
Here's the error message I get:
{noformat}
[exec] g++ -m64 -DARCH64 -dynamiclib
/Users/carl/Work/repos/hive-test/build/metastore/objs/ThriftHiveMetastore.o
/Users/carl/Work/repos/hive-test/build/metastore/objs/hive_metastore_constants.o
/Users/carl/Work/repos/hive-test/build/metastore/objs/hive_metastore_types.o
/Users/carl/Work/repos/hive-test/build/service/objs/ThriftHive.o
/Users/carl/Work/repos/hive-test/build/service/objs/hive_service_constants.o
/Users/carl/Work/repos/hive-test/build/service/objs/hive_service_types.o
/Users/carl/Work/repos/hive-test/build/ql/objs/queryplan_types.o
/Users/carl/Work/repos/hive-test/build/ql/objs/queryplan_constants.o
/Users/carl/Work/repos/hive-test/build/odbc/objs/hiveclient.o
/Users/carl/Work/repos/hive-test/build/odbc/objs/HiveResultSet.o
/Users/carl/Work/repos/hive-test/build/odbc/objs/HiveColumnDesc.o
/Users/carl/Work/repos/hive-test/build/odbc/objs/HiveRowSet.o
/Users/carl/Work/repos/hive-test/build/odbc/objs/hiveclienthelper.o
-L/usr/local/lib -lthrift -L/usr/local/lib -lfb303 -o
/Users/carl/Work/repos/hive-test/build/odbc/lib/libhiveclient.so.1.0.0 \
[exec] ln -sf libhiveclient.so.1.0.0
/Users/carl/Work/repos/hive-test/build/odbc/lib/libhiveclient.so
[exec] Undefined symbols for architecture x86_64:
[exec]
Apache::Hadoop::Hive::SkewedValueList::operator(Apache::Hadoop::Hive::SkewedValueList
const) const, referenced from:
[exec]
std::lessApache::Hadoop::Hive::SkewedValueList::operator()(Apache::Hadoop::Hive::SkewedValueList
const, Apache::Hadoop::Hive::SkewedValueList const) constin
hive_metastore_types.o
[exec] ld: symbol(s) not found for architecture x86_64
[exec] collect2: ld returned 1 exit status
[exec] make: ***
[/Users/carl/Work/repos/hive-test/build/odbc/lib/libhiveclient.so.1.0.0] Error 1
BUILD FAILED
/Users/carl/Work/repos/hive-test/build.xml:242: The following error occurred
while executing this line:
/Users/carl/Work/repos/hive-test/build.xml:109: The following error occurred
while executing this line:
/Users/carl/Work/repos/hive-test/build.xml:111: The following error occurred
while executing this line:
/Users/carl/Work/repos/hive-test/odbc/build.xml:69: exec returned: 2
{noformat}
The problem is that STL maps require the '' operator to be defined for the key
type, and SkewedValueList doesn't satisfy that requirement.
Fix C++ Thrift bindings broken in HIVE-4322
---
Key: HIVE-4433
URL: https://issues.apache.org/jira/browse/HIVE-4433
Project: Hive
Issue Type: Bug
Components: Metastore, Thrift API
Affects Versions: 0.11.0
Reporter: Carl Steinbach
Assignee: Samuel Yuan
Priority: Blocker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643876#comment-13643876 ] Ashutosh Chauhan commented on HIVE-4433: [~cwsteinbach] The original culprit HIVE-4322 is not on 0.11. So, I think this cannot be marked as blocker for 0.11. Do you agree? Fix C++ Thrift bindings broken in HIVE-4322 --- Key: HIVE-4433 URL: https://issues.apache.org/jira/browse/HIVE-4433 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Affects Versions: 0.11.0 Reporter: Carl Steinbach Assignee: Samuel Yuan Priority: Blocker -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[
https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643463#comment-13643463
]
Carl Steinbach commented on HIVE-4433:
--
HIVE-4322 added a SkewedValueList structure to the Metastore Thrift IDL, along
with a map field where this structure is used as a key:
{noformat}
// Workaround for HIVE-4322struct SkewedValueList {
1: liststring skewedValueList
}
// this object holds all the information about skewed table
struct SkewedInfo {
1: liststring skewedColNames, // skewed column names
2: listliststring skewedColValues, //skewed values
3: mapSkewedValueList, string skewedColValueLocationMaps, //skewed value to
location mappings
}
{noformat}
It turns out that this change breaks the C++ Thrift bindings (and possibly also
the bindings for other languages).
According to the Thrift documentation
[here|http://thrift.apache.org/docs/types/] it's a really bad idea to use a
struct or container type as a map key:
{quote}
N.B.: For maximal compatibility, the key type for map should be a basic type
rather than a struct or container type. There are some languages which do not
support more complex key types in their native map types. In addition the JSON
protocol only supports key types that are base types.
{quote}
I haven't had much time to look into this, but my hunch is that we probably
need to backout HIVE-4322.
Fix C++ Thrift bindings broken in HIVE-4322
---
Key: HIVE-4433
URL: https://issues.apache.org/jira/browse/HIVE-4433
Project: Hive
Issue Type: Bug
Components: Metastore, Thrift API
Affects Versions: 0.11.0
Reporter: Carl Steinbach
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643466#comment-13643466 ] Carl Steinbach commented on HIVE-4433: -- Marking this as a blocker for 0.11.0. Fix C++ Thrift bindings broken in HIVE-4322 --- Key: HIVE-4433 URL: https://issues.apache.org/jira/browse/HIVE-4433 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Affects Versions: 0.11.0 Reporter: Carl Steinbach Priority: Blocker -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
