Hi all, i have a very complex json that i need to insert in a hive table. A json example follws attached.
First of all i read a json file with Spark to make some data processing and then i write to a stage table with no Serde and with no any kind of compression and format (Parquet or ORC). Then i do an INSERT/SELECT into the "jsonTable" (create table attached) with no problems. This table use a json Serde (org.openx.data.jsonserde.JsonSerDe) and a ORC format and is also particioned by date and timezone. The problem is that after all this process every time a try to make a simple "select * from jsonTable" query i got this error message: "Failed with exception java.io.IOException:java.io.IOException: Error reading file: hdfs://ip-xxx-xxx-xxx-xxx.sa-east-1.compute.int ernal:8020/user/hive/warehouse/jsonTable/data_posicao_short=2017-12-02/ veitimezone=America-Sao_Paulo/000000_0" Actually i do not know if it is possible to use Serde, ORC and partition in the same table. Someone could help me? Thanks in advance. Best regards Carlos.
json_exemple.json
Description: application/json
jsonTable.sql
Description: Binary data