Hi,
We are making a rest URI request which hits Hive server 2 and translates in
to a hive query on TEZ.
Sample uri:
slc00yeb.us.oracle.com:8989/thirdeye-services/v1.0/OSCS/query/OSCSRecords.json?COMP_FAMILY=FMW&COMP_NAME=WLS&COMPTYPE1_FIRST=RC1&COMPTYPE1_SECOND=RC2&COMPTYPE2_FIRST=manifest.publish.label.WLS_GENERIC:carb-manifest:12.2.1.3.0-170722.1880245:pom&COMPTYPE2_SECOND=manifest.publish.label.WLS_GENERIC:carb-manifest:12.2.1.3.0-170726.1880744:pom&RELEASE_FIRST=12.2.1.3.0&RELEASE_SECOND=12.2.1.3.0&COMPTYPE=MANIFEST
Hadoop version : 2.4.0
Hive:1.2.2
Tez : 5.4.0
All my queries run fine in HIVE CLI , also via standalone program
connecting to hive via JDBC ODBC connection and returns hive query results,
but only when I hit the rest URL, results are not returned.
I have enabled the below settings for tez
tez.session.am.dag.submit.timeout.secs 90
tez.am.mode.session : true
tez.am.container.reuse.enable: true
tez.allow.disabled.timeline-domains : true
tez.yarn.ats.enabled : true
tez.use.cluster.hadoop-libs : true
The url on the browser keeps spinning without the json response, even after
the tez job on yarn cluster has finished successfully. Attached Tez job
diagnosis screenshot.
>From the dag log :
2018-03-19 11:34:36,367 INFO [AsyncDispatcher event handler] impl.DAGImpl:
dag_1521448362533_0019_1 transitioned from RUNNING to SUCCEEDED
2018-03-19 11:34:36,368 INFO [AsyncDispatcher event handler]
app.DAGAppMaster: DAG completed, dagId=dag_1521448362533_0019_1,
dagState=SUCCEEDED
{"entity":"dag_1521448362533_0019_1","entitytype":"TEZ_DAG_ID","events":[{"ts":1521459276335,"eventtype":"DAG_FINISHED"}],"otherinfo":{"startTime":1521459269619,"endTime":1521459276335,"timeTaken":6716,"status":"SUCCEEDED","diagnostics":"","counters":{"counterGroups":[{"counterGroupName":"org.apache.tez.common.counters.DAGCounter","counterGroupDisplayName":"org.apache.tez.common.counters.DAGCounter","counters":[{"counterName":"NUM_SUCCEEDED_TASKS","counterDisplayName":"NUM_SUCCEEDED_TASKS","counterValue":2},{"counterName":"TOTAL_LAUNCHED_TASKS","counterDisplayName":"TOTAL_LAUNCHED_TASKS","counterValue":2},{"counterName":"DATA_LOCAL_TASKS","counterDisplayName":"DATA_LOCAL_TASKS","counterValue":1}]},{"counterGroupName":"org.apache.tez.common.counters.FileSystemCounter","counterGroupDisplayName":"File
System
Counters","counters":[{"counterName":"FILE_BYTES_READ","counterDisplayName":"FILE_BYTES_READ","counterValue":41993},{"counterName":"FILE_BYTES_WRITTEN","counterDisplayName":"FILE_BYTES_WRITTEN","counterValue":83954},{"counterName":"FILE_READ_OPS","counterDisplayName":"FILE_READ_OPS","counterValue":0},{"counterName":"FILE_LARGE_READ_OPS","counterDisplayName":"FILE_LARGE_READ_OPS","counterValue":0},{"counterName":"FILE_WRITE_OPS","counterDisplayName":"FILE_WRITE_OPS","counterValue":0},{"counterName":"HDFS_BYTES_READ","counterDisplayName":"HDFS_BYTES_READ","counterValue":59231},{"counterName":"HDFS_BYTES_WRITTEN","counterDisplayName":"HDFS_BYTES_WRITTEN","counterValue":39631},{"counterName":"HDFS_READ_OPS","counterDisplayName":"HDFS_READ_OPS","counterValue":5},{"counterName":"HDFS_LARGE_READ_OPS","counterDisplayName":"HDFS_LARGE_READ_OPS","counterValue":0},{"counterName":"HDFS_WRITE_OPS","counterDisplayName":"HDFS_WRITE_OPS","counterValue":2}]},{"counterGroupName":"org.apache.tez.common.counters.TaskCounter","counterGroupDisplayName":"org.apache.tez.common.counters.TaskCounter","counters":[{"counterName":"REDUCE_INPUT_GROUPS","counterDisplayName":"REDUCE_INPUT_GROUPS","counterValue":190},{"counterName":"REDUCE_INPUT_RECORDS","counterDisplayName":"REDUCE_INPUT_RECORDS","counterValue":191},{"counterName":"COMBINE_INPUT_RECORDS","counterDisplayName":"COMBINE_INPUT_RECORDS","counterValue":0},{"counterName":"SPILLED_RECORDS","counterDisplayName":"SPILLED_RECORDS","counterValue":382},{"counterName":"NUM_SHUFFLED_INPUTS","counterDisplayName":"NUM_SHUFFLED_INPUTS","counterValue":1},{"counterName":"NUM_SKIPPED_INPUTS","counterDisplayName":"NUM_SKIPPED_INPUTS","counterValue":0},{"counterName":"NUM_FAILED_SHUFFLE_INPUTS","counterDisplayName":"NUM_FAILED_SHUFFLE_INPUTS","counterValue":0},{"counterName":"MERGED_MAP_OUTPUTS","counterDisplayName":"MERGED_MAP_OUTPUTS","counterValue":1},{"counterName":"GC_TIME_MILLIS","counterDisplayName":"GC_TIME_MILLIS","counterValue":84},{"counterName":"CPU_MILLISECONDS","counterDisplayName":"CPU_MILLISECONDS","counterValue":5520},{"counterName":"PHYSICAL_MEMORY_BYTES","counterDisplayName":"PHYSICAL_MEMORY_BYTES","counterValue":661254144},{"counterName":"VIRTUAL_MEMORY_BYTES","counterDisplayName":"VIRTUAL_MEMORY_BYTES","counterValue":4294270976},{"counterName":"COMMITTED_HEAP_BYTES","counterDisplayName":"COMMITTED_HEAP_BYTES","counterValue":402653184},{"counterName":"INPUT_RECORDS_PROCESSED","counterDisplayName":"INPUT_RECORDS_PROCESSED","counterValue":8445},{"counterName":"OUTPUT_RECORDS","counterDisplayName":"OUTPUT_RECORDS","counterValue":191},{"counterName":"OUTPUT_BYTES","counterDisplayName":"OUTPUT_BYTES","counterValue":41350},{"counterName":"OUTPUT_BYTES_WITH_OVERHEAD","counterDisplayName":"OUTPUT_BYTES_WITH_OVERHEAD","counterValue":41957},{"counterName":"OUTPUT_BYTES_PHYSICAL","counterDisplayName":"OUTPUT_BYTES_PHYSICAL","counterValue":41961},{"counterName":"ADDITIONAL_SPILLS_BYTES_WRITTEN","counterDisplayName":"ADDITIONAL_SPILLS_BYTES_WRITTEN","counterValue":41961},{"counterName":"ADDITIONAL_SPILLS_BYTES_READ","counterDisplayName":"ADDITIONAL_SPILLS_BYTES_READ","counterValue":41961},{"counterName":"ADDITIONAL_SPILL_COUNT","counterDisplayName":"ADDITIONAL_SPILL_COUNT","counterValue":0},{"counterName":"SHUFFLE_BYTES","counterDisplayName":"SHUFFLE_BYTES","counterValue":41961},{"counterName":"SHUFFLE_BYTES_DECOMPRESSED","counterDisplayName":"SHUFFLE_BYTES_DECOMPRESSED","counterValue":41957},{"counterName":"SHUFFLE_BYTES_TO_MEM","counterDisplayName":"SHUFFLE_BYTES_TO_MEM","counterValue":41961},{"counterName":"SHUFFLE_BYTES_TO_DISK","counterDisplayName":"SHUFFLE_BYTES_TO_DISK","counterValue":0},{"counterName":"SHUFFLE_BYTES_DISK_DIRECT","counterDisplayName":"SHUFFLE_BYTES_DISK_DIRECT","counterValue":0},{"counterName":"NUM_MEM_TO_DISK_MERGES","counterDisplayName":"NUM_MEM_TO_DISK_MERGES","counterValue":0},{"counterName":"NUM_DISK_TO_DISK_MERGES","counterDisplayName":"NUM_DISK_TO_DISK_MERGES","counterValue":0}]},{"counterGroupName":"HIVE","counterGroupDisplayName":"HIVE","counters":[{"counterName":"CREATED_FILES","counterDisplayName":"CREATED_FILES","counterValue":1},{"counterName":"DESERIALIZE_ERRORS","counterDisplayName":"DESERIALIZE_ERRORS","counterValue":0},{"counterName":"RECORDS_IN_Map_1","counterDisplayName":"RECORDS_I
But the json response isnt coming. What am I missing here?
Thanks,
Jyotsna
