[jira] [Created] (HIVE-17301) Make JSONMessageFactory.getTObj method thread safe

[Tao Li (JIRA)]

Tao Li created HIVE-17301:
-

 Summary: Make JSONMessageFactory.getTObj method thread safe
 Key: HIVE-17301
 URL: https://issues.apache.org/jira/browse/HIVE-17301
 Project: Hive
  Issue Type: Bug
Reporter: Tao Li


This static method is using a singleton instance of TDeserializer, which is not 
thread safe. Instead we want to create a new instance per method call. This 
class is lightweight, so it should be fine from perf perspective. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17299) Cannot validate SerDe even if it is in Hadoop classpath

[JIRA]

Loïc C. Chanel created HIVE-17299:
-

 Summary: Cannot validate SerDe even if it is in Hadoop classpath
 Key: HIVE-17299
 URL: https://issues.apache.org/jira/browse/HIVE-17299
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Affects Versions: 1.2.1
 Environment: HADOOP_CLASSPATH : 
/usr/hdp/2.3.4.0-3485/atlas/hook/hive/*:/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1.2.3.4.0-3485.jar:/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-server-extensions-1.2.1.2.3.4.0-3485.jar:/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/webhcat/java-client/hive-webhcat-java-client-1.2.1.2.3.4.0-3485.jar

2017-08-10 15:26:38,924 INFO  [main]: zookeeper.ZooKeeper 
(Environment.java:logEnv(100)) - Client 

[GitHub] hive pull request #227: HIVE-17289: EXPORT and IMPORT shouldn't perform dist...

[sankarh]

GitHub user sankarh opened a pull request:

https://github.com/apache/hive/pull/227

HIVE-17289: EXPORT and IMPORT shouldn't perform distcp with doAs privileged 
user.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sankarh/hive HIVE-17289

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/227.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #227


commit 29072b2905c1ffa83141a5d29f6095869e68a4e7
Author: Sankar Hariappan 
Date:   2017-08-11T08:15:02Z

HIVE-17289: EXPORT and IMPORT shouldn't perform distcp with doAs privileged 
user.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: adding the column to hive partition

[Chaoyu Tang]

Is not it right that the values for the new column of existing partition
should be null?

On Thu, Aug 10, 2017 at 9:53 PM, mallik new  wrote:

> Hello All,
>
>  how to add the column to already partitioned table in hive, after i have
> added the column using cascade, the new column is showing null values, how
> to eliminate nulls and how this column pick the values.
>
>
>  looking for your support.
>
> thanks,
> Mallik.
>

Re: Review Request 61439: HIVE-17256 add a notion of a guaranteed task to LLAP

[Sergey Shelukhin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61439/
---

(Updated Aug. 11, 2017, 7:41 p.m.)


Review request for hive and Siddharth Seth.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  
llap-common/src/gen/protobuf/gen-java/org/apache/hadoop/hive/llap/daemon/rpc/LlapDaemonProtocolProtos.java
 c19cf6358a 
  
llap-common/src/java/org/apache/hadoop/hive/llap/impl/LlapProtocolClientImpl.java
 3681797eed 
  llap-common/src/protobuf/LlapDaemonProtocol.proto e0c007061c 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/ContainerRunner.java 
c346aed3ba 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/ContainerRunnerImpl.java
 e030a765f7 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapDaemon.java 
68ef200f07 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapProtocolServerImpl.java
 eb7a8ebb9c 
  llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/Scheduler.java 
f19959379a 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorService.java
 047c504131 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/TaskRunnerCallable.java
 5159e2dcbd 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/FirstInFirstOutComparator.java
 8beb47ce6d 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/LlapQueueComparatorBase.java
 PRE-CREATION 
  
llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java
 e09b7e8797 
  
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TaskExecutorTestHelpers.java
 1fa76102a3 
  
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TestTaskExecutorService.java
 df563f4f5d 
  
llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/comparator/TestShortestJobFirstComparator.java
 a210499a2e 


Diff: https://reviews.apache.org/r/61439/diff/2/

Changes: https://reviews.apache.org/r/61439/diff/1-2/


Testing
---


Thanks,

Sergey Shelukhin

[GitHub] hive pull request #220: HIVE-17224 Moved all the JDO classes and package.jdo

[alanfgates]

Github user alanfgates closed the pull request at:

https://github.com/apache/hive/pull/220


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (HIVE-17302) ReduceRecordSource should not add batch string to Exception message

[slim bouguerra (JIRA)]

slim bouguerra created HIVE-17302:
-

 Summary: ReduceRecordSource should not add batch string to 
Exception message
 Key: HIVE-17302
 URL: https://issues.apache.org/jira/browse/HIVE-17302
 Project: Hive
  Issue Type: Bug
Reporter: slim bouguerra


ReduceRecordSource is adding the batch data as a string to the exception stack, 
this can lead to an OOM of the Query AM when the query fails due to other issue.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HIVE-17303) Missmatch between roaring bitmap library used by druid and the one coming from tez

[slim bouguerra (JIRA)]

slim bouguerra created HIVE-17303:
-

 Summary: Missmatch between roaring bitmap library used by druid 
and the one coming from tez
 Key: HIVE-17303
 URL: https://issues.apache.org/jira/browse/HIVE-17303
 Project: Hive
  Issue Type: Bug
  Components: Druid integration
Reporter: slim bouguerra
Assignee: slim bouguerra


{code} 


 
Caused by: java.util.concurrent.ExecutionException: 
java.lang.NoSuchMethodError: 
org.roaringbitmap.buffer.MutableRoaringBitmap.runOptimize()Z
  at 
org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
  at 
org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
  at 
org.apache.hive.druid.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
  at 
org.apache.hadoop.hive.druid.io.DruidRecordWriter.pushSegments(DruidRecordWriter.java:165)
  ... 25 more
Caused by: java.lang.NoSuchMethodError: 
org.roaringbitmap.buffer.MutableRoaringBitmap.runOptimize()Z
  at 
org.apache.hive.druid.com.metamx.collections.bitmap.WrappedRoaringBitmap.toImmutableBitmap(WrappedRoaringBitmap.java:65)
  at 
org.apache.hive.druid.com.metamx.collections.bitmap.RoaringBitmapFactory.makeImmutableBitmap(RoaringBitmapFactory.java:88)
  at 
org.apache.hive.druid.io.druid.segment.StringDimensionMergerV9.writeIndexes(StringDimensionMergerV9.java:348)
  at 
org.apache.hive.druid.io.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:218)
  at 
org.apache.hive.druid.io.druid.segment.IndexMerger.merge(IndexMerger.java:438)
  at 
org.apache.hive.druid.io.druid.segment.IndexMerger.persist(IndexMerger.java:186)
  at 
org.apache.hive.druid.io.druid.segment.IndexMerger.persist(IndexMerger.java:152)
  at 
org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl.persistHydrant(AppenderatorImpl.java:996)
  at 
org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl.access$200(AppenderatorImpl.java:93)
  at 
org.apache.hive.druid.io.druid.segment.realtime.appenderator.AppenderatorImpl$2.doCall(AppenderatorImpl.java:385)
  at 
org.apache.hive.druid.io.druid.common.guava.ThreadRenamingCallable.call(ThreadRenamingCallable.java:44)
  ... 4 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 
killedTasks:89, Vertex vertex_1502470020457_0005_12_05 [Reducer 2] 
killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to 
VERTEX_FAILURE. failedVertices:1 killedVertices:0 (state=08S01,code=2)

Options

Attachments
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61586: HIVE-17286

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61586/
---

(Updated Aug. 12, 2017, 12:16 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-17286
https://issues.apache.org/jira/browse/HIVE-17286


Repository: hive-git


Description (updated)
---

HIVE-17286-2


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/ndv/FMSketch.java 
160ce663ba5068ad029d51f431b87a40f97341a5 
  
common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimator.java
 4517b694ee38fd02ff32430ef24720d91c092f3a 
  
common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimatorFactory.java
 6a29859df56f7d41b87d95242367f6ef401b4060 
  common/src/java/org/apache/hadoop/hive/common/ndv/fm/FMSketchUtils.java 
b6f7fdda0c9363516f610d5280c3c4e0821cba09 
  common/src/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java 
182560afbe6a8b9e2ac10591a520603bac0eb8d5 
  
common/src/test/org/apache/hadoop/hive/common/ndv/fm/TestFMSketchSerialization.java
 74fdf58d2d3640a9e923e3dbd96f0704ba3b5f35 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
b3274ca6ddd5377187190c896cb058e8686f74c2 
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java 
d53ea4c5b2f294bbd4d530ce06a9a3fd1e640bde 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/ColumnStatsAggregatorFactory.java
 173e06fe8e6b3808dfe76c8e7266472ff822e186 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DateColumnStatsAggregator.java
 04a1eb5d8933eb6644f992c5b742b94a9fbaa22b 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DecimalColumnStatsAggregator.java
 d220e7fc86e28cb6e10ddb3679be3264470f7ebe 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DoubleColumnStatsAggregator.java
 1b44dd946f99daf65b91c824224d87f9a24de384 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/LongColumnStatsAggregator.java
 802ad1a1b185fb619aa07c818107a057dc96ba2c 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/StringColumnStatsAggregator.java
 e1a781fde977fcd9e5bba570468a48cda4c69611 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/cache/DateColumnStatsDataInspector.java
 PRE-CREATION 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/cache/DecimalColumnStatsDataInspector.java
 PRE-CREATION 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/cache/DoubleColumnStatsDataInspector.java
 PRE-CREATION 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/cache/LongColumnStatsDataInspector.java
 PRE-CREATION 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/cache/StringColumnStatsDataInspector.java
 PRE-CREATION 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/merge/ColumnStatsMergerFactory.java
 0ce1847d1cbc1007ebd0bb2bf665659479e9e626 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/merge/DateColumnStatsMerger.java
 2542a00d361b6b2be532b9288b5958c680fa40fa 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/merge/DecimalColumnStatsMerger.java
 4e8e1297585c18edc40fa71a3e33c0310b99750a 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/merge/DoubleColumnStatsMerger.java
 4ef5c39d1c107a329023f38769b60045b7499fb4 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/merge/LongColumnStatsMerger.java
 acf7f03c72adb64696f5d7c3e0cd1bb64e34f7fd 
  
metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/merge/StringColumnStatsMerger.java
 b3cd33c671ec8b478e233124671627238288a4e3 
  metastore/src/test/org/apache/hadoop/hive/metastore/TestOldSchema.java 
54828f2289657470e1da02cdb938dd14ef6488af 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 
d96f432fee9f4dde4ba76a21a28005299d3f9f7a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java 
2acc777eacc5e482318807cf171a2f31eea52504 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
 23800734f7f7a00e363b367db2e197bd3b4f640a 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 
8ee41bfab2b44e6c8b345fe5d9e3f6b30d2a328c 
  ql/src/test/results/clientpositive/autoColumnStats_4.q.out 
e8445b2c5cae1d84ef4f37532423dd0eac9b 
  ql/src/test/results/clientpositive/autoColumnStats_5.q.out 
29963975d38aa89a96744645a097278b7ce64f1f 
  ql/src/test/results/clientpositive/autoColumnStats_6.q.out 
1b125701d7c92fea66d1e6234e9a86a7b1922b17 
  ql/src/test/results/clientpositive/autoColumnStats_7.q.out 
9e2121e0deae1deb37b7f1222c22aea8c6839f48 
  ql/src/test/results/clientpositive/autoColumnStats_8.q.out 
cdf2082d53356d18ed445b0d04b6673133878b2d 
  ql/src/test/results/clientpositive/autoColumnStats_9.q.out 

[GitHub] hive pull request #228: HIVE-17241 Removed shims from metastore. For HDFS an...

[alanfgates]

GitHub user alanfgates opened a pull request:

https://github.com/apache/hive/pull/228

HIVE-17241 Removed shims from metastore.  For HDFS and getPassword I just 
access…

… those operations directly.  I copied all of the HadoopThriftAuthBridge 
stuff over from Hive common.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alanfgates/hive hive17241

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/228.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #228






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Review Request 61604: HIVE-17265

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61604/
---

(Updated Aug. 12, 2017, 2:16 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-17265
https://issues.apache.org/jira/browse/HIVE-17265


Repository: hive-git


Description
---

HIVE-17265


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/CompilationOpContext.java 
949f8738f8bf2a707e0b4514ef2aa5ca48ed7dd6 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
 56c016311d8141071a60d2a431749e4856ffbb66 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
1d49568c8cda2aa6c27568f023dc39ec20ab1949 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 
66248654e8c0b7d242a089f5ac1a624dfb06598d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 6cb0559ea04abebb1da38e93c2162666cdf95af9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
d6695ccbf2611551d2038582df7faa69efd4013f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 
565fbef0aa1a63c50679a4057cb23e8c0092bc7d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/PrunedPartitionList.java 
da2e1e2d8b06d04992763288709c423a4f048b4e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
4faec05bb0f7c728453fa8bd3a5ca2a6966048d4 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 
3b9ab41bed25cc4cb0122842897fffbcd0ec15c4 


Diff: https://reviews.apache.org/r/61604/diff/1/


Testing
---


Thanks,

Jesús Camacho Rodríguez

Review Request 61604: HIVE-17265

[Jesús Camacho Rodríguez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61604/
---

Review request for hive.


Repository: hive-git


Description
---

HIVE-17265


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/CompilationOpContext.java 
949f8738f8bf2a707e0b4514ef2aa5ca48ed7dd6 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
 56c016311d8141071a60d2a431749e4856ffbb66 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
1d49568c8cda2aa6c27568f023dc39ec20ab1949 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 
66248654e8c0b7d242a089f5ac1a624dfb06598d 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 6cb0559ea04abebb1da38e93c2162666cdf95af9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
d6695ccbf2611551d2038582df7faa69efd4013f 
  ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 
565fbef0aa1a63c50679a4057cb23e8c0092bc7d 
  ql/src/java/org/apache/hadoop/hive/ql/parse/PrunedPartitionList.java 
da2e1e2d8b06d04992763288709c423a4f048b4e 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
4faec05bb0f7c728453fa8bd3a5ca2a6966048d4 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 
3b9ab41bed25cc4cb0122842897fffbcd0ec15c4 


Diff: https://reviews.apache.org/r/61604/diff/1/


Testing
---


Thanks,

Jesús Camacho Rodríguez

[jira] [Created] (HIVE-17304) ThreadMXBean based memory allocation monitory for hash table loader

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-17304:


 Summary: ThreadMXBean based memory allocation monitory for hash 
table loader
 Key: HIVE-17304
 URL: https://issues.apache.org/jira/browse/HIVE-17304
 Project: Hive
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


Hash table memory monitoring is based on java data model which can be 
unreliable because of various reasons (wrong object size estimation, adding new 
variables to any class without accounting its size for memory monitoring, 
etc.). We can use allocation size per thread that is provided by ThreadMXBean 
and fallback to DataModel in case if JDK doesn't support thread based 
allocations. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSSION] WebUI query plan graphs

[Karen Coppage]

Hi Xuefu,

Thanks so much for your reply!

I created a JIRA as per your suggestion; it can be found here along with
actually visible screenshots:
https://issues.apache.org/jira/browse/HIVE-17300

Any and all input, comments, criticism will be very welcome.

Thanks again,

Karen



On Thu, Aug 10, 2017 at August 10, 2017 at 6:39 AM, Xuefu Zhang <
xu...@apache.org> wrote:

Hi Karen,

Thanks for reaching out. While your message doesn't seem showing any
images, I think the feature would be a great addition to Hive. (Hive
community always welcomes contributions like this.)

Please feel free to create an JIRA for easier discussion and tracking.

Thanks again for your interest.

--Xuefu


On Thu, Aug 10, 2017 at 6:25 AM, Karen Coppage 
wrote:

Hi all,

I’m working on a feature of the Hive WebUI Query Plan tab that would
provide the option to display the query plan as a nice graph (scroll down
for screenshots). If you click on one of the graph’s stages, the plan for
that stage appears as text below.

Stages are color-coded if they have a status (Success, Error, Running), and
the rest are grayed out. Coloring is based on status already available in
the WebUI, under the Stages tab.

There is an additional option to display stats for MapReduce tasks. This
includes the job’s ID, tracking URL (where the logs are found), and mapper
and reducer numbers/progress, among other info.

The library I’m using for the graph is called vis.js (http://visjs.org/).
It has an Apache license, and the only necessary file to be included from
this library is about 700 KB.

I tried to keep server-side changes minimal, and graph generation is taken
care of by the client. Plans with more than a given number of stages
(default: 25) won't be displayed in order to preserve resources.

I’d love to hear any and all input from the community about this feature:
do you think it’s useful, and is there anything important I’m missing?

Thanks,

Karen Coppage

*

A completely successful query:

[image: Inline image 1]


A MapReduce task selected, with MapReduce stats view on:

[image: Inline image 2]


Full MapReduce stats, lacking some information because the query was run in
local mode:

[image: Inline image 3]


A non-MapReduce stage selected:

[image: Inline image 4]


Last stage running:

[image: Inline image 5]


Last stage returns error:

[image: Inline image 6]

[jira] [Created] (HIVE-17300) WebUI query plan graphs

[Karen Coppage (JIRA)]

Karen Coppage created HIVE-17300:


 Summary: WebUI query plan graphs
 Key: HIVE-17300
 URL: https://issues.apache.org/jira/browse/HIVE-17300
 Project: Hive
  Issue Type: Improvement
  Components: Web UI
Reporter: Karen Coppage


Hi all,

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below. 

Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.

There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info. 

The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.

I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.

I’d love to hear any and all input from the community about this feature: do 
you think it’s useful, and is there anything important I’m missing?

Thanks,

Karen Coppage



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)