[jira] [Commented] (HIVE-17178) Spark Partition Pruning Sink Operator can't target multiple Works

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388480#comment-16388480 ] Sahil Takiar commented on HIVE-17178: - [~lirui] sorry for the delay. Mostly looks good

[jira] [Commented] (HIVE-15104) Hive on Spark generate more shuffle data than hive on mr

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-15104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388551#comment-16388551 ] Sahil Takiar commented on HIVE-15104: - Hey [~lirui] I found some time to do some inter

[jira] [Assigned] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-18883: --- > Add findbugs to yetus pre-commit checks > --- > >

[jira] [Commented] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388657#comment-16388657 ] Sahil Takiar commented on HIVE-18883: - [~szita], [~pvary] any tips on how to do this?

[jira] [Updated] (HIVE-18034) Improving logging with HoS executors spend lots of time in GC

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18034: Attachment: HIVE-18034.5.patch > Improving logging with HoS executors spend lots of time in GC > ---

[jira] [Updated] (HIVE-18034) Improving logging with HoS executors spend lots of time in GC

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18034: Attachment: (was: HIVE-18034.5.patch) > Improving logging with HoS executors spend lots of time

[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.1.patch > Differentiate errors that are thrown by Spark tasks > -

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1631#comment-1631 ] Sahil Takiar commented on HIVE-18831: - Before this patch the console output would look

[jira] [Assigned] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-18831: --- Assignee: Sahil Takiar > Differentiate errors that are thrown by Spark tasks > --

[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Status: Patch Available (was: Open) > Differentiate errors that are thrown by Spark tasks > ---

[jira] [Commented] (HIVE-18892) Fix NPEs in HiveMetastore.exchange_partitions method

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389801#comment-16389801 ] Sahil Takiar commented on HIVE-18892: - +1 pending Hive QA > Fix NPEs in HiveMetastore

[jira] [Commented] (HIVE-18436) Upgrade to Spark 2.3.0

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390151#comment-16390151 ] Sahil Takiar commented on HIVE-18436: - [Spark 2.3.0|https://spark.apache.org/releases/

[jira] [Commented] (HIVE-18888) Replace synchronizedMap with ConcurrentHashMap

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390463#comment-16390463 ] Sahil Takiar commented on HIVE-1: - +1 pending Peter's comment. > Replace synchron

[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.2.patch > Differentiate errors that are thrown by Spark tasks > -

[jira] [Commented] (HIVE-18869) Improve SparkTask OOM Error Parsing Logic

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390546#comment-16390546 ] Sahil Takiar commented on HIVE-18869: - When testing this, this is the error the Spark

[jira] [Updated] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18883: Attachment: HIVE-18883.1.patch > Add findbugs to yetus pre-commit checks > -

[jira] [Updated] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18883: Status: Patch Available (was: Open) > Add findbugs to yetus pre-commit checks > ---

[jira] [Updated] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18883: Attachment: HIVE-18883.1.patch > Add findbugs to yetus pre-commit checks > -

[jira] [Updated] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18883: Attachment: (was: HIVE-18883.1.patch) > Add findbugs to yetus pre-commit checks > --

[jira] [Commented] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390680#comment-16390680 ] Sahil Takiar commented on HIVE-18883: - Thanks for the tip [~pvary]! Helped a lot. What

[jira] [Commented] (HIVE-18436) Upgrade to Spark 2.3.0

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391477#comment-16391477 ] Sahil Takiar commented on HIVE-18436: - Test changes are due to the Netty upgrade. Nett

[jira] [Commented] (HIVE-16855) org.apache.hadoop.hive.ql.exec.mr.HashTableLoader Improvements

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391766#comment-16391766 ] Sahil Takiar commented on HIVE-16855: - I agree the call to {{Arrays#fill}} is weird, b

[jira] [Updated] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18883: Attachment: HIVE-18883.2.patch > Add findbugs to yetus pre-commit checks > -

[jira] [Commented] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391980#comment-16391980 ] Sahil Takiar commented on HIVE-18883: - Hive QA hasn't run yet, but attaching updated p

[jira] [Updated] (HIVE-18034) Improving logging with HoS executors spend lots of time in GC

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18034: Attachment: HIVE-18034.6.patch > Improving logging with HoS executors spend lots of time in GC > ---

[jira] [Commented] (HIVE-18901) Lower ResourceDownloader Logging to Debug

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392011#comment-16392011 ] Sahil Takiar commented on HIVE-18901: - Can you give an example of when this is causing

[jira] [Commented] (HIVE-16855) org.apache.hadoop.hive.ql.exec.mr.HashTableLoader Improvements

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392014#comment-16392014 ] Sahil Takiar commented on HIVE-16855: - +1 pending Hive QA > org.apache.hadoop.hive.ql

[jira] [Commented] (HIVE-18878) Lower MoveTask Lock Logging to Debug

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392025#comment-16392025 ] Sahil Takiar commented on HIVE-18878: - Do you have an example of when this causes a lo

[jira] [Assigned] (HIVE-18917) Add spark.home to hive.conf.restricted.list

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-18917: --- > Add spark.home to hive.conf.restricted.list > --- > >

[jira] [Updated] (HIVE-18917) Add spark.home to hive.conf.restricted.list

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18917: Attachment: HIVE-18917.1.patch > Add spark.home to hive.conf.restricted.list > -

[jira] [Updated] (HIVE-18917) Add spark.home to hive.conf.restricted.list

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18917: Status: Patch Available (was: Open) > Add spark.home to hive.conf.restricted.list > ---

[jira] [Commented] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392163#comment-16392163 ] Sahil Takiar commented on HIVE-18831: - Fixing test failure. > Differentiate errors th

[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.3.patch > Differentiate errors that are thrown by Spark tasks > -

[jira] [Updated] (HIVE-18915) Better client logging when a HoS session can't be opened

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18915: Component/s: Spark > Better client logging when a HoS session can't be opened >

[jira] [Updated] (HIVE-18916) SparkClientImpl doesn't error out if spark-submit fails

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18916: Component/s: Spark > SparkClientImpl doesn't error out if spark-submit fails > -

[jira] [Updated] (HIVE-18093) Improve logging when HoS application is killed

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18093: Attachment: HIVE-18093.2.patch > Improve logging when HoS application is killed > --

[jira] [Commented] (HIVE-17178) Spark Partition Pruning Sink Operator can't target multiple Works

2018-03-08 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392328#comment-16392328 ] Sahil Takiar commented on HIVE-17178: - +1 > Spark Partition Pruning Sink Operator can

[jira] [Commented] (HIVE-18917) Add spark.home to hive.conf.restricted.list

2018-03-09 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393413#comment-16393413 ] Sahil Takiar commented on HIVE-18917: - [~aihuaxu], [~vihangk1] could one of you take a

[jira] [Commented] (HIVE-18883) Add findbugs to yetus pre-commit checks

2018-03-09 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393427#comment-16393427 ] Sahil Takiar commented on HIVE-18883: - [~pvary] is there a good way to test this on Hi

[jira] [Updated] (HIVE-18831) Differentiate errors that are thrown by Spark tasks

2018-03-09 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18831: Attachment: HIVE-18831.4.patch > Differentiate errors that are thrown by Spark tasks > -

[jira] [Updated] (HIVE-18436) Upgrade to Spark 2.3.0

2018-03-09 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18436: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pu

[jira] [Updated] (HIVE-18915) Better client logging when a HoS session can't be opened

2018-03-09 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18915: Description: Users just get a {{FAILED: Execution Error, return code 30041 from org.apache.hadoop.hi

[jira] [Updated] (HIVE-18917) Add spark.home to hive.conf.restricted.list

2018-03-10 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18917: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pu

[jira] [Updated] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-11 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18533: Attachment: HIVE-18533.1.patch > Add option to use InProcessLauncher to submit spark jobs >

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-11 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394793#comment-16394793 ] Sahil Takiar commented on HIVE-18533: - Attaching WIP patch with the {{SparkLauncher}}

[jira] [Updated] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-11 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18533: Status: Patch Available (was: Open) > Add option to use InProcessLauncher to submit spark jobs > --

[jira] [Commented] (HIVE-18901) Lower ResourceDownloader Logging to Debug

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395500#comment-16395500 ] Sahil Takiar commented on HIVE-18901: - Thanks for the explanation [~belugabehr]. Makes

[jira] [Commented] (HIVE-18922) Hive is not cleaning up staging directories

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395744#comment-16395744 ] Sahil Takiar commented on HIVE-18922: - [~infinitymittal] can you provide some examples

[jira] [Commented] (HIVE-18922) Hive is not cleaning up staging directories

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395758#comment-16395758 ] Sahil Takiar commented on HIVE-18922: - Is this just when {{MERGE INTO}} queries are ru

[jira] [Updated] (HIVE-16855) org.apache.hadoop.hive.ql.exec.mr.HashTableLoader Improvements

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-16855: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pu

[jira] [Commented] (HIVE-18343) Remove LinkedList from ColumnStatsSemanticAnalyzer.java

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395896#comment-16395896 ] Sahil Takiar commented on HIVE-18343: - Can you create an RB for this? Its hard to revi

[jira] [Assigned] (HIVE-18932) Better console logging for lifecycle of a Spark job

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-18932: --- > Better console logging for lifecycle of a Spark job > -

[jira] [Updated] (HIVE-18034) Improving logging with HoS executors spend lots of time in GC

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18034: Attachment: HIVE-18034.7.patch > Improving logging with HoS executors spend lots of time in GC > ---

[jira] [Commented] (HIVE-18916) SparkClientImpl doesn't error out if spark-submit fails

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396254#comment-16396254 ] Sahil Takiar commented on HIVE-18916: - hmm yeah I'll have to dig out the stack-trace.

[jira] [Commented] (HIVE-18344) Remove LinkedList from SharedWorkOptimizer.java

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396353#comment-16396353 ] Sahil Takiar commented on HIVE-18344: - Think this needs to be rebased. Patch doesn't a

[jira] [Updated] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18533: Attachment: HIVE-18533.2.patch > Add option to use InProcessLauncher to submit spark jobs >

[jira] [Updated] (HIVE-18888) Replace synchronizedMap with ConcurrentHashMap

2018-03-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-1: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pu

[jira] [Commented] (HIVE-18922) Hive is not cleaning up staging directories

2018-03-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397491#comment-16397491 ] Sahil Takiar commented on HIVE-18922: - CC: [~ashutoshc] > Hive is not cleaning up st

[jira] [Assigned] (HIVE-18947) Add integration tests for HoS + kerberos

2018-03-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-18947: --- > Add integration tests for HoS + kerberos > - > >

[jira] [Updated] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18533: Attachment: HIVE-18533.3.patch > Add option to use InProcessLauncher to submit spark jobs >

[jira] [Commented] (HIVE-18343) Remove LinkedList from ColumnStatsSemanticAnalyzer.java

2018-03-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397899#comment-16397899 ] Sahil Takiar commented on HIVE-18343: - +1 > Remove LinkedList from ColumnStatsSemanti

[jira] [Updated] (HIVE-18343) Remove LinkedList from ColumnStatsSemanticAnalyzer.java

2018-03-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18343: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pu

[jira] [Updated] (HIVE-18034) Improving logging with HoS executors spend lots of time in GC

2018-03-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18034: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pu

[jira] [Commented] (HIVE-16858) Acumulo Utils Improvements

2018-03-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-16858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399507#comment-16399507 ] Sahil Takiar commented on HIVE-16858: - +1 > Acumulo Utils Improvements >

[jira] [Commented] (HIVE-18342) Remove LinkedList from HiveAlterHandler.java

2018-03-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399508#comment-16399508 ] Sahil Takiar commented on HIVE-18342: - +1 > Remove LinkedList from HiveAlterHandler.j

[jira] [Updated] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

2018-03-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18533: Attachment: HIVE-18533.4.patch > Add option to use InProcessLauncher to submit spark jobs >

[jira] [Commented] (HIVE-18344) Remove LinkedList from SharedWorkOptimizer.java

2018-03-14 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-18344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399603#comment-16399603 ] Sahil Takiar commented on HIVE-18344: - +1 > Remove LinkedList from SharedWorkOptimize

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-22 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588781#comment-16588781 ] Sahil Takiar commented on HIVE-14162: - [~szita], [~asinkovits] could you take a look?

[jira] [Commented] (HIVE-19814) RPC Server port is always random for spark

2018-08-22 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589134#comment-16589134 ] Sahil Takiar commented on HIVE-19814: - [~bharos92] the new unit test looks good. Just

[jira] [Commented] (HIVE-20479) Update content/people.mdtext in cms

2018-08-28 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595364#comment-16595364 ] Sahil Takiar commented on HIVE-20479: - +1 > Update content/people.mdtext in cms > -

[jira] [Updated] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-30 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-14162: Attachment: HIVE-14162.8.patch > Allow disabling of long running job on Hive On Spark On YARN > ---

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-30 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597637#comment-16597637 ] Sahil Takiar commented on HIVE-14162: - [~szita] thanks for taking a look. Addressed y

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-31 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598793#comment-16598793 ] Sahil Takiar commented on HIVE-14162: - Thanks Adam. Test failures don't look related,

[jira] [Updated] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-31 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-14162: Attachment: HIVE-14162.9.patch > Allow disabling of long running job on Hive On Spark On YARN > ---

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-31 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599260#comment-16599260 ] Sahil Takiar commented on HIVE-14162: - {{TestCliDriver.testCliDriver[test_teradatabin

[jira] [Updated] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-31 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-14162: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) P

[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN

2018-08-31 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599265#comment-16599265 ] Sahil Takiar commented on HIVE-14162: - [~Tagar] it was pushed to master, which is Hiv

[jira] [Updated] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-04 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17684: Attachment: HIVE-17684.05.patch > HoS memory issues with MapJoinMemoryExhaustionHandler > -

[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-04 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16603530#comment-16603530 ] Sahil Takiar commented on HIVE-17684: - [~mi...@cloudera.com] sorry for the delay on t

[jira] [Commented] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits

2018-09-05 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604809#comment-16604809 ] Sahil Takiar commented on HIVE-20506: - [~brocknoland] I think you might be hitting th

[jira] [Commented] (HIVE-20440) Create better cache eviction policy for SmallTableCache

2018-09-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606353#comment-16606353 ] Sahil Takiar commented on HIVE-20440: - Few high level comments: (1) The Guava cache

[jira] [Updated] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17684: Attachment: HIVE-17684.06.patch > HoS memory issues with MapJoinMemoryExhaustionHandler > -

[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606358#comment-16606358 ] Sahil Takiar commented on HIVE-17684: - [~mi...@cloudera.com] will do. The logs got cl

[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-06 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606614#comment-16606614 ] Sahil Takiar commented on HIVE-17684: - [~mi...@cloudera.com] took a look through some

[jira] [Commented] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits

2018-09-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607431#comment-16607431 ] Sahil Takiar commented on HIVE-20506: - The general idea makes sense to me. To confirm

[jira] [Commented] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits

2018-09-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607614#comment-16607614 ] Sahil Takiar commented on HIVE-20506: - Yeah, I'll file a follow up JIRA to do the re-

[jira] [Assigned] (HIVE-20519) Remove 30m min value for hive.spark.session.timeout

2018-09-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned HIVE-20519: --- > Remove 30m min value for hive.spark.session.timeout >

[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-07 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607622#comment-16607622 ] Sahil Takiar commented on HIVE-17684: - [~mi...@cloudera.com] yes we can configure it

[jira] [Commented] (HIVE-20440) Create better cache eviction policy for SmallTableCache

2018-09-10 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609496#comment-16609496 ] Sahil Takiar commented on HIVE-20440: - Then it might be better to have a two level ca

[jira] [Commented] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits

2018-09-10 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609504#comment-16609504 ] Sahil Takiar commented on HIVE-20506: - I think you can ignore it. PTest is able to ap

[jira] [Commented] (HIVE-19814) RPC Server port is always random for spark

2018-09-11 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610771#comment-16610771 ] Sahil Takiar commented on HIVE-19814: - +1 pending Hive QA > RPC Server port is alway

[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-11 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610774#comment-16610774 ] Sahil Takiar commented on HIVE-17684: - Sounds good to me. > HoS memory issues with M

[jira] [Commented] (HIVE-19814) RPC Server port is always random for spark

2018-09-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612515#comment-16612515 ] Sahil Takiar commented on HIVE-19814: - I think this patch is good to merge. I have a

[jira] [Updated] (HIVE-20519) Remove 30m min value for hive.spark.session.timeout

2018-09-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-20519: Attachment: HIVE-20519.1.patch > Remove 30m min value for hive.spark.session.timeout >

[jira] [Updated] (HIVE-20519) Remove 30m min value for hive.spark.session.timeout

2018-09-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-20519: Status: Patch Available (was: Open) > Remove 30m min value for hive.spark.session.timeout > --

[jira] [Updated] (HIVE-19814) RPC Server port is always random for spark

2018-09-12 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-19814: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) P

[jira] [Commented] (HIVE-20440) Create better cache eviction policy for SmallTableCache

2018-09-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613811#comment-16613811 ] Sahil Takiar commented on HIVE-20440: - Makes sense to me. > Create better cache evic

[jira] [Updated] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-17684: Attachment: HIVE-17684.08.patch > HoS memory issues with MapJoinMemoryExhaustionHandler > -

[jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler

2018-09-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614097#comment-16614097 ] Sahil Takiar commented on HIVE-17684: - [~mi...@cloudera.com] I attached an updated pa

[jira] [Updated] (HIVE-20519) Remove 30m min value for hive.spark.session.timeout

2018-09-13 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/HIVE-20519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-20519: Attachment: HIVE-20519.2.patch > Remove 30m min value for hive.spark.session.timeout >

<    1   2   3   4   5   6   7   8   9   10   >