[jira] [Updated] (HIVE-10143) HS2 fails to clean up Spark client state on timeout

[ https://issues.apache.org/jira/browse/HIVE-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-10143: -- Attachment: HIVE-10143.1-spark.patch HS2 fails to clean up Spark client state on timeout

[jira] [Commented] (HIVE-12569) Excessive console message from SparkClientImpl [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038083#comment-15038083 ] Marcelo Vanzin commented on HIVE-12569: --- You can set {{spark.yarn.report.interval}} to a really

[jira] [Commented] (HIVE-12568) Use the same logic finding HS2 host name in Spark client [Spark Branch]

[ https://issues.apache.org/jira/browse/HIVE-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037047#comment-15037047 ] Marcelo Vanzin commented on HIVE-12568: --- So, with the old code, couldn't you just set

[jira] [Commented] (HIVE-12650) Increase default value of hive.spark.client.server.connect.timeout to exceeds spark.yarn.am.waitTime

[ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126881#comment-15126881 ] Marcelo Vanzin commented on HIVE-12650: --- bq. Do you know what factors (such as a lack of available

[jira] [Commented] (HIVE-12650) Increase default value of hive.spark.client.server.connect.timeout to exceeds spark.yarn.am.waitTime

[ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128913#comment-15128913 ] Marcelo Vanzin commented on HIVE-12650: --- bq. could you please explain a little bit the use of the

[jira] [Commented] (HIVE-12650) Increase default value of hive.spark.client.server.connect.timeout to exceeds spark.yarn.am.waitTime

[ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128979#comment-15128979 ] Marcelo Vanzin commented on HIVE-12650: --- * hive.spark.client.connect.timeout That's the socket

[jira] [Commented] (HIVE-12650) Increase default value of hive.spark.client.server.connect.timeout to exceeds spark.yarn.am.waitTime

[ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126779#comment-15126779 ] Marcelo Vanzin commented on HIVE-12650: --- {{spark.yarn.am.waitTime}} is not the time Spark waits for

[jira] [Commented] (HIVE-13308) HiveOnSpark sumbit query very slow when hundred of beeline exectue at same time

[ https://issues.apache.org/jira/browse/HIVE-13308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280475#comment-15280475 ] Marcelo Vanzin commented on HIVE-13308: --- There needs to be a synchronization block to access the

[jira] [Commented] (HIVE-13525) HoS hangs when job is empty

[ https://issues.apache.org/jira/browse/HIVE-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244500#comment-15244500 ] Marcelo Vanzin commented on HIVE-13525: --- Patch LGTM. > HoS hangs when job is empty >

[jira] [Commented] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861611#comment-15861611 ] Marcelo Vanzin commented on HIVE-15671: --- bq. I was talking about server detecting a driver problem

[jira] [Commented] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864066#comment-15864066 ] Marcelo Vanzin commented on HIVE-15671: --- bq. In the mean time, I think Patch #1 is still needed

[jira] [Commented] (HIVE-15859) Hive client side shows Spark Driver disconnected while Spark Driver side could not get RPC header

[ https://issues.apache.org/jira/browse/HIVE-15859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864063#comment-15864063 ] Marcelo Vanzin commented on HIVE-15859: --- {{RpcDispatcher::handleCall}} is on the read side, which is

[jira] [Commented] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831138#comment-15831138 ] Marcelo Vanzin commented on HIVE-15671: --- I mean the current code without the patch.

[jira] [Commented] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831115#comment-15831115 ] Marcelo Vanzin commented on HIVE-15671: --- Hmm, the options are poorly named (my fault, and they

[jira] [Comment Edited] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831115#comment-15831115 ] Marcelo Vanzin edited comment on HIVE-15671 at 1/20/17 3:30 AM: Hmm, the

[jira] [Commented] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832159#comment-15832159 ] Marcelo Vanzin commented on HIVE-15671: --- [~xuefuz] I see what you mean, but I think your analysis is

[jira] [Commented] (HIVE-15671) RPCServer.registerClient() erroneously uses server/client handshake timeout for connection timeout

[ https://issues.apache.org/jira/browse/HIVE-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832257#comment-15832257 ] Marcelo Vanzin commented on HIVE-15671: --- bq. I got a different problem when the driver suddenly dies

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709208#comment-15709208 ] Marcelo Vanzin commented on HIVE-15302: --- bq. I plan to find the needed jars from the Spark installed

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15712522#comment-15712522 ] Marcelo Vanzin commented on HIVE-15302: --- bq. For #2, I think we still need SPARK_HOME unless we

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707398#comment-15707398 ] Marcelo Vanzin commented on HIVE-15302: --- You don't need to use the archive. You can use

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705967#comment-15705967 ] Marcelo Vanzin commented on HIVE-15302: --- As far as I know, HoS would need spark-core, spark-yarn and

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15970231#comment-15970231 ] Marcelo Vanzin commented on HIVE-15302: --- Livy doesn't figure out what spark.yarn.archive or

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975696#comment-15975696 ] Marcelo Vanzin commented on HIVE-16484: --- {{SparkLauncher}} is just a wrapper around spark-submit,

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980148#comment-15980148 ] Marcelo Vanzin commented on HIVE-16484: --- It was mostly added for testing. > Investigate

[jira] [Commented] (HIVE-16071) Spark remote driver misuses the timeout in RPC handshake

[ https://issues.apache.org/jira/browse/HIVE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890770#comment-15890770 ] Marcelo Vanzin commented on HIVE-16071: --- re: the timeout used in 2 places, it's been a while since I

[jira] [Commented] (HIVE-15302) Relax the requirement that HoS needs Spark built w/o Hive

[ https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969242#comment-15969242 ] Marcelo Vanzin commented on HIVE-15302: --- I'm not sure which approach you're trying here, but I'd

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511747#comment-16511747 ] Marcelo Vanzin commented on HIVE-16391: --- It would be good to get comments from people on the Hive

[jira] [Updated] (HIVE-19888) Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState

[ https://issues.apache.org/jira/browse/HIVE-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated HIVE-19888: -- Attachment: HIVE-19888.1.patch Status: Patch Available (was: Open) Been a while since

[jira] [Commented] (HIVE-19888) Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState

[ https://issues.apache.org/jira/browse/HIVE-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512777#comment-16512777 ] Marcelo Vanzin commented on HIVE-19888: --- Tests haven't run, did I name the patch wrongly or

[jira] [Commented] (HIVE-19888) Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState

[ https://issues.apache.org/jira/browse/HIVE-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512937#comment-16512937 ] Marcelo Vanzin commented on HIVE-19888: --- (Pinging [~stakiar] for help.) > Misleading

[jira] [Commented] (HIVE-19888) Misleading "METASTORE_FILTER_HOOK will be ignored" warning from SessionState

[ https://issues.apache.org/jira/browse/HIVE-19888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517443#comment-16517443 ] Marcelo Vanzin commented on HIVE-19888: --- Is there anything I should look at here? I can't even find

[jira] [Commented] (HIVE-16391) Publish proper Hive 1.2 jars (without including all dependencies in uber jar)

[ https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502035#comment-16502035 ] Marcelo Vanzin commented on HIVE-16391: --- bq. I'm not sure if there's a way to publish two pom files

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316759#comment-16316759 ] Marcelo Vanzin commented on HIVE-16484: --- {{InProcessLauncher}} also uses the launcher server. So

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316894#comment-16316894 ] Marcelo Vanzin commented on HIVE-16484: --- Yes, that's what the launcher does internally, but also,

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317318#comment-16317318 ] Marcelo Vanzin commented on HIVE-16484: --- Just amending my previous response: if the concern with

[jira] [Commented] (HIVE-18436) Upgrade to Spark 2.3.0

[ https://issues.apache.org/jira/browse/HIVE-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338541#comment-16338541 ] Marcelo Vanzin commented on HIVE-18436: --- Hmm, I remember reviewing changes in that area and

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308515#comment-16308515 ] Marcelo Vanzin commented on HIVE-16484: --- That's correct. Using it only with deploy mode == cluster

[jira] [Commented] (HIVE-16484) Investigate SparkLauncher for HoS as alternative to bin/spark-submit

[ https://issues.apache.org/jira/browse/HIVE-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308460#comment-16308460 ] Marcelo Vanzin commented on HIVE-16484: --- {{InProcessLauncher}} works for client mode, but I wouldn't

[jira] [Commented] (HIVE-18533) Add option to use InProcessLauncher to submit spark jobs

[ https://issues.apache.org/jira/browse/HIVE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410347#comment-16410347 ] Marcelo Vanzin commented on HIVE-18533: --- That looks like a bug in Spark's {{LauncherBackend}}. It