[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975959#comment-15975959
]
Rui Li commented on HIVE-15302:
---
Thanks [~xuefuz] for the suggestions. I'll investigate how
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973590#comment-15973590
]
Xuefu Zhang commented on HIVE-15302:
[~lirui] I think Marcelo's suggestion makes sense
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970629#comment-15970629
]
Rui Li commented on HIVE-15302:
---
Hi [~vanzin], the goal here is just to figure out what to s
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970231#comment-15970231
]
Marcelo Vanzin commented on HIVE-15302:
---
Livy doesn't figure out what spark.yarn.arc
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970228#comment-15970228
]
Rui Li commented on HIVE-15302:
---
Thanks [~vanzin] for the suggestions. I'm trying to figure
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969242#comment-15969242
]
Marcelo Vanzin commented on HIVE-15302:
---
I'm not sure which approach you're trying h
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969031#comment-15969031
]
Rui Li commented on HIVE-15302:
---
Following is the identified min set of required jars:
{nofo
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15713754#comment-15713754
]
Rui Li commented on HIVE-15302:
---
[~kellyzly], if we use the config to make HoS run against S
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15713661#comment-15713661
]
liyunzhang_intel commented on HIVE-15302:
-
[~lirui]:
{quote}
We only care about
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712522#comment-15712522
]
Marcelo Vanzin commented on HIVE-15302:
---
bq. For #2, I think we still need SPARK_HOM
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711818#comment-15711818
]
Rui Li commented on HIVE-15302:
---
We only care about yarn-client and yarn-cluster, because sp
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711352#comment-15711352
]
liyunzhang_intel commented on HIVE-15302:
-
[~lirui]: understand the requirement.
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711207#comment-15711207
]
Rui Li commented on HIVE-15302:
---
[~kellyzly], you're right about the ideas. But the needed s
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710922#comment-15710922
]
Xuefu Zhang commented on HIVE-15302:
I think there are two dependency on Spark from Hi
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710852#comment-15710852
]
liyunzhang_intel commented on HIVE-15302:
-
[~lirui]: The idea is good. so the fl
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710478#comment-15710478
]
Rui Li commented on HIVE-15302:
---
We don't only depend on Spark jars, but also the scripts li
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709208#comment-15709208
]
Marcelo Vanzin commented on HIVE-15302:
---
bq. I plan to find the needed jars from the
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707443#comment-15707443
]
Rui Li commented on HIVE-15302:
---
Thanks for your suggestions, Marcelo. I'll use spark.yarn.j
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707398#comment-15707398
]
Marcelo Vanzin commented on HIVE-15302:
---
You don't need to use the archive. You can
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707362#comment-15707362
]
Rui Li commented on HIVE-15302:
---
To clarify, the method here only works for yarn-cluster mod
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707353#comment-15707353
]
Rui Li commented on HIVE-15302:
---
Yeah my plan is to put the jars to HDFS. For example, if us
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707343#comment-15707343
]
Rui Li commented on HIVE-15302:
---
Hi [~vanzin], the potential conflicts introduced by transit
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15705967#comment-15705967
]
Marcelo Vanzin commented on HIVE-15302:
---
As far as I know, HoS would need spark-core
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15705364#comment-15705364
]
Xuefu Zhang commented on HIVE-15302:
This sounds good to me. If we know the minimum se
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704529#comment-15704529
]
Rui Li commented on HIVE-15302:
---
Basically HoS only needs the "core" functionalities of spar
[
https://issues.apache.org/jira/browse/HIVE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704508#comment-15704508
]
Rui Li commented on HIVE-15302:
---
With Spark 2.0, we can use {{spark.yarn.archive}} or {{spar
26 matches
Mail list logo
