[jira] [Commented] (SPARK-6112) Provide external block store support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596277#comment-14596277 ] Zhan Zhang commented on SPARK-6112: --- [~bghit] Here is one example link for the ramdisk c

[jira] [Commented] (SPARK-5111) HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5

[ https://issues.apache.org/jira/browse/SPARK-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596244#comment-14596244 ] Zhan Zhang commented on SPARK-5111: --- [~bolke] Thanks for the feedback. I will take a loo

[jira] [Commented] (SPARK-6112) Provide external block store support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14594935#comment-14594935 ] Zhan Zhang commented on SPARK-6112: --- Thansk [~arpitagarwal] for the detail setup. [~bghi

[jira] [Commented] (SPARK-8458) ORC data source can only write to the file system defined in Hadoop configuration

[ https://issues.apache.org/jira/browse/SPARK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592616#comment-14592616 ] Zhan Zhang commented on SPARK-8458: --- Thanks [~yhuai]. I didn't cover this test in my loc

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590680#comment-14590680 ] Zhan Zhang commented on SPARK-7009: --- [~airhorns] Please refer to https://issues.apache.o

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589108#comment-14589108 ] Zhan Zhang commented on SPARK-7009: --- The PR may be outdated, and not working against cur

[jira] [Updated] (SPARK-6112) Provide external block store support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Summary: Provide external block store support through HDFS RAM_DISK (was: Provide OffHeap support throu

[jira] [Commented] (SPARK-7449) createPhysicalRDD should use RDD output as schema instead of relation.schema

[ https://issues.apache.org/jira/browse/SPARK-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533294#comment-14533294 ] Zhan Zhang commented on SPARK-7449: --- In private[this] def createPhysicalRDD, it use ex

[jira] [Created] (SPARK-7449) createPhysicalRDD should use RDD output as schema instead of relation.schema

Zhan Zhang created SPARK-7449: - Summary: createPhysicalRDD should use RDD output as schema instead of relation.schema Key: SPARK-7449 URL: https://issues.apache.org/jira/browse/SPARK-7449 Project: Spark

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503209#comment-14503209 ] Zhan Zhang commented on SPARK-7009: --- How about another approach: Create another jar wit

[jira] [Commented] (SPARK-5111) HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5

[ https://issues.apache.org/jira/browse/SPARK-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493483#comment-14493483 ] Zhan Zhang commented on SPARK-5111: --- [~crystal_gaoyu] I am not sure. You may try to patc

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: spark-6479-tachyon.patch patch with Tachyon migration. Not complete patch, as it will add to

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486143#comment-14486143 ] Zhan Zhang commented on SPARK-6479: --- [~rxin] I updated the doc. If you think the overall

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: SPARK-6479OffheapAPIdesign (1).pdf Add exception from implementation > Create off-heap bloc

[jira] [Comment Edited] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393590#comment-14393590 ] Zhan Zhang edited comment on SPARK-6479 at 4/2/15 10:24 PM: [~

[jira] [Comment Edited] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393590#comment-14393590 ] Zhan Zhang edited comment on SPARK-6479 at 4/2/15 10:23 PM: [~

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393590#comment-14393590 ] Zhan Zhang commented on SPARK-6479: --- [~rxin] Thanks for the feedback. I updated the docu

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: SPARK-6479OffheapAPIdesign.pdf Add failure case handling overall design and example. > Crea

[jira] [Commented] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393287#comment-14393287 ] Zhan Zhang commented on SPARK-6112: --- Design spec for API attached to SPARK-6479 and wait

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: SPARK-6479.pdf This is the updated version for offheap store internal api design. > Create

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393141#comment-14393141 ] Zhan Zhang edited comment on SPARK-2883 at 4/2/15 7:54 PM: --- Foll

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393141#comment-14393141 ] Zhan Zhang edited comment on SPARK-2883 at 4/2/15 7:54 PM: --- Foll

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393146#comment-14393146 ] Zhan Zhang commented on SPARK-3720: --- [~iward] I have update the patch with new api suppo

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393141#comment-14393141 ] Zhan Zhang commented on SPARK-2883: --- Following code demonstrate the usage of the orc sup

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384872#comment-14384872 ] Zhan Zhang commented on SPARK-6479: --- I have a short version for this API and will post i

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376978#comment-14376978 ] Zhan Zhang commented on SPARK-6479: --- The current API may not be good enough as it has so

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: SparkOffheapsupportbyHDFS.pdf The design doc also includes stuff from SPARK-6112 > Create o

[jira] [Updated] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Attachment: SparkOffheapsupportbyHDFS.pdf Design doc for hdfs offheap support > Provide OffHeap support

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376361#comment-14376361 ] Zhan Zhang commented on SPARK-3720: --- [~iward] Since this jiar is duplicated to Spark-288

[jira] [Updated] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Summary: Provide OffHeap support through HDFS RAM_DISK (was: Leverage HDFS RAM_DISK capacity to provide

[jira] [Created] (AMBARI-9952) Populate keytab to all spark components

Zhan Zhang created AMBARI-9952: -- Summary: Populate keytab to all spark components Key: AMBARI-9952 URL: https://issues.apache.org/jira/browse/AMBARI-9952 Project: Ambari Issue Type: Bug

[jira] [Updated] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Component/s: (was: Spark Core) Block Manager > Leverage HDFS RAM_DISK capacity to p

[jira] [Commented] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343808#comment-14343808 ] Zhan Zhang commented on SPARK-6112: --- Will start scoping it. > Leverage HDFS RAM_DISK ca

[jira] [Updated] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Description: HDFS Lazy_Persist policy provide possibility to cache the RDD off_heap in hdfs. We may wan

[jira] [Updated] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Description: HDFS Lazy_Persist policy provide possibility to cache the RDD off_heap in hdfs. We may wan

[jira] [Updated] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Component/s: (was: YARN) > Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to >

[jira] [Updated] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Component/s: Spark Core > Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to > Tach

[jira] [Created] (SPARK-6112) Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon

Zhan Zhang created SPARK-6112: - Summary: Leverage HDFS RAM_DISK capacity to provide off_heap feature similar to Tachyon Key: SPARK-6112 URL: https://issues.apache.org/jira/browse/SPARK-6112 Project: Spar

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329885#comment-14329885 ] Zhan Zhang commented on SPARK-1537: --- [~vanzin] We should centralized all comments and r

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329828#comment-14329828 ] Zhan Zhang commented on SPARK-1537: --- [~sowen] In JIRA, we share the code so that other p

[jira] [Issue Comment Deleted] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-1537: -- Comment: was deleted (was: [~sowen] By the way, I am not waiting for someone to give me the patch. It i

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329704#comment-14329704 ] Zhan Zhang commented on SPARK-1537: --- [~sowen] By the way, I am not waiting for someone t

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329700#comment-14329700 ] Zhan Zhang commented on SPARK-1537: --- [~sowen] From the whole context, I believe you unde

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329678#comment-14329678 ] Zhan Zhang commented on SPARK-1537: --- [~vanzin] I declare "integrate your code" from the

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329664#comment-14329664 ] Zhan Zhang commented on SPARK-1537: --- [~vanzin] If you don't have bandwidth, or don't kno

[jira] [Comment Edited] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329649#comment-14329649 ] Zhan Zhang edited comment on SPARK-1537 at 2/20/15 10:14 PM: -

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329649#comment-14329649 ] Zhan Zhang commented on SPARK-1537: --- [~vanzin] Thanks for the comments. I don't unders

[jira] [Updated] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-1537: -- Attachment: SPARK-1537.txt High level design doc for spark ATS integration. > Add integration with Yarn

[jira] [Updated] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-1537: -- Attachment: spark-1573.patch Patch against v1.2.1 > Add integration with Yarn's Application Timeline Se

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326778#comment-14326778 ] Zhan Zhang commented on SPARK-1537: --- I have sent a PR with WIP for people who are intere

[jira] [Commented] (SPARK-5889) remove pid file in spark-daemon.sh after killing the process.

[ https://issues.apache.org/jira/browse/SPARK-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326443#comment-14326443 ] Zhan Zhang commented on SPARK-5889: --- https://github.com/apache/spark/pull/4676 > remove

[jira] [Created] (SPARK-5889) remove pid file in spark-daemon.sh after killing the process.

Zhan Zhang created SPARK-5889: - Summary: remove pid file in spark-daemon.sh after killing the process. Key: SPARK-5889 URL: https://issues.apache.org/jira/browse/SPARK-5889 Project: Spark Issue

[jira] [Commented] (AMBARI-9583) Add kerberos support for spark

[ https://issues.apache.org/jira/browse/AMBARI-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317191#comment-14317191 ] Zhan Zhang commented on AMBARI-9583: ReviewBoard: https://reviews.apache.org/r/30896/

[jira] [Updated] (AMBARI-9583) Add kerberos support for spark

[ https://issues.apache.org/jira/browse/AMBARI-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated AMBARI-9583: --- Attachment: (was: Ambari-9583.patch) > Add kerberos support for spark > -

[jira] [Updated] (AMBARI-9583) Add kerberos support for spark

[ https://issues.apache.org/jira/browse/AMBARI-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated AMBARI-9583: --- Attachment: 0001-spark-kerberos-support.patch Change the patch with right format > Add kerberos supp

[jira] [Updated] (AMBARI-9583) Add kerberos support for spark

[ https://issues.apache.org/jira/browse/AMBARI-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated AMBARI-9583: --- Attachment: Ambari-9583.patch patch for kerberos support > Add kerberos support for spark >

[jira] [Created] (AMBARI-9583) Add kerberos support for spark

Zhan Zhang created AMBARI-9583: -- Summary: Add kerberos support for spark Key: AMBARI-9583 URL: https://issues.apache.org/jira/browse/AMBARI-9583 Project: Ambari Issue Type: Bug Compone

[jira] [Closed] (SPARK-5110) Spark-on-Yarn does not work on windows platform

[ https://issues.apache.org/jira/browse/SPARK-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang closed SPARK-5110. - Resolution: Duplicate > Spark-on-Yarn does not work on windows platform >

[jira] [Commented] (SPARK-5110) Spark-on-Yarn does not work on windows platform

[ https://issues.apache.org/jira/browse/SPARK-5110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14266744#comment-14266744 ] Zhan Zhang commented on SPARK-5110: --- You are right. I will make this duplicated. > Spar

[jira] [Updated] (SPARK-5108) Need to make jackson dependency version consistent with hadoop-2.6.0.

[ https://issues.apache.org/jira/browse/SPARK-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-5108: -- Summary: Need to make jackson dependency version consistent with hadoop-2.6.0. (was: Need to add more j

[jira] [Created] (SPARK-5111) HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5

Zhan Zhang created SPARK-5111: - Summary: HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5 Key: SPARK-5111 URL: https://issues.apache.org/jira/browse/SPARK-5111 Project: Spark

[jira] [Created] (SPARK-5110) Spark-on-Yarn does not work on windows platform

Zhan Zhang created SPARK-5110: - Summary: Spark-on-Yarn does not work on windows platform Key: SPARK-5110 URL: https://issues.apache.org/jira/browse/SPARK-5110 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5108) Need to add more jackson dependency for hadoop-2.6.0 support.

[ https://issues.apache.org/jira/browse/SPARK-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14266530#comment-14266530 ] Zhan Zhang commented on SPARK-5108: --- java.lang.NoSuchMethodError: org.codehaus.jackson.

[jira] [Commented] (SPARK-5108) Need to add more jackson dependency for hadoop-2.6.0 support.

[ https://issues.apache.org/jira/browse/SPARK-5108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14266529#comment-14266529 ] Zhan Zhang commented on SPARK-5108: --- [~sowen] You are right. > Need to add more jacks

[jira] [Created] (SPARK-5108) Need to add more jackson dependency for hadoop-2.6.0 support.

Zhan Zhang created SPARK-5108: - Summary: Need to add more jackson dependency for hadoop-2.6.0 support. Key: SPARK-5108 URL: https://issues.apache.org/jira/browse/SPARK-5108 Project: Spark Issue

[jira] [Comment Edited] (SPARK-4461) Pass java options to yarn master to handle system properties correctly.

[ https://issues.apache.org/jira/browse/SPARK-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221502#comment-14221502 ] Zhan Zhang edited comment on SPARK-4461 at 11/21/14 10:23 PM: --

[jira] [Commented] (SPARK-4461) Pass java options to yarn master to handle system properties correctly.

[ https://issues.apache.org/jira/browse/SPARK-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221502#comment-14221502 ] Zhan Zhang commented on SPARK-4461: --- Thanks for the information Marcelo. I changed the t

[jira] [Commented] (SPARK-4461) Pass java options to yarn master to handle system properties correctly.

[ https://issues.apache.org/jira/browse/SPARK-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221474#comment-14221474 ] Zhan Zhang commented on SPARK-4461: --- I changed the title so that to reflect the issue fi

[jira] [Updated] (SPARK-4461) Pass java options to yarn master to handle system properties correctly.

[ https://issues.apache.org/jira/browse/SPARK-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-4461: -- Summary: Pass java options to yarn master to handle system properties correctly. (was: Spark should not

[jira] [Updated] (SPARK-4461) Spark should not relies on mapred-site.xml for classpath

[ https://issues.apache.org/jira/browse/SPARK-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-4461: -- Description: Currently spark read mapred-site.xml to get the class path. From hadoop 2.6, the library i

[jira] [Created] (SPARK-4461) Spark should not relies on mapred-site.xml for classpath

Zhan Zhang created SPARK-4461: - Summary: Spark should not relies on mapred-site.xml for classpath Key: SPARK-4461 URL: https://issues.apache.org/jira/browse/SPARK-4461 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195427#comment-14195427 ] Zhan Zhang commented on SPARK-2883: --- [~neoword] , As [~marmbrus] mentioned, the PR need

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195402#comment-14195402 ] Zhan Zhang commented on SPARK-3720: --- [~neoword] wangfei and me are working together to s

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191052#comment-14191052 ] Zhan Zhang commented on SPARK-1537: --- Yarn-2521 can make client easier to use, but not cr

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189651#comment-14189651 ] Zhan Zhang commented on SPARK-1537: --- Hi Marcelo, Do you have update on this? If you don

[jira] [Comment Edited] (SPARK-4103) Clean up SessionState in HiveContext

[ https://issues.apache.org/jira/browse/SPARK-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185889#comment-14185889 ] Zhan Zhang edited comment on SPARK-4103 at 10/27/14 11:10 PM: --

[jira] [Comment Edited] (SPARK-4103) Clean up SessionState in HiveContext

[ https://issues.apache.org/jira/browse/SPARK-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185889#comment-14185889 ] Zhan Zhang edited comment on SPARK-4103 at 10/27/14 10:56 PM: --

[jira] [Commented] (SPARK-4103) Clean up SessionState in HiveContext

[ https://issues.apache.org/jira/browse/SPARK-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185889#comment-14185889 ] Zhan Zhang commented on SPARK-4103: --- There are already some efforts (Spark-4037) target

[jira] [Created] (SPARK-4103) Clean up SessionState in HiveContext

Zhan Zhang created SPARK-4103: - Summary: Clean up SessionState in HiveContext Key: SPARK-4103 URL: https://issues.apache.org/jira/browse/SPARK-4103 Project: Spark Issue Type: Bug Compon

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183493#comment-14183493 ] Zhan Zhang commented on SPARK-2706: --- Michael, Please ignore my last email. I thought the

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183452#comment-14183452 ] Zhan Zhang commented on SPARK-2706: --- I just check the trunk, the change is already there

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174073#comment-14174073 ] Zhan Zhang commented on SPARK-2706: --- The code does not go to upstream yet. To build 0.13

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173269#comment-14173269 ] Zhan Zhang commented on SPARK-2883: --- It is scala patch, and I am not very familiar with

[jira] [Updated] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-2883: -- Attachment: orc.diff Just for completion, I also attach the diff here besides spark-3720. > Spark Supp

[jira] [Updated] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-3720: -- Attachment: orc.diff This is the diff for orc file support. Because I also work on this item in paralle

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 11:59 PM: --

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 11:04 PM: --

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 10:44 PM: --

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 10:44 PM: --

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171654#comment-14171654 ] Zhan Zhang commented on SPARK-2883: --- I almost finished the prototype, and following is t

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 10:41 PM: --

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164224#comment-14164224 ] Zhan Zhang edited comment on SPARK-3720 at 10/8/14 9:47 PM: By

[jira] [Comment Edited] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164222#comment-14164222 ] Zhan Zhang edited comment on SPARK-3720 at 10/8/14 9:45 PM: Th

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164224#comment-14164224 ] Zhan Zhang commented on SPARK-3720: --- By the way, I don't think the current approach can

[jira] [Commented] (SPARK-3720) support ORC in spark sql

[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164222#comment-14164222 ] Zhan Zhang commented on SPARK-3720: --- There is another gira spark-2883 opened on 06/Aug/1

[jira] [Commented] (SPARK-3652) upgrade spark sql hive version to 0.13.1

[ https://issues.apache.org/jira/browse/SPARK-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152201#comment-14152201 ] Zhan Zhang commented on SPARK-3652: --- Supporting different hive version is required. I th

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147939#comment-14147939 ] Zhan Zhang commented on SPARK-3633: --- Increasing timeout does not help my case either. I

[jira] [Commented] (SPARK-1537) Add integration with Yarn's Application Timeline Server

[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139396#comment-14139396 ] Zhan Zhang commented on SPARK-1537: --- Do you have any update on this, or any schedule in

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137759#comment-14137759 ] Zhan Zhang commented on SPARK-2883: --- I am starting to prototyping the last feature with

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136490#comment-14136490 ] Zhan Zhang commented on SPARK-2883: --- ORC format is self describing with schema of itself

<    1   2   3   4   5   >