Re: Spark SQL 1.0.1 error on reading fixed length byte array

Hi, We have a PR to support fixed length byte array in parquet file. https://github.com/apache/spark/pull/1737 Can someone help verifying it? Thanks. 2014-07-15 19:23 GMT+08:00 Pei-Lun Lee : > Sorry, should be SPARK-2489 > > > 2014-07-15 19:22 GMT+08:00 Pei-Lun Lee : > > Filed SPARK-2446 >> >

Re: -1s on pull requests?

Sure thing - feel free to ping me off list if you need pointers. The script just does string concatenation and a curl to post the comment... I think it should be pretty accessible! - Patrick On Sun, Aug 3, 2014 at 9:12 PM, Nicholas Chammas wrote: > On Sun, Aug 3, 2014 at 11:29 PM, Patrick Wend

Re: -1s on pull requests?

On Sun, Aug 3, 2014 at 11:29 PM, Patrick Wendell wrote: Nick - Any interest in doing these? this is all doable from within the > spark repo itself because our QA harness scripts are in there: > > https://github.com/apache/spark/blob/master/dev/run-tests-jenkins > > If not, could you make a JIRA f

Re: Scala 2.11 external dependencies

Hey Anand, Thanks for looking into this - it's great to see momentum towards Scala 2.11 and I'd love if this land in Spark 1.2. For the external dependencies, it would be good to create a sub-task of SPARK-1812 to track our efforts encouraging other projects to upgrade. In certain cases (e.g. Kaf

Re: Low Level Kafka Consumer for Spark

I'll let TD chime on on this one, but I'm guessing this would be a welcome addition. It's great to see community effort on adding new streams/receivers, adding a Java API for receivers was something we did specifically to allow this :) - Patrick On Sat, Aug 2, 2014 at 10:09 AM, Dibyendu Bhattach

Compiling Spark master (6ba6c3eb) with sbt/sbt assembly

On the latest pull today (6ba6c3ebfe9a47351a50e45271e241140b09bf10) meet assembly problem. $ ./sbt/sbt assembly Using /usr/lib/jvm/java-7-oracle as default JAVA_HOME. Note, this will be overridden by -java-home if it is set. [info] Loading project definition from ~/spark/project/project [info] L

Re: -1s on pull requests?

> >1. Include the commit hash in the "tests have started/completed" >messages, so that it's clear what code exactly is/has been tested for > each >test cycle. > Great idea - I think this is easy to do given the current architecture. We already have access to the commit ID in the same s

spark

-- *(주)사이버다임 프로세스마이닝 팀서울 강남구 논현동 278-3 프라임저축은행빌딩 6층조 인 수 대리* 회사전화 : 02-546-6990 휴대전화 : 010-4310-0826 전자우편 : auso...@cyberdigm.co.kr This com

(send this email to subscribe)

Re: I would like to contribute

The Contributing to Spark guide on the Spark Wiki provides a good overview on how to start contributing: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark On August 3, 2014 at 5:14:23 PM, pritish (prit...@nirvana-international.com) wrote: Hi We would like to contribute

I would like to contribute

Hi We would like to contribute to Spark but we are not sure how. We can offer project management, release management to begin with. Please advice on how to get engaged. Thank you!! Regards Pritish Nirvana International Inc. Big Data, Hadoop, Oracle EBS and IT Solutions VA - SWaM, MD - MBE Certif

Re: -1s on pull requests?

On Mon, Jul 21, 2014 at 4:44 PM, Kay Ousterhout wrote: > This also happens when something accidentally gets merged after the tests > have started but before tests have passed. > Some improvements to SparkQA could help with this. May I suggest: 1. Include the comm

Re:Re: Intellij IDEA can not recognize the MLlib package

Got it and add spark-mllib libraryDependency:libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.0.0" It works and many thanks! BR Kitaev At 2014-08-03 05:09:23, "Sean Owen" wrote: >Yes, but it is nowhere in your project dependencies. > >On Sun, Aug 3, 2014 at 10:06 AM, jun wrote:

Re: Intellij IDEA can not recognize the MLlib package

You missed the mllib artifact? that would certainly explain it! all I see is core. On Sun, Aug 3, 2014 at 10:03 AM, jun wrote: > Hi, > > > I have started my spark exploration in intellij IDEA local model and want to > focus on MLlib part. > but when I put some example codes in IDEA, It can not r

Re:Intellij IDEA can not recognize the MLlib package

Sorry the color is missing. the "mllib" is red word and "import" sentence is grey.>>import org.apache.spark.mllib.recommendation.ALS At 2014-08-03 05:03:31, jun" wrote: >Hi, > > >I have started my spark exploration in intellij IDEA local model and want to focus on MLlib part. >but when I put s

Intellij IDEA can not recognize the MLlib package

Hi, I have started my spark exploration in intellij IDEA local model and want to focus on MLlib part. but when I put some example codes in IDEA, It can not recognize mllib package, just loos like that: > > import org.apache.spark.SparkContext >import org.apache.spark.mllib.recommendation.ALS