Re: Different partition number of GroupByKey leads different result

2015-10-09 Thread Sean Owen
gt;>> num is the number of partition and uid is a filter id for result >>>> comparision. >>>> TagsWritable implements WritableComparable and Serializable. >>>> >>>> I used GroupByKey on text file, the result was right. >>>> >>>

Re: Different partition number of GroupByKey leads different result

2015-10-09 Thread Devin Huang
ritable implements WritableComparable and Serializable. >>> >>> I used GroupByKey on text file, the result was right. >>> >>> Thanks, >>> Devin Huang >>> >>> >>> >>> >>> -- >>> View this messag

Re: Different partition number of GroupByKey leads different result

2015-10-09 Thread Devin Huang
may be mismatched on the shuffle stage. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Different-partition-number-of-GroupByKey-leads-different-result-tp24989p24990.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: Different partition number of GroupByKey leads different result

2015-10-09 Thread Sean Owen
on and uid is a filter id for result >> comparision. >> TagsWritable implements WritableComparable and Serializable. >> >> I used GroupByKey on text file, the result was right. >> >> Thanks, >> Devin Huang >> >> >> >> >> -- >

Re: Different partition number of GroupByKey leads different result

2015-10-09 Thread Sean Owen
e and Serializable. > > I used GroupByKey on text file, the result was right. > > Thanks, > Devin Huang > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Different-partition-number-of-GroupByKey-lea

Different partition number of GroupByKey leads different result

2015-10-09 Thread Devin Huang
on and uid is a filter id for result comparision. TagsWritable implements WritableComparable and Serializable. I used GroupByKey on text file, the result was right. Thanks, Devin Huang -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Different-partit