+1 On Tue, Nov 15, 2016 at 9:23 AM, Dmitriy Lyubimov <[email protected]> wrote:
> +1 on version checking. > And, there's a little bug as well. this error is technically generated by > something like > > dense(Set.empty[Vector]), > > i.e., it cannot form a matrix out of an empty collection of vectors. While > this is true, i suppose it needs a `require(...)` insert there to generate > a more meaningful response instead of allowing Scala complaining about > empty collection. > > -d > > > On Mon, Nov 14, 2016 at 7:32 AM, Andrew Palumbo <[email protected]> > wrote: > > > +1 > > > > > > > > Sent from my Verizon Wireless 4G LTE smartphone > > > > > > -------- Original message -------- > > From: Trevor Grant <[email protected]> > > Date: 11/14/2016 6:49 AM (GMT-08:00) > > To: [email protected] > > Subject: [DISCUSS] More meaningful error when running on Spark 2.0 > > > > Hi, > > > > currently when running on Spark 2.0 the user will hit some sort of error, > > one such error is: > > > > java.util.NoSuchElementException: next on empty iterator > > at scala.collection.Iterator$$anon$2.next(Iterator.scala:39) > > at scala.collection.Iterator$$anon$2.next(Iterator.scala:37) > > at scala.collection.IndexedSeqLike$Elements.next( > IndexedSeqLike.scala:63) > > at scala.collection.IterableLike$class.head(IterableLike.scala:107) > > at scala.collection.mutable.ArrayOps$ofRef.scala$collection$Ind > > exedSeqOptimized$$super$head(ArrayOps.scala:186) > > at scala.collection.IndexedSeqOptimized$class.head(IndexedSeqOp > > timized.scala:126) > > at scala.collection.mutable.ArrayOps$ofRef.head(ArrayOps.scala:186) > > at org.apache.mahout.math.scalabindings.package$$anonfun$1.appl > > y(package.scala:155) > > at org.apache.mahout.math.scalabindings.package$$anonfun$1.appl > > y(package.scala:133) > > at scala.collection.TraversableLike$$anonfun$map$1.apply(Traver > > sableLike.scala:234) > > at scala.collection.TraversableLike$$anonfun$map$1.apply(Traver > > sableLike.scala:234) > > at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSe > > qOptimized.scala:33) > > at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35) > > at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) > > at scala.collection.AbstractTraversable.map(Traversable.scala:104) > > at org.apache.mahout.math.scalabindings.package$.dense( > package.scala:133) > > at org.apache.mahout.sparkbindings.SparkEngine$.drmSampleKRows( > > SparkEngine.scala:289) > > at org.apache.mahout.math.drm.package$.drmSampleKRows(package.scala:149) > > at org.apache.mahout.math.drm.package$.drmSampleToTSV(package.scala:165) > > ... 58 elided > > > > With the recent Zeppelin-Mahout integration, there are going to be a lot > of > > users unknowingly attempting to run on Mahout on Spark 2.0. I think it > > would be simple to implement yet save a lot of time on the Zeppelin and > > Mahout mailing lists to do something like: > > > > if sc.version > 1.6.2 then: > > error("Spark versions ${sc.verion} isn't supported. Please see > > MAHOUT-... (appropriate jira info)") > > > > I'd like to put something together and, depending on how many issues > people > > have on Zeppelin list, be prepared to do a hotfix on 0.12.2 if it becomes > > prudent. Everyone always complaining that Zeppelin doesn't work because > of > > some mystical error, is bad pr. It DOES say in the notebook and > elsewhere > > that we're not 2.0 compliant, however one of the advantages/drawbacks of > > Zeppelin is that without having to really know what you're doing you can > > get a functional local cluster of Flink, Spark, etc. all going. > > > > So we easily could have a space where someone read none of the docs, and > is > > whining. Surely few if any would ever do such a thing, but still I > think a > > prudent fix to have in the back pocket. > > > > tg > > > > Trevor Grant > > Data Scientist > > https://github.com/rawkintrevo > > http://stackexchange.com/users/3002022/rawkintrevo > > http://trevorgrant.org > > > > *"Fortunate is he, who is able to know the causes of things." -Virgil* > > >
