Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2356#discussion_r18063664
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala ---
@@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {
}
/**
+ * Java stub for Python mllib Word2Vec fit(). This stub returns a
+ * handle to the Java object instead of the content of the Java object.
+ * Extra care needs to be taken in the Python code to ensure it gets
freed on
+ * exit; see the Py4J documentation.
+ * @param dataJRDD Input JavaRDD
+ * @return A handle to java Word2VecModel instance at python side
+ */
+ def trainWord2Vec(
+ dataJRDD: JavaRDD[java.util.ArrayList[String]]
+ ): Word2VecModel = {
+ val data = dataJRDD.rdd.map(_.toArray(new
Array[String](0)).toSeq).cache()
+ val word2vec = new Word2Vec()
+ val model = word2vec.fit(data)
+ model
+ }
+
+ /**
+ * Java stub for Python mllib Word2VecModel transform
+ * @param model Word2VecModel instance
+ * @param word a word
+ * @return serialized vector representation of word
+ */
+ def Word2VecModelTransform(
+ model: Word2VecModel,
+ word: String
+ ): Vector = {
+ model.transform(word)
+ }
+
+ /**
+ * Java stub for Python mllib Word2VecModel findSynonyms
+ * @param model Word2VecModel instance
+ * @param word a word
+ * @param num number of synonyms to find
+ * @return a java LinkedList containing serialized version of
+ * synonyms and similarities
+ */
+ def Word2VecModelSynonyms(
--- End diff --
Shall we define `Word2VecModelWrapper` that implements `transform` and
`findSynonyms`? Let `trainWord2Vec` return the wrapper and you can call
`.transform` and `.synonyms` on the python side.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
