Not sure how that would work. Really I want to tack on an extra column onto the DF with a UDF that can take a Row object.
On Tue, Sep 8, 2015 at 1:54 AM, Jörn Franke <jornfra...@gmail.com> wrote: > Can you use a map or list with different properties as one parameter? > Alternatively a string where parameters are Comma-separated... > > Le lun. 7 sept. 2015 à 8:35, Night Wolf <nightwolf...@gmail.com> a écrit : > >> Is it possible to have a UDF which takes a variable number of arguments? >> >> e.g. df.select(myUdf($"*")) fails with >> >> org.apache.spark.sql.AnalysisException: unresolved operator 'Project >> [scalaUDF(*) AS scalaUDF(*)#26]; >> >> What I would like to do is pass in a generic data frame which can be then >> passed to a UDF which does scoring of a model. The UDF needs to know the >> schema to map column names in the model to columns in the DataFrame. >> >> The model has 100s of factors (very wide), so I can't just have a scoring >> UDF that has 500 parameters (for obvious reasons). >> >> Cheers, >> ~N >> >
