Hi SystemML folks,
Are there any recommended Spark configurations when running SystemML on a
single machine? I.e. is there a difference between launching Spark with
master=local[*] and running SystemML as a standard process in the JVM as
opposed to launching a single node spark cluster? If the
further
information.
Best,
Anthony Thomas
Hi SystemML folks,
I'm trying to pass some data from Spark to a DML script via the MLContext
API. The data is derived from a parquet file containing a dataframe with
the schema: [label: Integer, features: SparseVector]. I am doing the
following:
val input_data =
egative number of columns or block sizes), which prevents
> the sparse rows from growing to the necessary sizes.
>
> Regards,
> Matthias
>
> On 12/22/2017 10:42 PM, Anthony Thomas wrote:
>
>> Hi Matthias,
>>
>> Thanks for the help! In response to your questions
lease (a) also provide the
>> stacktrace of calling dataFrameToBinaryBlock with SystemML 1.0, and (b)
>> try calling your IJV conversion script via spark submit to exclude that
>> this issue is API-related? Thanks.
>>
>> Regards,
>> Matthias
>>
>> On
, df, mc, containsID,
> isVector),
> where jsc is the java spark context, df is the dataset, mc are matrix
> characteristics (if unknown, simply use new MatrixCharacteristics()),
> containsID indicates if the dataset contains a column "__INDEX" with the
> row indexes, and isVe
+1
I ran the Python test suite on Red Hat Linux under Spark 2.2.0 (Python
2.7.5) and encountered no errors.
Regards,
Anthony
On Tue, Aug 21, 2018 at 7:49 AM Guobao Li wrote:
> +1
>
> As an initiator and user of paramserv func, I just launched several tests
> on local pc with a script using
