Hey Kaustubh,
Could it be that the input file is not readable?
=> String inputFileUrl = (String) hyperparams.get("inputFileUrl”);
Best,
—alex
> On Apr 11, 2025, at 14:41, Kaustubh Beedkar <kbeed...@gmail.com> wrote:
>
> We were trying to run a SGD example based on the ml4all example.
>
> PlanFunction planFunction = (operand, pb, hyperparams) -> {
> // Step 0: Cast operand and extract hyperparams
> List<Double> weights = (List<Double>) operand;
> String inputFileUrl = (String) hyperparams.get("inputFileUrl");
> int datasetSize = (int) hyperparams.get("datasetSize");
>
> // Step 1: Define ML operators
> Sample sampleOp = new SGDSample();
> Transform transformOp = new LibSVMTransform(29);
> Compute computeOp = new ComputeLogisticGradient();
>
> // Step 2: Create weight DataQuanta
> var weightsBuilder = pb
> .loadCollection(weights)
> .withName("weights");
>
> // Step 3: Load dataset and apply transform
> DataQuantaBuilder transformBuilder = (DataQuantaBuilder) pb
> .readTextFile(inputFileUrl)
> .withName("source")
> .mapPartitions(new
> TransformPerPartitionWrapper(transformOp))
> .withName("transform");
>
> Collection<?> parsedData = transformBuilder.collect();
> for (Object row : parsedData) {
> System.out.println(row);
> }
>
> // Step 4: Sample, compute gradient, and broadcast weights
> DataQuantaBuilder result = (DataQuantaBuilder) transformBuilder
> .sample(sampleOp.sampleSize())
> .withSampleMethod(sampleOp.sampleMethod())
> .withDatasetSize(datasetSize)
> .map(new ComputeWrapper<>(computeOp))
> .withBroadcast(weightsBuilder, "weights");
>
> // Step 5: Return final operator
> return result.dataQuanta().operator();
> };
> this is the plan that we've written but for some reason it's not working,
> i.e when we execute it we get the following error
>
> "executing T[JavaTextFileSource[source]] failed."
>
> Do you know why this may be so?
>
> Thanks a lot.
--
*Scalytics Connect*
The foundation for secure, scalable, and transparent
AI.
--
3401 N. MIAMI AVE. STE 230
33127 Miami, Florida
United States
www.scalytics.io <http://www.scalytics.io>
-- Please consider the
environment before printing this email --
Disclaimer:
The content of this
message is confidential. If you have received it by mistake, please inform
us by an email reply and then delete the message. It is forbidden to copy,
forward, or in any way reveal the contents of this message to anyone. The
integrity and security of this email cannot be guaranteed over the
Internet. Therefore, the sender will not be held liable for any damage
caused by the message.
