Re: Survival Curves using AFT implementation in Spark

[Yanbo Liang]

Hi Stuti,

AFTSurvivalRegression does not support computing the predicted survival
functions/curves currently.
I don't know whether the quantile predictions can help you, you can refer
the example

.
Maybe we can add this feature later.

Thanks
Yanbo

2016-02-26 14:35 GMT+08:00 Stuti Awasthi :

> Hi All,
>
> I wanted to apply Survival Analysis using Spark AFT algorithm
> implementation. Now I perform the same in R using coxph model and passing
> the model in Survfit() function to generate survival curves
>
> Then I can visualize the survival curve on validation data to understand
> how good my model fits.
>
>
>
> R: Code
>
> fit <- coxph(Surv(futime, fustat) ~ age, data = ovarian)
>
> plot(survfit(fit,newdata=data.frame(age=60)))
>
>
>
> I wanted to achieve something similar with Spark. Hence I created the AFT
> model using Spark and passed my Test dataframe for prediction. The result
> of prediction is single prediction value for single input data which is as
> expected. But now how can I use this model to generate the Survival curves
> for visualization.
>
>
>
> Eg: Spark Code model.transform(test_final).show()
>
>
>
> standardized_features|   prediction|
>
> +-+-+
>
> | [0.0,0.0,0.743853...|48.33071792204102|
>
> +-+-+
>
>
>
> Can any suggest how to use the developed model for plotting Survival
> Curves for “test_final” data which is a dataframe feature[vector].
>
>
>
> Thanks
>
> Stuti Awasthi
>
>
>
>
>
> ::DISCLAIMER::
>
> 
>
> The contents of this e-mail and any attachment(s) are confidential and
> intended for the named recipient(s) only.
> E-mail transmission is not guaranteed to be secure or error-free as
> information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or may contain viruses in
> transmission. The e mail and its contents
> (with or without referred errors) shall therefore not attach any liability
> on the originator or HCL or its affiliates.
> Views or opinions, if any, presented in this email are solely those of the
> author and may not necessarily reflect the
> views or opinions of HCL or its affiliates. Any form of reproduction,
> dissemination, copying, disclosure, modification,
> distribution and / or publication of this message without the prior
> written consent of authorized representative of
> HCL is strictly prohibited. If you have received this email in error
> please delete it and notify the sender immediately.
> Before opening any email and/or attachments, please check them for viruses
> and other defects.
>
>
> 
>

Survival Curves using AFT implementation in Spark

[Stuti Awasthi]

Hi All,
I wanted to apply Survival Analysis using Spark AFT algorithm implementation. 
Now I perform the same in R using coxph model and passing the model in 
Survfit() function to generate survival curves
Then I can visualize the survival curve on validation data to understand how 
good my model fits.



R: Code

fit <- coxph(Surv(futime, fustat) ~ age, data = ovarian)

plot(survfit(fit,newdata=data.frame(age=60)))


I wanted to achieve something similar with Spark. Hence I created the AFT model 
using Spark and passed my Test dataframe for prediction. The result of 
prediction is single prediction value for single input data which is as 
expected. But now how can I use this model to generate the Survival curves for 
visualization.

Eg: Spark Code model.transform(test_final).show()

standardized_features|   prediction|
+-+-+
| [0.0,0.0,0.743853...|48.33071792204102|
+-+-+

Can any suggest how to use the developed model for plotting Survival Curves for 
"test_final" data which is a dataframe feature[vector].

Thanks
Stuti Awasthi



::DISCLAIMER::


The contents of this e-mail and any attachment(s) are confidential and intended 
for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information 
could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in 
transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on 
the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the 
author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, 
dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written 
consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please 
delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and 
other defects.