Re: [R-sig-Geo] Parameter tuning of the bfastlite function
Please follow up on https://github.com/bfast2/bfast/issues/113. Roger --- Roger Bivand Emeritus Professor Department of Economics Norwegian School of Economics, Bergen, Norway Fra: R-sig-Geo p� vegne av Roger Bivand Sendt: fredag, september 27, 2024 10:33:25 a.m. Til: Hugo Costa Kopi: r-sig-geo Emne: Re: [R-sig-Geo] Parameter tuning of the bfastlite function The last update was 2021-05-10; the package is available on CRAN, so no worries, but also only one small correction to documentation since then. On that basis, I'd reach out to the maintainer directly. Roger -- Roger Bivand Emeritus Professor Norwegian School of Economics Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway [email protected] From: Hugo Costa Sent: 27 September 2024 10:00 To: Roger Bivand Cc: Nikolaos Tziokas; r-sig-geo Subject: Re: [R-sig-Geo] Parameter tuning of the bfastlite function You don't often get email from [email protected]. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> It's a pity if BFAST is not developed and maintained... Hugo Roger Bivand mailto:[email protected]>> escreveu (sexta, 27/09/2024 �(s) 08:47): Could I suggest waiting until https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbfast2%2Fbfast%2Fissues%2F113&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055257289%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ABYk2bkfc06Mg2%2FjDRQ%2FVyF2Ip1t25i9Pk05kMD0nWg%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbfast2%2Fbfast%2Fissues%2F113&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055277013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=70FJ6V8Tbwys%2FGgzXeEqcLGvZ4AAYOFp05l3noQobts%3D&reserved=0><https://github.com/bfast2/bfast/issues/113> which contains the same content has been attended to? Alternatively link to the list archives in the github issue to ensure that any responses to one channel do not get overlooked on the other channel (I've linked for now)? Emailing directly or github pinging the package developers may also help if this is urgent. Also consider reaching out to authors/maintainers of packages using bfast https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fpackage%3Dbfast&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055287053%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=AimFwdWsVSLJPtJam5bP8pTX85fv1IdgGgeX6pSnb%2Bs%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fpackage%3Dbfast&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055298284%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=P%2B%2BSDER6GodY%2BUwdIQnheP6q%2B8iHi75lccuUMDRy6tY%3D&reserved=0><https://cran.r-project.org/package=bfast>, and look (I'm sure you already have looked) among the many articles citing the underlying work. The package description mentions a forthcoming paper on BFAST Lite - this seems to be https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.3390%2Frs13163308&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055308702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=v2NugDxN7r7kI6RWHCkCQy%2BnuNAYFv3EqvNc71wiM3M%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.3390%2Frs13163308&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055318828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=sfeir%2BQ4PTWJ1h2JS%2F8uns5V9pNAFY4iPC2hIdtm8sc%3D&reserved=0><https://doi.org/10.3390/rs13163308>. This isn't my field, but I think that bfast isn't being developed actively, and you may need to search broadly to try to resolve your issues, so looking at how other users handle this may help. Hope this helps, Roger -- Roger Bivand Emeritus Professor Norwegian School of Economics Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway [email protected]<mailto:[email protected]> ___
Re: [R-sig-Geo] Parameter tuning of the bfastlite function
The last update was 2021-05-10; the package is available on CRAN, so no worries, but also only one small correction to documentation since then. On that basis, I'd reach out to the maintainer directly. Roger -- Roger Bivand Emeritus Professor Norwegian School of Economics Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway [email protected] From: Hugo Costa Sent: 27 September 2024 10:00 To: Roger Bivand Cc: Nikolaos Tziokas; r-sig-geo Subject: Re: [R-sig-Geo] Parameter tuning of the bfastlite function You don't often get email from [email protected]. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> It's a pity if BFAST is not developed and maintained... Hugo Roger Bivand mailto:[email protected]>> escreveu (sexta, 27/09/2024 à(s) 08:47): Could I suggest waiting until https://github.com/bfast2/bfast/issues/113<https://github.com/bfast2/bfast/issues/113> which contains the same content has been attended to? Alternatively link to the list archives in the github issue to ensure that any responses to one channel do not get overlooked on the other channel (I've linked for now)? Emailing directly or github pinging the package developers may also help if this is urgent. Also consider reaching out to authors/maintainers of packages using bfast https://cran.r-project.org/package=bfast<https://cran.r-project.org/package=bfast>, and look (I'm sure you already have looked) among the many articles citing the underlying work. The package description mentions a forthcoming paper on BFAST Lite - this seems to be https://doi.org/10.3390/rs13163308<https://doi.org/10.3390/rs13163308>. This isn't my field, but I think that bfast isn't being developed actively, and you may need to search broadly to try to resolve your issues, so looking at how other users handle this may help. Hope this helps, Roger -- Roger Bivand Emeritus Professor Norwegian School of Economics Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway [email protected]<mailto:[email protected]> From: R-sig-Geo mailto:[email protected]>> on behalf of Nikolaos Tziokas mailto:[email protected]>> Sent: 27 September 2024 02:19 To: r-sig-geo Subject: [R-sig-Geo] Parameter tuning of the bfastlite function [You don't often get email from [email protected]<mailto:[email protected]>. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] I am using the bfastlite() function from the BFAST package to run a time-series analysis. From the author's paper (BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis) (table 2), I quote: "Needs parameter tuning to optimise performance, does not differentiate between breaks in seasonality and trend" So far, I was fine-tuning the model manually, that is, I was changing the parameters one by one, which is time-consuming. Does someone have a better solution regarding the fine-tuning of the model? To see which parameters of the model achieve the best results, I was checking the dates in the detected breakpoints (visual inspection). I am not sure if that method (visual inspection) is appropriate. I apologize if this question sound a bit vague, so let me expand a little bit. After running the bfastlite() using the default parameters (i.e., bp = bfastlite(datats)), we get a result. Is there a way to measure (something like rmse, or r-squared) how well the algorithm modeled the ts? What I basically mean is that if there is an index equivalent to let's say rmse when someone is running a linear regression. For example, what if the parameter breaks with BIC instead of LWZ detects more accurate the breakpoints (by visually inspecting the detected breakpoints)? Apart from the visual inspection, shouldn't be some other way to measure the performance of the model? Based on the above, is there a more efficient way to optimize the parameters of the model (based on some metric)? What do I mean by optimizing the parameters? I think with an example I can explain it better. When someone is tuning a random forest model, he/she can perform a full grid search to find the optimal parameters of the model (mtry, number of trees, etc) by searching all the possible combinations and for each combination he/she checks the rmse (or mse, r-squared). Is this what the authors of the paper meant when they said "Needs parameter tuning to optimise performance"? And if so, how did they do it? library(bfast) plot(simts) # stl object containing simulated NDVI time series datats <- ts(rowSums(simts$time.series)) # sum of all the components (season,abrupt,remainder) tsp(datats) <- tsp(simts$time.series) # assign correct time series attributes plot(datats) # Detect breaks. default parameters bp = bfastlite(datats)
Re: [R-sig-Geo] Parameter tuning of the bfastlite function
It's a pity if BFAST is not developed and maintained... Hugo Roger Bivand escreveu (sexta, 27/09/2024 à(s) 08:47): > Could I suggest waiting until https://github.com/bfast2/bfast/issues/113 > which contains the same content has been attended to? Alternatively link to > the list archives in the github issue to ensure that any responses to one > channel do not get overlooked on the other channel (I've linked for now)? > Emailing directly or github pinging the package developers may also help if > this is urgent. Also consider reaching out to authors/maintainers of > packages using bfast https://cran.r-project.org/package=bfast, and look > (I'm sure you already have looked) among the many articles citing the > underlying work. The package description mentions a forthcoming paper on > BFAST Lite - this seems to be https://doi.org/10.3390/rs13163308. This > isn't my field, but I think that bfast isn't being developed actively, and > you may need to search broadly to try to resolve your issues, so looking at > how other users handle this may help. > > Hope this helps, > > Roger > > -- > Roger Bivand > Emeritus Professor > Norwegian School of Economics > Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway > [email protected] > > > From: R-sig-Geo on behalf of Nikolaos > Tziokas > Sent: 27 September 2024 02:19 > To: r-sig-geo > Subject: [R-sig-Geo] Parameter tuning of the bfastlite function > > [You don't often get email from [email protected]. Learn why this > is important at https://aka.ms/LearnAboutSenderIdentification ] > > I am using the bfastlite() function from the BFAST package to run a > time-series analysis. From the author's paper (BFAST Lite: A Lightweight > Break Detection Method for Time Series Analysis) (table 2), I quote: > > "Needs parameter tuning to optimise performance, does not differentiate > between breaks in seasonality and trend" > > So far, I was fine-tuning the model manually, that is, I was changing the > parameters one by one, which is time-consuming. Does someone have a better > solution regarding the fine-tuning of the model? > > To see which parameters of the model achieve the best results, I was > checking the dates in the detected breakpoints (visual inspection). I am > not sure if that method (visual inspection) is appropriate. > > I apologize if this question sound a bit vague, so let me expand a little > bit. After running the bfastlite() using the default parameters (i.e., bp = > bfastlite(datats)), we get a result. Is there a way to measure (something > like rmse, or r-squared) how well the algorithm modeled the ts? What I > basically mean is that if there is an index equivalent to let's say rmse > when someone is running a linear regression. For example, what if the > parameter breaks with BIC instead of LWZ detects more accurate the > breakpoints (by visually inspecting the detected breakpoints)? Apart from > the visual inspection, shouldn't be some other way to measure the > performance of the model? > > Based on the above, is there a more efficient way to optimize the > parameters of the model (based on some metric)? What do I mean by > optimizing the parameters? I think with an example I can explain it better. > When someone is tuning a random forest model, he/she can perform a full > grid search to find the optimal parameters of the model (mtry, number of > trees, etc) by searching all the possible combinations and for each > combination he/she checks the rmse (or mse, r-squared). Is this what the > authors of the paper meant when they said "Needs parameter tuning to > optimise performance"? And if so, how did they do it? > > library(bfast) > > plot(simts) # stl object containing simulated NDVI time series > datats <- ts(rowSums(simts$time.series)) > > # sum of all the components (season,abrupt,remainder) > tsp(datats) <- tsp(simts$time.series) # assign correct time series > attributes > plot(datats) > > # Detect breaks. default parameters > bp = bfastlite(datats) > plot(bp) > > # optimized model ?? > bp_opt <- bfastlite() > > R 4.4.1, bfast 1.6.1, Windows 11. > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > ___ > R-sig-Geo mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > [[alternative HTML version deleted]] ___ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Parameter tuning of the bfastlite function
Could I suggest waiting until https://github.com/bfast2/bfast/issues/113 which contains the same content has been attended to? Alternatively link to the list archives in the github issue to ensure that any responses to one channel do not get overlooked on the other channel (I've linked for now)? Emailing directly or github pinging the package developers may also help if this is urgent. Also consider reaching out to authors/maintainers of packages using bfast https://cran.r-project.org/package=bfast, and look (I'm sure you already have looked) among the many articles citing the underlying work. The package description mentions a forthcoming paper on BFAST Lite - this seems to be https://doi.org/10.3390/rs13163308. This isn't my field, but I think that bfast isn't being developed actively, and you may need to search broadly to try to resolve your issues, so looking at how other users handle this may help. Hope this helps, Roger -- Roger Bivand Emeritus Professor Norwegian School of Economics Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway [email protected] From: R-sig-Geo on behalf of Nikolaos Tziokas Sent: 27 September 2024 02:19 To: r-sig-geo Subject: [R-sig-Geo] Parameter tuning of the bfastlite function [You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] I am using the bfastlite() function from the BFAST package to run a time-series analysis. From the author's paper (BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis) (table 2), I quote: "Needs parameter tuning to optimise performance, does not differentiate between breaks in seasonality and trend" So far, I was fine-tuning the model manually, that is, I was changing the parameters one by one, which is time-consuming. Does someone have a better solution regarding the fine-tuning of the model? To see which parameters of the model achieve the best results, I was checking the dates in the detected breakpoints (visual inspection). I am not sure if that method (visual inspection) is appropriate. I apologize if this question sound a bit vague, so let me expand a little bit. After running the bfastlite() using the default parameters (i.e., bp = bfastlite(datats)), we get a result. Is there a way to measure (something like rmse, or r-squared) how well the algorithm modeled the ts? What I basically mean is that if there is an index equivalent to let's say rmse when someone is running a linear regression. For example, what if the parameter breaks with BIC instead of LWZ detects more accurate the breakpoints (by visually inspecting the detected breakpoints)? Apart from the visual inspection, shouldn't be some other way to measure the performance of the model? Based on the above, is there a more efficient way to optimize the parameters of the model (based on some metric)? What do I mean by optimizing the parameters? I think with an example I can explain it better. When someone is tuning a random forest model, he/she can perform a full grid search to find the optimal parameters of the model (mtry, number of trees, etc) by searching all the possible combinations and for each combination he/she checks the rmse (or mse, r-squared). Is this what the authors of the paper meant when they said "Needs parameter tuning to optimise performance"? And if so, how did they do it? library(bfast) plot(simts) # stl object containing simulated NDVI time series datats <- ts(rowSums(simts$time.series)) # sum of all the components (season,abrupt,remainder) tsp(datats) <- tsp(simts$time.series) # assign correct time series attributes plot(datats) # Detect breaks. default parameters bp = bfastlite(datats) plot(bp) # optimized model ?? bp_opt <- bfastlite() R 4.4.1, bfast 1.6.1, Windows 11. [[alternative HTML version deleted]] ___ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo ___ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
[R-sig-Geo] Parameter tuning of the bfastlite function
I am using the bfastlite() function from the BFAST package to run a time-series analysis. From the author's paper (BFAST Lite: A Lightweight Break Detection Method for Time Series Analysis) (table 2), I quote: "Needs parameter tuning to optimise performance, does not differentiate between breaks in seasonality and trend" So far, I was fine-tuning the model manually, that is, I was changing the parameters one by one, which is time-consuming. Does someone have a better solution regarding the fine-tuning of the model? To see which parameters of the model achieve the best results, I was checking the dates in the detected breakpoints (visual inspection). I am not sure if that method (visual inspection) is appropriate. I apologize if this question sound a bit vague, so let me expand a little bit. After running the bfastlite() using the default parameters (i.e., bp = bfastlite(datats)), we get a result. Is there a way to measure (something like rmse, or r-squared) how well the algorithm modeled the ts? What I basically mean is that if there is an index equivalent to let's say rmse when someone is running a linear regression. For example, what if the parameter breaks with BIC instead of LWZ detects more accurate the breakpoints (by visually inspecting the detected breakpoints)? Apart from the visual inspection, shouldn't be some other way to measure the performance of the model? Based on the above, is there a more efficient way to optimize the parameters of the model (based on some metric)? What do I mean by optimizing the parameters? I think with an example I can explain it better. When someone is tuning a random forest model, he/she can perform a full grid search to find the optimal parameters of the model (mtry, number of trees, etc) by searching all the possible combinations and for each combination he/she checks the rmse (or mse, r-squared). Is this what the authors of the paper meant when they said "Needs parameter tuning to optimise performance"? And if so, how did they do it? library(bfast) plot(simts) # stl object containing simulated NDVI time series datats <- ts(rowSums(simts$time.series)) # sum of all the components (season,abrupt,remainder) tsp(datats) <- tsp(simts$time.series) # assign correct time series attributes plot(datats) # Detect breaks. default parameters bp = bfastlite(datats) plot(bp) # optimized model ?? bp_opt <- bfastlite() R 4.4.1, bfast 1.6.1, Windows 11. [[alternative HTML version deleted]] ___ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
