Re: [FFmpeg-devel] GSoC mentored project: derain filter
On Wed, Feb 20, 2019 at 06:35:22PM +0800, 孟学苇 wrote: > Hi Dev-Community, > > > > > I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, > Peking University. I wish to contribute as a GSoC applicant this year. > > I am interested in Deep Learning. I want to add a derain filter in ffmpeg. If > you have any suggestion or question, we can contact by email. My motivation > and plans are as follows. > > > > >Motivation > > Rain and fog are very common weather in actual life. However, it can affect > the visibility. Especially in heavy rain, rain streaks from various > directions accumulate and make the background scene misty, which will > seriously influence the accuracy of many computer vision systems, including > video surveillance, object detection and tracking in autonomous driving, etc. > Therefore, it is an important task to remove the rain and fog, and recover > the background from rain images. It can be used for image and video > processing to make them clearer and it can be a preprocessing method for many > computer vision systems. > > > > >Proposed Idea > > We propose to implement this technology in ffmpeg. For video [1][2], we can > utilize the relationship between frames to remove rain and fog. For single > image [3], we can use traditional methods, such as discriminative sparse > coding, low rank representation and the Gaussian mixture model. We can also > use some deep learning methods. We should investigate these methods, and > ultimately consider the effect of rain/fog removal and the complexity of the > algorithm, and choose the optimal scheme. > > > > >Practical application > > The derain and dehaze method can improve the subjective quality of videos and > images. > > > > > Development plan > > I would like to start working on my qualification task and try to solve my > problems. Overall, I will follow the following steps to complete the project. > > (1)Literature and algorithm investigation > > (2)Data sets preparation > > (3)Coding: Implement network, training code, inference code and so on > > (4)Select the best method and transplantation it into ffmpeg I would suggest to do the coding within FFmpeg from as early as possible, not to implement things outside and transplant later. This should be less work and avoids underestimating the work to transplant the code [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you think the mosad wants you dead since a long time then you are either wrong or dead since a long time. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] GSoC mentored project: derain filter
> -Original Message- > From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf > Of Liu Steven > Sent: Wednesday, February 20, 2019 7:18 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Thilo Borgmann ; Liu Steven > > Subject: Re: [FFmpeg-devel] GSoC mentored project: derain filter > > > > > 在 2019年2月20日，下午6:35，孟学苇 写道： > > > > Hi Dev-Community, > > > > > > > > > > I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, > Peking University. I wish to contribute as a GSoC applicant this year. > > > > I am interested in Deep Learning. I want to add a derain filter in ffmpeg. > > If > you have any suggestion or question, we can contact by email. My > motivation and plans are as follows. > > > > > > > > > > Motivation > > > > Rain and fog are very common weather in actual life. However, it can affect > the visibility. Especially in heavy rain, rain streaks from various directions > accumulate and make the background scene misty, which will seriously > influence the accuracy of many computer vision systems, including video > surveillance, object detection and tracking in autonomous driving, etc. > Therefore, it is an important task to remove the rain and fog, and recover the > background from rain images. It can be used for image and video processing > to make them clearer and it can be a preprocessing method for many > computer vision systems. > > > > > > > > > > Proposed Idea > > > > We propose to implement this technology in ffmpeg. For video [1][2], we > can utilize the relationship between frames to remove rain and fog. For > single image [3], we can use traditional methods, such as discriminative > sparse coding, low rank representation and the Gaussian mixture model. We > can also use some deep learning methods. We should investigate these > methods, and ultimately consider the effect of rain/fog removal and the > complexity of the algorithm, and choose the optimal scheme. > > > > > > > > > > Practical application > > > > The derain and dehaze method can improve the subjective quality of > videos and images. > > > > > > > > > > Development plan > > > > I would like to start working on my qualification task and try to solve my > problems. Overall, I will follow the following steps to complete the project. > > > > (1)Literature and algorithm investigation > > > > (2)Data sets preparation > > > > (3)Coding: Implement network, training code, inference code and so on > > > > (4)Select the best method and transplantation it into ffmpeg > > > > > > > > > > Reference > > > > [1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining > > temporal > and chromatic properties[C]//2006 IEEE International Conference on > Multimedia and Expo. IEEE, 2006: 461-464. > > > > [2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a > review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430. > > > > [3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context > aggregation net for single image deraining[C]//Proceedings of the European > Conference on Computer Vision (ECCV). 2018: 254-269. > > > > > > > I think this can reference libavflter/sr.c to implementation, maybe you can > try two ways to implement it, one is native and the other is model. > and currently, only TensorFlow model is supported via tensorflow C API, you can easily save the model file in python with function tf.graph_util.convert_variables_to_constants and tf.train.write_graph. For the native mode (executed with CPU), two operations (CONV and DEPTH_TO_SPACE) are supported now, you might add more. > > Thanks > Steven > > > > > > > > > > > Thanks, > > > > Regards, > > > > Iris Meng > > ___ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] GSoC mentored project: derain filter
> 在 2019年2月20日，下午6:35，孟学苇 写道： > > Hi Dev-Community, > > > > > I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, > Peking University. I wish to contribute as a GSoC applicant this year. > > I am interested in Deep Learning. I want to add a derain filter in ffmpeg. If > you have any suggestion or question, we can contact by email. My motivation > and plans are as follows. > > > > > Motivation > > Rain and fog are very common weather in actual life. However, it can affect > the visibility. Especially in heavy rain, rain streaks from various > directions accumulate and make the background scene misty, which will > seriously influence the accuracy of many computer vision systems, including > video surveillance, object detection and tracking in autonomous driving, etc. > Therefore, it is an important task to remove the rain and fog, and recover > the background from rain images. It can be used for image and video > processing to make them clearer and it can be a preprocessing method for many > computer vision systems. > > > > > Proposed Idea > > We propose to implement this technology in ffmpeg. For video [1][2], we can > utilize the relationship between frames to remove rain and fog. For single > image [3], we can use traditional methods, such as discriminative sparse > coding, low rank representation and the Gaussian mixture model. We can also > use some deep learning methods. We should investigate these methods, and > ultimately consider the effect of rain/fog removal and the complexity of the > algorithm, and choose the optimal scheme. > > > > > Practical application > > The derain and dehaze method can improve the subjective quality of videos and > images. > > > > > Development plan > > I would like to start working on my qualification task and try to solve my > problems. Overall, I will follow the following steps to complete the project. > > (1)Literature and algorithm investigation > > (2)Data sets preparation > > (3)Coding: Implement network, training code, inference code and so on > > (4)Select the best method and transplantation it into ffmpeg > > > > > Reference > > [1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining > temporal and chromatic properties[C]//2006 IEEE International Conference on > Multimedia and Expo. IEEE, 2006: 461-464. > > [2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a > review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430. > > [3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context > aggregation net for single image deraining[C]//Proceedings of the European > Conference on Computer Vision (ECCV). 2018: 254-269. > > > I think this can reference libavflter/sr.c to implementation, maybe you can try two ways to implement it, one is native and the other is model. Thanks Steven > > > > > Thanks, > > Regards, > > Iris Meng > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] GSoC mentored project: derain filter
Hi Dev-Community, I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, Peking University. I wish to contribute as a GSoC applicant this year. I am interested in Deep Learning. I want to add a derain filter in ffmpeg. If you have any suggestion or question, we can contact by email. My motivation and plans are as follows. Motivation Rain and fog are very common weather in actual life. However, it can affect the visibility. Especially in heavy rain, rain streaks from various directions accumulate and make the background scene misty, which will seriously influence the accuracy of many computer vision systems, including video surveillance, object detection and tracking in autonomous driving, etc. Therefore, it is an important task to remove the rain and fog, and recover the background from rain images. It can be used for image and video processing to make them clearer and it can be a preprocessing method for many computer vision systems. Proposed Idea We propose to implement this technology in ffmpeg. For video [1][2], we can utilize the relationship between frames to remove rain and fog. For single image [3], we can use traditional methods, such as discriminative sparse coding, low rank representation and the Gaussian mixture model. We can also use some deep learning methods. We should investigate these methods, and ultimately consider the effect of rain/fog removal and the complexity of the algorithm, and choose the optimal scheme. Practical application The derain and dehaze method can improve the subjective quality of videos and images. Development plan I would like to start working on my qualification task and try to solve my problems. Overall, I will follow the following steps to complete the project. (1)Literature and algorithm investigation (2)Data sets preparation (3)Coding: Implement network, training code, inference code and so on (4)Select the best method and transplantation it into ffmpeg Reference [1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining temporal and chromatic properties[C]//2006 IEEE International Conference on Multimedia and Expo. IEEE, 2006: 461-464. [2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430. [3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 254-269. Thanks, Regards, Iris Meng ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
