Hey Marcus, So it would ideally be the same because a modular structure will allow us to build resnet with different layers and I have gone through the source code that PyTorch has they only create it in modules because it allows creating different Resnets without every time writing the same block(multiple layers), So I think as to narrowing down my proposal I would leave segmentation, for now, I can totally do it even after GSoC and would propose a modular resnet and if time persists I would also like to add mobilenetv3 in a modular way because if I can start that work in the GSoC period I will finish it even after GSoC ends.
Feeling really stocked to start working on this and as I finish iterating over my proposal, I will submit an initial draft through GSoC so you can take a look and reach back for feedback and help around if I forgot to mention something because I won't like proposing a well-documented project is no joke and the applications get big and it's hard for you to even read them. Best, Aakash. On Mon, Mar 29, 2021 at 6:58 PM Marcus Edel <[email protected]> wrote: > Agreed, ideally, the models should be customizable, but I don't think we > have to > provide an implementation that has the same functionalities as PyTorch, > it's good > to think about possible extensions, though. Also, I agree that we should > implement > the parts as independent blocks in the models repository. Depending on > what you > like to propose, it would be helpful to reduce the project's scope, like > if you are > planning to implement resnet with the same functionalities as PyTorch, I > think that > alone is enough work to fill the summer. > > On 28. Mar 2021, at 12:19, Aakash kaushik <[email protected]> > wrote: > > Hey Marcus, > > Thank you for reaching back, So when implementing resnet we would need to > implement it in such a way that it supports all the versions that Pytorch > has to offer and on top of that all the extras that are needed to use them > as backbones amd the same goes with mobilenet we cannot kist implement a > single flavour of them we would need to implement these models do as they > are customisable in themselves just like Pytorch. What do you think about > that and what do you propose i go ahead with? > > Best, > Aakash > > On Sun, 28 Mar, 2021, 9:45 pm Marcus Edel, <[email protected]> > wrote: > >> Hello Aakash, >> >> I really like the thoughts you put into the project, as you already >> pointed >> out for deeplab v3 we would need resnet 50, resnet 101, and MobileNet v3, >> but looks like MobileNet is missing from the second mail timeline. >> >> Looking at the timeline, I'm wondering if it makes more sense to implement >> MobileNetV3 and restnet 50 as independent modules first and maybe do >> the rest if there is time left. That said, I guess implementing MobileNet >> with >> some strong test could already fill the summer. >> >> Let me know what you think. >> >> Thanks, >> Marcus >> >> On 28. Mar 2021, at 11:48, Aakash kaushik <[email protected]> >> wrote: >> >> Slight changes to the previous email I would like to mention in my >> proposal that the aim will be to try to extract the weights from PyTorch >> and make the models pretrained and not just predefined using the converter >> but I am not 100% sure if it would work so will that be something >> acceptable? >> >> On Sun, Mar 28, 2021 at 8:44 PM Aakash kaushik < >> [email protected]> wrote: >> >>> Adding to the previous email it should be easily possible to add both >>> the version of deeplab_v3 which are with the resnet 50 and resnet 101 >>> backbone because they have the same blocks but just different params with >>> the different number of layers so I want to discuss the points mentioned in >>> the previous mail but also to add to that I want to discuss the scope of >>> the project should I keep it concretely to a data loader and to predefined >>> but not pre-trained models which are deeplab_v3 with resnet 50 and resnet >>> 101 backbone. >>> For an iteration, I would like to propose adding the following to just >>> make it clear. >>> 1. segmentation data loader >>> 2. deeplabv3 (resnet50 and resnet101) predefined and not pre-trained >>> >>> I know we previously talked about adding another model if time permits, >>> but I am not really sure about that, so I wanted to discuss these and see >>> if I should add the third model to my proposal and mention the time >>> constraint and just let it be a potential model or if I should completely >>> remove the idea of the third model. >>> >>> Best, >>> Aakash >>> >>> On Sun, Mar 28, 2021 at 2:33 PM Aakash kaushik < >>> [email protected]> wrote: >>> >>>> Hey Everyone, >>>> >>>> This is a continued mail regarding my proposal and I have been taking a >>>> deeper look and found out that PyTorch implements these segmentations >>>> models above their pre-existing pipelines of backbones and I proposed to >>>> implement 3 things 1 data loader for segmentation task and 2 models from >>>> which one would be a potential model only if time permits but the way I see >>>> for the present time they can be implemented in mlpack is by creating block >>>> and creating completed models because no such customizable backbones exist >>>> as of now that can be called. and so this opens another question as to such >>>> deeplab_v3 consists of three backbones in PyTorch which are resnet 50, >>>> resnet 101, and mobilenet v3 large and as coco is such a huge dataset and >>>> the tool for converting weights from torch to mlpack is a bit flack should >>>> I go with a proposal to include predefined models which can be trained >>>> rather than pre-trained models for now and if it permits we can add the >>>> weights for coco in the future. I think these were some of the doubts I had >>>> while I was writing the exacts of my proposal and I will be glad to have a >>>> discussion on it in terms of how concrete it is and if you guys see a >>>> problem on how they can be implemented or any other doubts or questions. >>>> >>>> Best, >>>> Aakash >>>> >>>> On Thu, Mar 18, 2021 at 5:35 AM Aakash kaushik < >>>> [email protected]> wrote: >>>> >>>>> Hey Marcus, >>>>> >>>>> I totally got it and i think 1 data loader and 2 models from which 1 >>>>> will be a potential model if only time permits. >>>>> >>>>> Thank you for the feedback and help. :D >>>>> >>>>> Best, >>>>> Aakash >>>>> >>>>> >>>>> On Wed, 17 Mar, 2021, 9:33 pm Marcus Edel, <[email protected]> >>>>> wrote: >>>>> >>>>>> Yes, that’s what I had in mine, but at the end it’s your decision. >>>>>> About the >>>>>> model either is fine, you can select whatever you find more >>>>>> interesting. >>>>>> >>>>>> On 17. Mar 2021, at 10:45, Aakash kaushik < >>>>>> [email protected]> wrote: >>>>>> >>>>>> HI Marcus, >>>>>> >>>>>> Thank you so much for reaching back, So just to clarify i would keep >>>>>> the deliverables to just two which will be: >>>>>> >>>>>> 1. Semantic segmentation dataloader in the format of COCO dataset . >>>>>> 2. One semantic segmentation model >>>>>> >>>>>> If I understood you correctly, will you be able to help me decide >>>>>> which kind of model I should add, should i go for a model that is more >>>>>> generally used such as U-Net or one from the above list that PyTorch has >>>>>> ? >>>>>> >>>>>> Best, >>>>>> Aakash >>>>>> >>>>>> On Wed, Mar 17, 2021 at 7:55 PM Marcus Edel <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hello Aakash, >>>>>>> >>>>>>> thanks for the interest in the project and all the contributions; >>>>>>> what you proposed >>>>>>> looks quite useful to me and as you already pointed out would >>>>>>> integrate really well >>>>>>> with some of the existing functionalities. >>>>>>> >>>>>>> I guess for loading segmentation datasets we will stick with a >>>>>>> common format e.g. >>>>>>> COCO, and add support for the data loader and potentially add >>>>>>> support for other >>>>>>> formats later? >>>>>>> >>>>>>> One remark about the scope, you might want to remove one model from >>>>>>> the list, and >>>>>>> add a note to the proposal something along the lines of, if there is >>>>>>> time left at the end >>>>>>> of the summer, I propose to work on z, but the focus is on x and y. >>>>>>> >>>>>>> I hope what I said was useful; please don't hesitate to ask if >>>>>>> anything needs clarification. >>>>>>> >>>>>>> Thanks, >>>>>>> Marcus >>>>>>> >>>>>>> On 16. Mar 2021, at 00:16, Aakash kaushik < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> Hey everyone, >>>>>>> >>>>>>> My name is Aakash Kaushik <https://github.com/Aakash-kaushik> and I >>>>>>> have been contributing for some time specifically on the ANN codebase in >>>>>>> mlpack. >>>>>>> >>>>>>> And the project idea that is ready to use Models in mlpack peaks my >>>>>>> interest. So initially i would like to propose a data loader and 2 >>>>>>> models >>>>>>> for semantic segmentation because i see that the data loaders for image >>>>>>> classification and object detection are already there and including a >>>>>>> couple of models and a data loader in GSOC for semantic segmentation >>>>>>> will >>>>>>> open the gates for further contribution of models in all three fields as >>>>>>> they would only need to worry about the model and not loading the data >>>>>>> and >>>>>>> also will have some reference models in that field >>>>>>> >>>>>>> So the data loader would be capable of taking up image segmentation >>>>>>> data that is the real image, segmentation map, segmentation map to class >>>>>>> mapping, and for the models i am a bit confused as if we want some basic >>>>>>> nets such as U-nets or a combination of both a basic net and state of >>>>>>> the >>>>>>> art model, or two state of the art model. Pytorch supports couple of >>>>>>> models >>>>>>> in the semantic segmentation fields which are: >>>>>>> >>>>>>> 1. FCN ResNet50, ResNet101 >>>>>>> 2. DeepLabV3 ResNet50, ResNet101, MobileNetV3-Large >>>>>>> 3. LR-ASPP MobileNetV3-Large >>>>>>> >>>>>>> And so i should be able to convert their weights from pytorch to >>>>>>> mlpack by modifying the utility created by kartik dutt which is >>>>>>> mlpack-PyTorch-Weight-Translator >>>>>>> <https://github.com/kartikdutt18/mlpack-PyTorch-Weight-Translator> >>>>>>> >>>>>>> I am trying to keep the deliverables to just three which is a data >>>>>>> loader and 2 models as the GSOC period is reduced to just 1.5 months and >>>>>>> for these three things i would have to write tests, documentation and >>>>>>> example usage in the example repository. >>>>>>> >>>>>>> And before this work as we are in the process of removing boost >>>>>>> visitors from the ANN codebase and had couple of major changes to the >>>>>>> mlpack codebase the models repo wasn't able to keep up with it so my >>>>>>> main >>>>>>> goal before GSOC starts would be to work on the PR that is to Swap >>>>>>> boost::variant with vtable >>>>>>> <https://github.com/mlpack/mlpack/pull/2777> and then make changes >>>>>>> to the code in models repo to adjust the change in boost >>>>>>> visitors, serialization and porting tests to catch2. >>>>>>> >>>>>>> I wanted to hear from you if this is the right path and if the >>>>>>> number of deliverables are right for this and help in choosing the exact >>>>>>> models that i should pick that would be the most helpful or beneficial >>>>>>> to >>>>>>> the library. >>>>>>> >>>>>>> Best, >>>>>>> Aakash >>>>>>> _______________________________________________ >>>>>>> mlpack mailing list >>>>>>> [email protected] >>>>>>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack >>>>>>> >>>>>>> >>>>>>> >>>>>> >> >
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
