Adding to the previous email it should be easily possible to add both the version of deeplab_v3 which are with the resnet 50 and resnet 101 backbone because they have the same blocks but just different params with the different number of layers so I want to discuss the points mentioned in the previous mail but also to add to that I want to discuss the scope of the project should I keep it concretely to a data loader and to predefined but not pre-trained models which are deeplab_v3 with resnet 50 and resnet 101 backbone. For an iteration, I would like to propose adding the following to just make it clear. 1. segmentation data loader 2. deeplabv3 (resnet50 and resnet101) predefined and not pre-trained
I know we previously talked about adding another model if time permits, but I am not really sure about that, so I wanted to discuss these and see if I should add the third model to my proposal and mention the time constraint and just let it be a potential model or if I should completely remove the idea of the third model. Best, Aakash On Sun, Mar 28, 2021 at 2:33 PM Aakash kaushik <[email protected]> wrote: > Hey Everyone, > > This is a continued mail regarding my proposal and I have been taking a > deeper look and found out that PyTorch implements these segmentations > models above their pre-existing pipelines of backbones and I proposed to > implement 3 things 1 data loader for segmentation task and 2 models from > which one would be a potential model only if time permits but the way I see > for the present time they can be implemented in mlpack is by creating block > and creating completed models because no such customizable backbones exist > as of now that can be called. and so this opens another question as to such > deeplab_v3 consists of three backbones in PyTorch which are resnet 50, > resnet 101, and mobilenet v3 large and as coco is such a huge dataset and > the tool for converting weights from torch to mlpack is a bit flack should > I go with a proposal to include predefined models which can be trained > rather than pre-trained models for now and if it permits we can add the > weights for coco in the future. I think these were some of the doubts I had > while I was writing the exacts of my proposal and I will be glad to have a > discussion on it in terms of how concrete it is and if you guys see a > problem on how they can be implemented or any other doubts or questions. > > Best, > Aakash > > On Thu, Mar 18, 2021 at 5:35 AM Aakash kaushik < > [email protected]> wrote: > >> Hey Marcus, >> >> I totally got it and i think 1 data loader and 2 models from which 1 will >> be a potential model if only time permits. >> >> Thank you for the feedback and help. :D >> >> Best, >> Aakash >> >> >> On Wed, 17 Mar, 2021, 9:33 pm Marcus Edel, <[email protected]> >> wrote: >> >>> Yes, that’s what I had in mine, but at the end it’s your decision. About >>> the >>> model either is fine, you can select whatever you find more interesting. >>> >>> On 17. Mar 2021, at 10:45, Aakash kaushik <[email protected]> >>> wrote: >>> >>> HI Marcus, >>> >>> Thank you so much for reaching back, So just to clarify i would keep the >>> deliverables to just two which will be: >>> >>> 1. Semantic segmentation dataloader in the format of COCO dataset . >>> 2. One semantic segmentation model >>> >>> If I understood you correctly, will you be able to help me decide which >>> kind of model I should add, should i go for a model that is more generally >>> used such as U-Net or one from the above list that PyTorch has ? >>> >>> Best, >>> Aakash >>> >>> On Wed, Mar 17, 2021 at 7:55 PM Marcus Edel <[email protected]> >>> wrote: >>> >>>> Hello Aakash, >>>> >>>> thanks for the interest in the project and all the contributions; what >>>> you proposed >>>> looks quite useful to me and as you already pointed out would integrate >>>> really well >>>> with some of the existing functionalities. >>>> >>>> I guess for loading segmentation datasets we will stick with a common >>>> format e.g. >>>> COCO, and add support for the data loader and potentially add support >>>> for other >>>> formats later? >>>> >>>> One remark about the scope, you might want to remove one model from the >>>> list, and >>>> add a note to the proposal something along the lines of, if there is >>>> time left at the end >>>> of the summer, I propose to work on z, but the focus is on x and y. >>>> >>>> I hope what I said was useful; please don't hesitate to ask if anything >>>> needs clarification. >>>> >>>> Thanks, >>>> Marcus >>>> >>>> On 16. Mar 2021, at 00:16, Aakash kaushik <[email protected]> >>>> wrote: >>>> >>>> Hey everyone, >>>> >>>> My name is Aakash Kaushik <https://github.com/Aakash-kaushik> and I >>>> have been contributing for some time specifically on the ANN codebase in >>>> mlpack. >>>> >>>> And the project idea that is ready to use Models in mlpack peaks my >>>> interest. So initially i would like to propose a data loader and 2 models >>>> for semantic segmentation because i see that the data loaders for image >>>> classification and object detection are already there and including a >>>> couple of models and a data loader in GSOC for semantic segmentation will >>>> open the gates for further contribution of models in all three fields as >>>> they would only need to worry about the model and not loading the data and >>>> also will have some reference models in that field >>>> >>>> So the data loader would be capable of taking up image segmentation >>>> data that is the real image, segmentation map, segmentation map to class >>>> mapping, and for the models i am a bit confused as if we want some basic >>>> nets such as U-nets or a combination of both a basic net and state of the >>>> art model, or two state of the art model. Pytorch supports couple of models >>>> in the semantic segmentation fields which are: >>>> >>>> 1. FCN ResNet50, ResNet101 >>>> 2. DeepLabV3 ResNet50, ResNet101, MobileNetV3-Large >>>> 3. LR-ASPP MobileNetV3-Large >>>> >>>> And so i should be able to convert their weights from pytorch to mlpack >>>> by modifying the utility created by kartik dutt which is >>>> mlpack-PyTorch-Weight-Translator >>>> <https://github.com/kartikdutt18/mlpack-PyTorch-Weight-Translator> >>>> >>>> I am trying to keep the deliverables to just three which is a data >>>> loader and 2 models as the GSOC period is reduced to just 1.5 months and >>>> for these three things i would have to write tests, documentation and >>>> example usage in the example repository. >>>> >>>> And before this work as we are in the process of removing boost >>>> visitors from the ANN codebase and had couple of major changes to the >>>> mlpack codebase the models repo wasn't able to keep up with it so my main >>>> goal before GSOC starts would be to work on the PR that is to Swap >>>> boost::variant with vtable <https://github.com/mlpack/mlpack/pull/2777> and >>>> then make changes to the code in models repo to adjust the change in boost >>>> visitors, serialization and porting tests to catch2. >>>> >>>> I wanted to hear from you if this is the right path and if the number >>>> of deliverables are right for this and help in choosing the exact models >>>> that i should pick that would be the most helpful or beneficial to the >>>> library. >>>> >>>> Best, >>>> Aakash >>>> _______________________________________________ >>>> mlpack mailing list >>>> [email protected] >>>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack >>>> >>>> >>>> >>>
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
