Re: [mlpack] Robotic Arm - GSoC Project Idea

[Marcus Edel]

Hello Vaibhav,

looks fine to me, my only comment would be, do you think you could add a comment
about what part of mlpack could be used for the project?

Thanks,
Marcus

> On 27. Mar 2018, at 04:42, Vaibhav Jain  wrote:
> 
> Hey Marcus,
> I submitted my proposal draft quite a while ago but haven't received any 
> inputs yet. I can understand that there are a lot of proposals coming through 
> at this point of time. But since final deadline line very near, can you let 
> me know what you think at once.
> Thanks for your hard work.
> 
> Regards
> -- Vaibhav Jain

___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Vaibhav Jain]

Hey Marcus,
I submitted my proposal draft quite a while ago but haven't received any
inputs yet. I can understand that there are a lot of proposals coming
through at this point of time. But since final deadline line very near, can
you let me know what you think at once.
Thanks for your hard work.

Regards
-- Vaibhav Jain
___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Vaibhav Jain]

Hey Marcus.


> That is definitely one way, but perhaps we can simplify the way, e.g. by
> using
> only a single camera, this makes the overall training more challenging
> since,
> the preprocessing step is more complex; but at the end, you can directly
> use the
> pipeline for a broad range of applications, without extracting a lot of
> model-
> specific information. Let me know what you think.
>
> About the setup of the simulation, it would be best to use a single Kinect
Sensor which will cover the field of operation as well as the whole arm. We
can run any other package (like this 
one) to detect the motion of the arm. The main challenge would be to train
the arm to position itself for better grip on the object. By that I mean,
it is easy to move the end effector from one position in 3D space to
another. We do not need machine learning for that. What we have to train
the robot is how to position itself around the object so that it can pick
random objects of different dimensions. That's my opinion. What do you
think?

Regards,
-- 
- Vaibhav Jain
___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Marcus Edel]

Hello Vaibhav,

thanks for the update.

> # First we need to find the required final coordinates of the object (in the
>   simulation we would already know that, but in the real world, we need to use
>   some 3D camera).
> # Then we can find the optimal path using trajectory planning techniques
> # After that we can find the joint angles using inverse kinematics
> # Using these joint angles, we can train our model

That is definitely one way, but perhaps we can simplify the way, e.g. by using
only a single camera, this makes the overall training more challenging since,
the preprocessing step is more complex; but at the end, you can directly use the
pipeline for a broad range of applications, without extracting a lot of model-
specific information. Let me know what you think.

> I wanted to you ask how to train mlpack models on large datasets on cloud. It
> would help me to create a detailed timeline for my proposal. Please let me 
> know
> what you think.

Do you mean distributed training? I think for this project we can start small,
but we could we can use Downpour SGD to distribute the load.

Thanks,
Marcus

> On 12. Mar 2018, at 09:18, Vaibhav Jain  wrote:
> 
> Hey Marcus,
> I think this would be a good approach to work on this project:
> # First we need to find the required final coordinates of the object (in the 
> simulation we would already know that, but in the real world, we need to use 
> some 3D camera).
> # Then we can find the optimal path using trajectory planning techniques
> # After that we can find the joint angles using inverse kinematics
> # Using these joint angles, we can train our model
> 
> For learning manipulator kinematics/dynamics, lectures by Prof. Oussama 
> Khatib at Stanford, are very good. The book "Introduction to Robotics: 
> Mechanics and Control" by J. Craig is also very good.
> For simulation, the environments you suggested would be surely helpful. 
> Although, I need to work a little bit more to see the extent to which they 
> can be used.
> 
> I wanted to you ask how to train mlpack models on large datasets on cloud. It 
> would help me to create a detailed timeline for my proposal.
> Please let me know what you think.
> 
> On Wed, Feb 28, 2018 at 6:22 PM, Marcus Edel  > wrote:
> Hello Vaibhav,
> 
> OpenAI released a couple of robotics environments wich could be interesting:
> https://github.com/openai/gym/tree/master/gym/envs/robotics 
> 
> 
> Best,
> Marcus
> 
>> On 27. Feb 2018, at 17:00, Vaibhav Jain > > wrote:
>> 
>> Hello Marcus,
>> Thanks for this resource. I will sure check it out. In the meantime, I will 
>> also research some other previous similar projects.
>> 
>> 
>> On Feb 27, 2018 03:36, "Marcus Edel" > > wrote:
>> Hello Vaibhav,
>> 
>> thanks for the input, I agree ROS is definitely a good option to get started,
>> the available resources is just one point. My initial idea was to integrate 
>> the
>> simulator in the OpenAI Gym framework, mlpack already follows the gym 
>> interface
>> so the interaction is already in place, but I think we could use the Gazebo
>> extension for Gym to get it working. This looks promising
>> (https://github.com/erlerobot/gym-gazebo 
>> ). Not sure the install process is
>> straightforward, in the best case we can install everything with a few clicks
>> and most packages are available for a bunch of distributions.
>> 
>> Thanks,
>> Marcus
>> 
>> __
>> Vaibhav Jain
>> 
> 
> 
> 
> Regards, 
> -- 
> - Vaibhav Jain

___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Vaibhav Jain]

Hey Marcus,
I think this would be a good approach to work on this project:
# First we need to find the required final coordinates of the object (in
the simulation we would already know that, but in the real world, we need
to use some 3D camera).
# Then we can find the optimal path using trajectory planning techniques
# After that we can find the joint angles using inverse kinematics
# Using these joint angles, we can train our model

For learning manipulator kinematics/dynamics, lectures by Prof. Oussama
Khatib at Stanford, are very good. The book "Introduction to Robotics:
Mechanics and Control" by J. Craig is also very good.
For simulation, the environments you suggested would be surely helpful.
Although, I need to work a little bit more to see the extent to which they
can be used.

I wanted to you ask how to train mlpack models on large datasets on cloud.
It would help me to create a detailed timeline for my proposal.
Please let me know what you think.

On Wed, Feb 28, 2018 at 6:22 PM, Marcus Edel 
wrote:

> Hello Vaibhav,
>
> OpenAI released a couple of robotics environments wich could be
> interesting:
> https://github.com/openai/gym/tree/master/gym/envs/robotics
>
> Best,
> Marcus
>
> On 27. Feb 2018, at 17:00, Vaibhav Jain  wrote:
>
> Hello Marcus,
> Thanks for this resource. I will sure check it out. In the meantime, I
> will also research some other previous similar projects.
>
>
> On Feb 27, 2018 03:36, "Marcus Edel"  wrote:
>
> Hello Vaibhav,
>
> thanks for the input, I agree ROS is definitely a good option to get
> started,
> the available resources is just one point. My initial idea was to
> integrate the
> simulator in the OpenAI Gym framework, mlpack already follows the gym
> interface
> so the interaction is already in place, but I think we could use the Gazebo
> extension for Gym to get it working. This looks promising
> (https://github.com/erlerobot/gym-gazebo). Not sure the install process is
> straightforward, in the best case we can install everything with a few
> clicks
> and most packages are available for a bunch of distributions.
>
> Thanks,
> Marcus
>
>
> __
> Vaibhav Jain
>
>
>

Regards,
-- 
- Vaibhav Jain
___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Marcus Edel]

Hello Vaibhav,

OpenAI released a couple of robotics environments wich could be interesting:
https://github.com/openai/gym/tree/master/gym/envs/robotics

Best,
Marcus

> On 27. Feb 2018, at 17:00, Vaibhav Jain  wrote:
> 
> Hello Marcus,
> Thanks for this resource. I will sure check it out. In the meantime, I will 
> also research some other previous similar projects.
> 
> 
> On Feb 27, 2018 03:36, "Marcus Edel"  > wrote:
> Hello Vaibhav,
> 
> thanks for the input, I agree ROS is definitely a good option to get started,
> the available resources is just one point. My initial idea was to integrate 
> the
> simulator in the OpenAI Gym framework, mlpack already follows the gym 
> interface
> so the interaction is already in place, but I think we could use the Gazebo
> extension for Gym to get it working. This looks promising
> (https://github.com/erlerobot/gym-gazebo 
> ). Not sure the install process is
> straightforward, in the best case we can install everything with a few clicks
> and most packages are available for a bunch of distributions.
> 
> Thanks,
> Marcus
> 
> __
> Vaibhav Jain
> 

___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Vaibhav Jain]

Hello Marcus,
Thanks for this resource. I will sure check it out. In the meantime, I will
also research some other previous similar projects.


On Feb 27, 2018 03:36, "Marcus Edel"  wrote:

Hello Vaibhav,

thanks for the input, I agree ROS is definitely a good option to get
started,
the available resources is just one point. My initial idea was to integrate
the
simulator in the OpenAI Gym framework, mlpack already follows the gym
interface
so the interaction is already in place, but I think we could use the Gazebo
extension for Gym to get it working. This looks promising
(https://github.com/erlerobot/gym-gazebo). Not sure the install process is
straightforward, in the best case we can install everything with a few
clicks
and most packages are available for a bunch of distributions.

Thanks,
Marcus


__
Vaibhav Jain
___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Marcus Edel]

Hello Vaibhav,

thanks for the input, I agree ROS is definitely a good option to get started,
the available resources is just one point. My initial idea was to integrate the
simulator in the OpenAI Gym framework, mlpack already follows the gym interface
so the interaction is already in place, but I think we could use the Gazebo
extension for Gym to get it working. This looks promising
(https://github.com/erlerobot/gym-gazebo). Not sure the install process is
straightforward, in the best case we can install everything with a few clicks
and most packages are available for a bunch of distributions.

Thanks,
Marcus

> On 26. Feb 2018, at 10:42, Vaibhav Jain  wrote:
> 
> Hey Marcus,
> I think ROS is a better platform to work on since it already has many 
> built-in libraries to handle the control system of the arm. It has a large 
> community and libraries dedicated to robotic systems which will help us to 
> focus on our task rather than writing scripts for control and path planning 
> from scratch. For eg., we can use kinect sensor to monitor the orientation of 
> the arm which can be easily added to gazebo. Through many available 
> libraries, we can just get joint angles of the arm. Also, it will be easier 
> to implement this on an actual robotic arm later if we use ROS as a platform 
> (if someone plans to do it).
> 
> As far as for computation concerns, we probably won't be training on the same 
> machine with ROS unless we have a GPU onboard. If GPU is available, then 
> there won't be any problems with the performance since ROS (and gazebo) can 
> easily run on any modern machine. But assuming we proceed with the aim in 
> mind that the project should eventually be able to implementable on actual 
> manipulators, most manipulators will not have GPU on them (*I say "most" but 
> I don't think any of them does*). So we will have to be able to access remote 
> GPU.
> Although we can still use OpenGL, I think using ROS is a better and faster 
> idea. I also think there wasn't much work done on mlpack with ROS, so I think 
> this is a good opportunity to start with. Let me know what you think.
> 
> Regards,
> 
> -- 
> - Vaibhav Jain
> 
> On Sun, Feb 25, 2018 at 6:39 PM, Marcus Edel  > wrote:
> Hello Vaibhav,
> 
> thanks for getting in touch.
> 
> > One of your project, Robotic Arm, specially catches my eye as I have 
> > previously
> > worked on a fair amount of projects based on robotic manipulators and 
> > humanoid.
> > One of them also includes a gesture-controlled robotic arm in which a 
> > robotic
> > arm mimics the motion of the user's arm. I have also worked on ROS and 
> > gazebo
> > simulator. It would be an interesting idea to use mlpack with ROS which 
> > could
> > give Gazebo more flexibility.
> 
> That sounds really interesting, I was also thinking about using ROS for the
> simulation, I'm not a ROS expert but do you think that is a good option as 
> e.g.
> compared to a simple python OpenGL script? There are two points that I think 
> are
> important, performance: the training is going to consume all resources, so the
> simulator should not be demanding; easy to install, I think everyone likes to
> keep the number of dependencies low. Let me know what you think.
> 
> > Right now, I have build and gone through some tutorials on mlpack. I am 
> > trying
> > to get more familiar with mlpack's codebase. If you can give me some 
> > pointers
> > for the project, it would be awesome. Thanks.
> 
> Sounds good, for this project the ann and RL code is especially important, so
> taking a closer look at the tests is a good way to get familiar with that part
> of the codebase.
> 
> Let me know if I should clarify anything.
> 
> Thanks,
> Marcus
> 
> 

___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Vaibhav Jain]

Hey Marcus,
I think ROS is a better platform to work on since it already has many
built-in libraries to handle the control system of the arm. It has a large
community and libraries dedicated to robotic systems which will help us to
focus on our task rather than writing scripts for control and path planning
from scratch. For eg., we can use kinect sensor to monitor the orientation
of the arm which can be easily added to gazebo. Through many available
libraries, we can just get joint angles of the arm. Also, it will be easier
to implement this on an actual robotic arm later if we use ROS as a
platform (if someone plans to do it).

As far as for computation concerns, we probably won't be training on the
same machine with ROS unless we have a GPU onboard. If GPU is available,
then there won't be any problems with the performance since ROS (and
gazebo) can easily run on any modern machine. But assuming we proceed with
the aim in mind that the project should eventually be able to implementable
on actual manipulators, most manipulators will not have GPU on them (*I say
"most" but I don't think any of them does*). So we will have to be able to
access remote GPU.
Although we can still use OpenGL, I think using ROS is a better and faster
idea. I also think there wasn't much work done on mlpack with ROS, so I
think this is a good opportunity to start with. Let me know what you think.

Regards,

-- 
- Vaibhav Jain

On Sun, Feb 25, 2018 at 6:39 PM, Marcus Edel 
wrote:

> Hello Vaibhav,
>
> thanks for getting in touch.
>
> > One of your project, Robotic Arm, specially catches my eye as I have
> previously
> > worked on a fair amount of projects based on robotic manipulators and
> humanoid.
> > One of them also includes a gesture-controlled robotic arm in which a
> robotic
> > arm mimics the motion of the user's arm. I have also worked on ROS and
> gazebo
> > simulator. It would be an interesting idea to use mlpack with ROS which
> could
> > give Gazebo more flexibility.
>
> That sounds really interesting, I was also thinking about using ROS for the
> simulation, I'm not a ROS expert but do you think that is a good option as
> e.g.
> compared to a simple python OpenGL script? There are two points that I
> think are
> important, performance: the training is going to consume all resources, so
> the
> simulator should not be demanding; easy to install, I think everyone likes
> to
> keep the number of dependencies low. Let me know what you think.
>
> > Right now, I have build and gone through some tutorials on mlpack. I am
> trying
> > to get more familiar with mlpack's codebase. If you can give me some
> pointers
> > for the project, it would be awesome. Thanks.
>
> Sounds good, for this project the ann and RL code is especially important,
> so
> taking a closer look at the tests is a good way to get familiar with that
> part
> of the codebase.
>
> Let me know if I should clarify anything.
>
> Thanks,
> Marcus
>
>
___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Robotic Arm - GSoC Project Idea

[Marcus Edel]

Hello Vaibhav,

thanks for getting in touch.

> One of your project, Robotic Arm, specially catches my eye as I have 
> previously
> worked on a fair amount of projects based on robotic manipulators and 
> humanoid.
> One of them also includes a gesture-controlled robotic arm in which a robotic
> arm mimics the motion of the user's arm. I have also worked on ROS and gazebo
> simulator. It would be an interesting idea to use mlpack with ROS which could
> give Gazebo more flexibility.

That sounds really interesting, I was also thinking about using ROS for the
simulation, I'm not a ROS expert but do you think that is a good option as e.g.
compared to a simple python OpenGL script? There are two points that I think are
important, performance: the training is going to consume all resources, so the
simulator should not be demanding; easy to install, I think everyone likes to
keep the number of dependencies low. Let me know what you think.

> Right now, I have build and gone through some tutorials on mlpack. I am trying
> to get more familiar with mlpack's codebase. If you can give me some pointers
> for the project, it would be awesome. Thanks.

Sounds good, for this project the ann and RL code is especially important, so
taking a closer look at the tests is a good way to get familiar with that part
of the codebase.

Let me know if I should clarify anything.

Thanks,
Marcus


> On 25. Feb 2018, at 13:13, Vaibhav Jain  wrote:
> 
> Hello,
> My name is Vaibhav Jain. I am an undergraduate student majoring in IT and 
> Maths at Delhi University. I am very much interested in participating in GSoC 
> this year with your organization.
> One of your project, Robotic Arm, specially catches my eye as I have 
> previously worked on a fair amount of projects based on robotic manipulators 
> and humanoid. One of them also includes a gesture-controlled robotic arm in 
> which a robotic arm mimics the motion of the user's arm. I have also worked 
> on ROS and gazebo simulator. It would be an interesting idea to use mlpack 
> with ROS which could give Gazebo more flexibility. I am also familiar with 
> machine learning and deep learning, and I am looking forward to implementing 
> these on a robotic arm.
> Right now, I have build and gone through some tutorials on mlpack. I am 
> trying to get more familiar with mlpack's codebase. If you can give me some 
> pointers for the project, it would be awesome. Thanks.
>  --
> Vaibhav Jain
> University of Delhi
> ___
> mlpack mailing list
> mlpack@lists.mlpack.org
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack