On 2019-09-24 18:44:PM, YKY (Yan King Yin, 甄景贤) wrote:
On Wed, Sep 25, 2019 at 12:03 AM doddy <[email protected]
<mailto:[email protected]>> wrote:
how effecient is it compared to self supervised learning?
You mean unsupervised? I am not seeing much of a difference between
the 2 notions.
"Self-supervised learning" is an important part of the "deep learning"
revolution. It consists of the observation that classical supervised
learning techniques can be applied to what were previously considered
to be "unsupervised" learning tasks. A classic example is reinforcement
learning. That was previously categorized as a form of unsupervised
learning. However, a famous paper from Deep Mind titled "Playing Atari
with Deep Reinforcement Learning" showed how to apply classical neural
networks to them, using supervised learning. More generally, if part
of the input can be considered to be obscured or hidden, inferring
it can be treated as a supervised learning task. Applications are
numerous. For example if you have a collection of photos, you can
hide part of them with an artificial "thumb" and then use the
obscured photo as an input and the original photo as an output -
and train a convolutional net to produce one from the other.
Another application is forecasting tasks. If you want to predict
the next element in a stream, and you have historical training
data relating to that stream, you can train a network with incomplete
sequences as the input and the next symbol as the output. It even
works if you want to predict a symbol far downstream. Because
forecasting is so central to cognition, this is a big deal.
Yann LeCun has promoted the term "self-supervised learning"
to refer to this whole idea. Previously, it was common to
hear researchers lamenting the lack of labelled training data to
feed to their supervised learning algorithms. Now no longer.
If so many input streams can usefully be considered to include
their own labels, and finding them is valuable, suddenly almost
any problem turns into one to which classical supervised learning
techniques can be applied. Where does this leave unsupervised
learning? Kind-of in the dust. We once thought that was a useful
category, but now, not so much. No supervisor? Supervise yourself.
The revolution will not be supervised.
--
__________
|im |yler http://timtyler.org/
------------------------------------------
Artificial General Intelligence List: AGI
Permalink:
https://agi.topicbox.com/groups/agi/T77af318d4abfa8a8-Mc3d79cac1434e3086ae78aa4
Delivery options: https://agi.topicbox.com/groups/agi/subscription
