Re: [theano-users] Return a tuple of length-1 from scan?

[Frédéric Bastien]

I answered in the other email on this subject.

On Mon, Jan 16, 2017 at 9:57 AM, Peter O'Connor 
wrote:

> Hi all.
>
> There's some really unintuitive/inconsistent behaviour with theano's scan
> and I'm wondering how I should work around it.
>
> Take the following test, which I would expect to pass:
>
> import theano
> import theano.tensor as tt
>
>
> def test_scan_tuple_output():
>
> x = tt.matrix()
>
> def mul_by_2_and_3(data):
> return (data*2, data*3)
>
> y, updates = theano.scan(mul_by_2_and_3, sequences=x)
>
> assert len(y)==2
> assert len(updates)==0
>
> def mul_by_2(data):
> return (data*2, )
>
> y, updates = theano.scan(mul_by_2, sequences = [x])
>
> assert len(y)==1
> assert len(updates) == 0
>
>
> if __name__ == '__main__':
> test_scan_tuple_output()
>
> It fails on
> It
> It fails on
>
> assert len(y)==1
>
> with "TypeError: object of type 'TensorVariable' has no len()" because if
> scan detects a tuple of length-1, it just returns the data inside this
> tuple.
>
> This becomes annoying when trying to make a scannable function with
> variable return length, (one of which can be one).
>
> Is there a way I can make scan not totally change its behaviour when the
> tuple of returned arguments happens to have length 1?
>
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Scan: Return tuple of length 1?

[Frédéric Bastien]

Thanks for the idea.

At a few places, we allow to pass return_list=True to force the return a of
list. I added that option to scan in that PR:

https://github.com/Theano/Theano/pull/5420

Why we don't always return a list? It is because the best interface depend
of what you want. If you use Theano normally not returning a list is
better, but when you try to automatize stuff, always having a list is
better. That is why we provide both at a few places.

Can you confirm this do what you want?

Fred

On Mon, Jan 16, 2017 at 9:59 AM, Peter O'Connor 
wrote:

> Hi all.
>
> There's some really unintuitive/inconsistent behaviour with theano's scan
> and I'm wondering how I should work around it.
>
> Take the following test, which I would expect to pass:
>
> import theano
> import theano.tensor as tt
>
>
> def test_scan_tuple_output():
>
> x = tt.matrix()
>
> def mul_by_2_and_3(data):
> return (data*2, data*3)
>
> y, updates = theano.scan(mul_by_2_and_3, sequences=[x])
>
> assert len(y)==2
> assert len(updates)==0
>
> def mul_by_2(data):
> return (data*2, )
>
> y, updates = theano.scan(mul_by_2, sequences = [x])
>
> assert len(y)==1
> assert len(updates) == 0
>
>
> if __name__ == '__main__':
> test_scan_tuple_output()
>
> It fails on
> It
> It fails on
>
> assert len(y)==1
>
> with "TypeError: object of type 'TensorVariable' has no len()" because if
> scan detects a tuple of length-1, it just returns the data inside this
> tuple.
>
> This becomes annoying when trying to make a scannable function with
> variable return length, (one of which can be one).
>
> Is there a way I can make scan not totally change its behaviour when the
> tuple of returned arguments happens to have length 1?
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Network does not update its weights

[Beatriz G.]

Hi.

I have been trying to develop imagenet, I have the base of LENET and 
modifying it and adding things I would like to get Imagenet. My problem is 
that weights are not updated so the network does not learn and I can not 
figure out where the problem is and I do not know what to do! I am really 
stressed 
and blocked. So any help or idea would be welcome!!

I have attached two files, "layers.py" where the layers are described and 
"network.py" where the architecture and the learning/testing processing is 
described and carried out.

Thank you very much in advance.

Regards.

Beatriz


-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
import numpy as np
import theano
import theano.tensor as T
from theano.tensor.signal import pool
from theano.tensor.nnet import conv2d
from pylearn2.expr.normalize import CrossChannelNormalization



class Fully_Connected_Dropout(object):
# http://christianherta.de/lehre/dataScience/machineLearning/neuralNetworks/Dropout.php

def __init__(self, rng, is_train, input, n_in, n_out, W=None, b=None, p=0.5):
self.input = input
# end-snippet-1

rng = np.random.RandomState(1234)
srng = T.shared_randomstreams.RandomStreams(rng.randint(99))

# for a discussion of the initialization, see
# https://plus.google.com/+EricBattenberg/posts/f3tPKjo7LFa
if W is None:
W_values = np.asarray(
rng.uniform(
low=-np.sqrt(6. / (n_in + n_out)),
high=np.sqrt(6. / (n_in + n_out)),
size=(n_in, n_out)
),
dtype=theano.config.floatX
)
W = theano.shared(value=W_values, name='W', borrow=True)

# init biases to positive values, so we should be initially in the linear regime of the linear rectified function
if b is None:
b_values = np.ones((n_out,), dtype=theano.config.floatX) * np.cast[theano.config.floatX](0.01)
b = theano.shared(value=b_values, name='b', borrow=True)

self.W = W
self.b = b

lin_output = T.dot(input, self.W) + self.b

output = theano.tensor.nnet.relu(lin_output)

# multiply output and drop -> in an approximation the scaling effects cancel out

input_drop = np.cast[theano.config.floatX](1. / p) * output

mask = srng.binomial(n=1, p=p, size=input_drop.shape, dtype=theano.config.floatX)
train_output = input_drop * mask

# is_train is a pseudo boolean theano variable for switching between training and prediction
self.output = T.switch(T.neq(is_train, 0), train_output, output)

# parameters of the model
self.params = [self.W, self.b]




class Fully_Connected_Softmax(object):
def __init__(self, rng, input, n_in, n_out, W=None, b=None,
 activation=theano.tensor.nnet.relu):

self.input = input



if W is None:
W_values = np.asarray(
rng.uniform(
low=-np.sqrt(6. / (n_in + n_out)),
high=np.sqrt(6. / (n_in + n_out)),
size=(n_in, n_out)
),
dtype=theano.config.floatX
)
if activation == theano.tensor.nnet.sigmoid:
W_values *= 4

W = theano.shared(value=W_values, name='W', borrow=True)

if b is None:
b_values = np.zeros((n_out,), dtype=theano.config.floatX)
b = theano.shared(value=b_values, name='b', borrow=True)

self.W = W
self.b = b

lin_output = T.nnet.softmax(T.dot(input, self.W) + self.b)

self.output = (
lin_output if activation is None
else activation(lin_output)
)


self.params = [self.W, self.b]






class LeNetConvPoolLRNLayer(object):
def __init__(self, rng, input, filter_shape, image_shape, poolsize=(2, 2), stride=(1, 1), lrn=False):
"""
Allocate a LeNetConvPoolLayer with shared variable internal parameters.

:type rng: numpy.random.RandomState
:param rng: a random number generator used to initialize weights

:type input: theano.tensor.dtensor4
:param input: symbolic image tensor, of shape image_shape

:type filter_shape: tuple or list of length 4
:param filter_shape: (number of filters, num input feature maps,
  filter height, filter width)

:type image_shape: tuple or list of length 4
:param image_shape: (batch size, num input feature maps,
 image height, image width)

:type poolsize: tuple or list of 

[theano-users] Scan: Return tuple of length 1?

[Peter O'Connor]

Hi all.

There's some really unintuitive/inconsistent behaviour with theano's scan 
and I'm wondering how I should work around it.

Take the following test, which I would expect to pass:

import theano
import theano.tensor as tt


def test_scan_tuple_output():

x = tt.matrix()

def mul_by_2_and_3(data):
return (data*2, data*3)

y, updates = theano.scan(mul_by_2_and_3, sequences=[x])

assert len(y)==2
assert len(updates)==0

def mul_by_2(data):
return (data*2, )

y, updates = theano.scan(mul_by_2, sequences = [x])

assert len(y)==1
assert len(updates) == 0


if __name__ == '__main__':
test_scan_tuple_output()

It fails on   
It 
It fails on

assert len(y)==1

with "TypeError: object of type 'TensorVariable' has no len()" because if 
scan detects a tuple of length-1, it just returns the data inside this 
tuple.

This becomes annoying when trying to make a scannable function with 
variable return length, (one of which can be one).

Is there a way I can make scan not totally change its behaviour when the 
tuple of returned arguments happens to have length 1? 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Scan: Return tuple of length 1?

[Peter O'Connor]

Hi all.

There's some really unintuitive/inconsistent behaviour with theano's scan 
and I'm wondering how I should work around it.

Take the following test, which I would expect to pass:

import theano
import theano.tensor as tt


def test_scan_tuple_output():

x = tt.matrix()

def mul_by_2_and_3(data):
return (data*2, data*3)

y, updates = theano.scan(mul_by_2_and_3, sequences=x)

assert len(y)==2
assert len(updates)==0

def mul_by_2(data):
return (data*2, )

y, updates = theano.scan(mul_by_2, sequences = [x])

assert len(y)==1
assert len(updates) == 0


if __name__ == '__main__':
test_scan_tuple_output()

It fails on   
It 
It fails on

assert len(y)==1

with "TypeError: object of type 'TensorVariable' has no len()" because if 
scan detects a tuple of length-1, it just returns the data inside this 
tuple.

This becomes annoying when trying to make a scannable function with 
variable return length, (one of which can be one).

Is there a way I can make scan not totally change its behaviour when the 
tuple of returned arguments happens to have length 1? 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] nvidia TX1 Embedded Board not support

[Frédéric Bastien]

Which version of Theano do you use? Can you confirm you use Theano
development version? 0.9.devN?

Fred

On Mon, Jan 16, 2017 at 3:11 AM, sang hyun Yoon  wrote:

> can i help..
> Thank you..
>
> WARNING (theano.gof.cmodule): OPTIMIZATION WARNING: Theano was not able to
> find the default g++ parameters. This is needed to tune the compilation to
> your specific CPU. This can slow down the execution of Theano functions.
> Please submit the following lines to Theano's mailing list so that we can
> fix this problem:
>  ['# 1 ""\n', '# 1 ""\n', '# 1 ""\n', '# 1
> "/usr/include/stdc-predef.h" 1 3 4\n', '# 1 "" 2\n', '# 1
> ""\n', 'Using built-in specs.\n', 'COLLECT_GCC=/usr/bin/g++\n',
> 'Target: aarch64-linux-gnu\n', "Configured with: ../src/configure -v
> --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4'
> --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs
> --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr
> --program-suffix=-5 --enable-shared --enable-linker-build-id
> --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
> --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
> --with-default-libstdcxx-abi=new --enable-gnu-unique-object
> --disable-libquadmath --enable-plugin --with-system-zlib
> --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo
> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre
> --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64
> --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64
> --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
> --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror
> --enable-checking=release --build=aarch64-linux-gnu
> --host=aarch64-linux-gnu --target=aarch64-linux-gnu\n", 'Thread model:
> posix\n', 'gcc version 5.4.0 20160609 (Ubuntu/Linaro
> 5.4.0-6ubuntu1~16.04.4) \n', "COLLECT_GCC_OPTIONS='-E' '-v'
> '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'\n", '
> /usr/lib/gcc/aarch64-linux-gnu/5/cc1 -E -quiet -v -imultiarch
> aarch64-linux-gnu - -mlittle-endian -mabi=lp64 -fstack-protector-strong
> -Wformat -Wformat-security\n', 'ignoring nonexistent directory
> "/usr/local/include/aarch64-linux-gnu"\n', 'ignoring nonexistent
> directory 
> "/usr/lib/gcc/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/include"\n',
> '#include "..." search starts here:\n', '#include <...> search starts
> here:\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include\n', '
> /usr/local/include\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include-fixed\n',
> ' /usr/include/aarch64-linux-gnu\n', ' /usr/include\n', 'End of search
> list.\n', 'COMPILER_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/
> gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/:/
> usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/\n',
> 'LIBRARY_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/
> gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/:/usr/
> lib/gcc/aarch64-linux-gnu/5/../../../../lib/:/lib/aarch64-
> linux-gnu/:/lib/../lib/:/usr/lib/aarch64-linux-gnu/:/usr/
> lib/../lib/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../:/lib/:/usr/lib/\n',
> "COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mlittle-endian'
> '-mabi=lp64'\n"]
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Mixture Density Networks

[qixiaoning1217]

I also need it for some experience. Can u post it to me?thanks. 

在 2014年4月22日星期二 UTC+8下午1:29:45，Sank写道：
>
> I need it. Can u post it?
>
> On Saturday, 19 April 2014 03:08:14 UTC+5:30, Sid wrote:
>>
>> Hi Fred, 
>>
>> I implemented it myself, I can post it here if anybody needs. 
>> Thanks :)
>>
>> -Sid
>>
>>
>> On Tue, Apr 15, 2014 at 3:52 PM, Frédéric Bastien  
>> wrote:
>>
>>> As you didn't got answer in the last 3 days, try to email pylearn 
>>> mailing list. Maybe you will have an answer there.
>>>
>>> Also, many people are at ICLR, so the response time can be slower.
>>>
>>> Fred
>>>
>>>
>>> On Sat, Apr 12, 2014 at 6:58 AM, Sid  wrote:
>>>
 Hello, 

 I wanted to run some experiments with Mixture Density Networks. I was 
 wondering if anyone here has tried implementing MDNs themselves or has an 
 implementation of GMMs using theano. I don't mind implementing it myself, 
 but it would be easier if pylearn2 or somebody else already has an 
 implementation of it. 

 Thanks :)
 -Sid

 -- 

 --- 
 You received this message because you are subscribed to the Google 
 Groups "theano-users" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to theano-users...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> -- 
>>>
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "theano-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to theano-users...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: How to: data parallelism in new GPU backend

[Adam Stooke]

Thinking of a different design:

1. Master python process builds and compiles all theano functions like 
normal (for GPU), and pickles them.
2. Worker processes initialize on other GPUs and unpickle all the functions.
3. User calls wrapped theano functions in master process, which signals to 
workers.
4. Workers run infinite loop, waiting for signal of what to do (some switch 
statement), e.g.:
a. call some function (can take inputs from multiprocessing shared 
variables) and communicate result
b. copy multiprocessing shared variables to update local theano GPU 
shared variables
c. do collective GPU comms.
d. etc.

The workers are "dumb" and never have to bother with any graphs.  It's a 
bit of a pain to set up the multiprocessing shared variables (have to 
declare data sizes ahead of time) but not so bad.  

What I'm running into trouble with now is the theano shared variables. 
 They get unpickled under the function's input_storage, but each function 
ends up with a separate set of objects here.  I can manipulate them 
individually, but *is there a way to get multiple unpickled functions to 
refer to the same memory for corresponding shared variables?*  (Simply 
setting the input_storage entries to another function's does not work.)

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.