Re: [theano-users] Re: Theano running very slow on amazon ec2 P2.xlarge machine

[Frédéric Bastien]

Try to use Theano profiling to know where the time is spent.

Le jeu. 11 mai 2017 15:48, Gavin Weiguang Ding  a
écrit :

> hi daksh,
>
> have you figured out the difference? i have the same issue here.
>
> Thanks,
> Gavin
>
>
> On Friday, 17 February 2017 02:37:00 UTC-5, Daksh Varshneya wrote:
>>
>> Hi,
>>
>> I have a piece of code which basically trains a GRU to learn from some
>> sequences. I am training the neural network on each individual
>> sequence(batch size = 1). When I am running the training code on my local
>> machine which has a Nvidia gtx 960m, it takes about 3.09 secs for ~ 10 data
>> points. But when I run the same code on aws ec2 p2.xlarge machine which has
>> a Nvidia Tesla K80 GPU, surprisingly the time taken for exactly same
>> sequences is around 4.93 seconds. This is very disheartening right now. I
>> have tried switching AMI's as well. Originally I used a clean install of
>> ubuntu 14.04 and installed cuda, cudnn and theano myself. The second image
>> I took from amazon marketplace which had all these things installed.
>> The code is exactly the same with no difference at all. Any pointers
>> which can help?
>>
>> Thanks,
>> Daksh
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] error when running theano with GPU：/usr/bin/ld: cannot find -lcudnn；Optimization failure due to: local_log_softmax_dnn

[Frédéric Bastien]

Update Theano to version 0.9 or the dev version. 0..8 is very old.

You can manually delete the cache. To know where it is just run

theano-cache

It will give you is location.

Le mer. 10 mai 2017 03:16,  a écrit :

> Hello:
>   I am trying to use theano with GPU on Ubuntu. I am getting some
> weird errors that I can't solve.using theano 0.8.2, python 2.7. cndnn v5,
> CUDA 8.0
>
> The code will print a long code and print the next two error:
> 1.
> /usr/bin/ld: cannot find -lcudnn
> collect2: error: ld returned 1 exit status
> I have export LD_LIBRARY_PATH in ~/.bashrc file. and there are the .so
> file in the path.
>
> 2.
> ERROR (theano.gof.opt): Optimization failure due to: local_log_softmax_dnn
> ERROR (theano.gof.opt): node:
> GpuElemwise{exp,no_inplace}()
> ERROR (theano.gof.opt): TRACEBACK:
> ERROR (theano.gof.opt): Traceback (most recent call last):
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py",
> line 1772, in process_node
> replacements = lopt.transform(node)
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/dnn.py",
> line 2529, in local_log_softmax_dnn
> if not dnn_available() or version() < (3000, 3000):
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/__init__.py",
> line 412, in dnn_version
> profile=False)
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/compile/function.py",
> line 320, in function
> output_keys=output_keys)
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py", line
> 479, in pfunc
> output_keys=output_keys)
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/compile/function_module.py",
> line 1777, in orig_function
> defaults)
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/compile/function_module.py",
> line 1641, in create
> input_storage=input_storage_lists, storage_map=storage_map)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/link.py",
> line 690, in make_thunk
> storage_map=storage_map)[:3]
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/vm.py", line
> 1003, in make_all
> no_recycling))
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/__init__.py",
> line 256, in make_thunk
> compute_map, no_recycling)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/op.py", line
> 970, in make_thunk
> no_recycling)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/op.py", line
> 879, in make_c_thunk
> output_storage=node_output_storage)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/cc.py", line
> 1200, in make_thunk
> keep_lock=keep_lock)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/cc.py", line
> 1143, in __compile__
> keep_lock=keep_lock)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/cc.py", line
> 1595, in cthunk_factory
> key=key, lnk=self, keep_lock=keep_lock)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/cmodule.py",
> line 1142, in module_from_key
> module = lnk.compile_cmodule(location)
>   File "/opt/anaconda2/lib/python2.7/site-packages/theano/gof/cc.py", line
> 1506, in compile_cmodule
> preargs=preargs)
>   File
> "/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/nvcc_compiler.py",
> line 399, in compile_str
> 'for cmd', ' '.join(cmd))
> Exception: ('The following error happened while compiling the node',
> (), '\n', 'nvcc
> return status', 1, 'for cmd', 'nvcc -shared -O3 -Xlinker
> -rpath,/usr/local/cuda-8.0/lib64 -arch=sm_61 -m64 -Xcompiler
> -fno-math-errno,-Wno-unused-label,-Wno-unused-variable,-Wno-write-strings,-DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden
> -Xlinker
> -rpath,/home/yjf/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-stretch-sid-x86_64-2.7.13-64/cuda_ndarray
> -I/usr/local/cuda-8.0/include
> -I/opt/anaconda2/lib/python2.7/site-packages/numpy/core/include
> -I/opt/anaconda2/include/python2.7
> -I/opt/anaconda2/lib/python2.7/site-packages/theano/gof
> -I/opt/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda -o
> /home/yjf/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-stretch-sid-x86_64-2.7.13-64/tmpQtQkqO/265abc51f7c376c224983485238ff1a5.so
> mod.cu -L/opt/anaconda2/lib -lcudnn -lpython2.7 -lcudart',
> '[()]')
>
> I see someone said that theano cache is corrupt,delete it.But I cannot run
>  theano-cache clear successfully , the two errors above continue appear.
> Thanks in advance. Looking forward to your reply.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because y

[theano-users] Re: Theano running very slow on amazon ec2 P2.xlarge machine

[Gavin Weiguang Ding]

hi daksh,

have you figured out the difference? i have the same issue here.

Thanks,
Gavin

On Friday, 17 February 2017 02:37:00 UTC-5, Daksh Varshneya wrote:
>
> Hi,
>
> I have a piece of code which basically trains a GRU to learn from some 
> sequences. I am training the neural network on each individual 
> sequence(batch size = 1). When I am running the training code on my local 
> machine which has a Nvidia gtx 960m, it takes about 3.09 secs for ~ 10 data 
> points. But when I run the same code on aws ec2 p2.xlarge machine which has 
> a Nvidia Tesla K80 GPU, surprisingly the time taken for exactly same 
> sequences is around 4.93 seconds. This is very disheartening right now. I 
> have tried switching AMI's as well. Originally I used a clean install of 
> ubuntu 14.04 and installed cuda, cudnn and theano myself. The second image 
> I took from amazon marketplace which had all these things installed. 
> The code is exactly the same with no difference at all. Any pointers which 
> can help?
>
> Thanks,
> Daksh
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Why thano does not update?

[Giuseppe Angora]

Thanks Pascal your suggestion works greatly!!

Il giorno giovedì 11 maggio 2017 17:40:53 UTC+2, Pascal Lamblin ha scritto:
>
> When you do: 
>
> self.W -= self.rate * gz 
>
> You are not updating the value of the shared variable that was in 
> self.W. You are creating a new symbolic variable representing 
> "self.W - self.rate * gz", and re-assigning the member of Test to 
> point to that new variable. 
>
> Instead, you want to pass "updates" to theano.function. 
>
> For instance: 
>   def start(self, x, y): 
> new_W = self.W 
> for x in range(5): 
>   z = ... 
>   gz = ... 
>   new_W -= self.rate * gz 
> return z, (self.W, new_W) 
>
> ... 
> cost, update = test.start(x, y) 
> ... 
> train = theano.function( 
>   ..., 
>   updates=[update]) 
>
>
> On Thu, May 11, 2017, Giuseppe Angora wrote: 
> > Hi, 
> > I'm tying to resolve the follow problem: a theano function has as 
> outputs 
> > the value that a class method return after has made a while loop, within 
> > which a parameter is updated: 
> > 
> > import theanoimport theano.tensor as Timport numpy as npimport copy 
> > theano.config.exception_verbosity = 'high' 
> > class Test(object): 
> > def __init__(self): 
> > self.rate=0.01 
> > W_val=40.00 
> > self.W=theano.shared(value=W_val, borrow=True) 
> > def start(self, x, y): 
> > for i in range(5): 
> > z=T.mean(x*self.W/y) 
> > gz=T.grad(z, self.W) 
> > self.W-=self.rate*gz 
> > return z 
> > 
> > x_set=np.array([1.,2.,1.,2.,1.,2.,1.,2.,1.,2.]) 
> > y_set=np.array([1,2,1,2,1,2,1,2,1,2]) 
> > x_set = theano.shared(x_set, borrow=True) 
> > y_set = theano.shared(y_set, borrow=True) 
> > y_set=T.cast(y_set, 'int32') 
> > batch_size=2 
> > 
> > x = T.dvector('x') 
> > y = T.ivector('y') 
> > index = T.lscalar() 
> > 
> > test = Test() 
> > cost=test.start(x,y) 
> > 
> > train = theano.function( 
> > inputs=[index], 
> > outputs=cost, 
> > givens={ 
> > x: x_set[index * batch_size: (index + 1) * batch_size], 
> > y: y_set[index * batch_size: (index + 1) * batch_size] 
> > }) 
> > for i in range(5): 
> > result=train(i) 
> > print(result) 
> > 
> > this is the result of the print: 
> > 
> > 
> 39.960008940739.960008940739.960008940739.960008940739.9600089407
>  
>
> > 
> > Now the gradient of mean(x*W/y) is equal to 1 (because x and y always 
> have the same value). So the first time i should have 39.95, than 39.90 and 
> so on... Why i always have the same result?? 
> > 
> > Thanks 
> > 
> > -- 
> > 
> > --- 
> > You received this message because you are subscribed to the Google 
> Groups "theano-users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to theano-users...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
> -- 
> Pascal 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Theano: MissingInputError

[Pascal Lamblin]

The error is raised by the line that says:
fz=theano.function([], z, updates=updates)

because inputs are missing there.

Moreover, even if you provided inputs, the "out" returned would just be
a numpy array, that would be treated like a constant by the rest of the
graph.

The thing you want to do is keeping the updates symbolic until you pass
then to "train", not try to apply them through another theano functin.

On Tue, May 09, 2017, Giuseppe Angora wrote:
> Hi,
> 
> this is a semplification of the original code that i'm trying to fix. I 
> have invented a problem that reflects what the original code does and which 
> gives me the same error but is written in a minimal form:
> 
> import theanoimport theano.tensor as Timport numpy as npimport copy
> theano.config.exception_verbosity = 'high'
> 
> class Test(object):
> def __init__(self):
> self.rate=0.01
> def start(self, x, y):
> W_val=40.89
> W=theano.shared(value=W_val, borrow=True)
> z=T.mean(x*W/y)
> gz=T.grad(z, W)
> updates=[(W, W-self.rate*gz)]
> 
> fz=theano.function([], z, updates=updates)
> for i in range(100):
> out=fz()
> self.out=out
> return out
> 
>  x_set=np.random.rand(10)
>  y_set=np.random.randint(low=0, high=5, size=10, dtype=int)
>  batch_size=2
> 
>  x = T.dvector('x')
>  y = T.ivector('y')
>  index = T.lscalar()
> 
> test = Test()
> cost=test.start(x,y)
> 
> train = theano.function(
> inputs=[index],
> outputs=cost,
> givens={
> x: x_set[index * batch_size: (index + 1) * batch_size],
> y: y_set[index * batch_size: (index + 1) * batch_size]
>})
> for i in range(5):
> result=train(i)
> print(result)
> 
> 
> This is the TraceBack:
> 
> Traceback (most recent call last):
>   File "functions.py", line 33, in 
> cost=test.start(x,y)
>   File "functions.py", line 18, in start
> fz=theano.function([], z, updates=updates)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function.py", line 320, in 
> function
> output_keys=output_keys)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\pfunc.py", line 479, in pfunc
> output_keys=output_keys)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 
> 1776, in orig_function
> output_keys=output_keys).create(
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 
> 1428, in __init__
> accept_inplace)
>   File "C:\Program 
> Files\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 
> 177, in std_fgraph
> update_mapping=update_mapping)
>   File "C:\Program Files\Anaconda3\lib\site-packages\theano\gof\fg.py", line 
> 171, in __init__
> self.__import_r__(output, reason="init")
>   File "C:\Program Files\Anaconda3\lib\site-packages\theano\gof\fg.py", line 
> 360, in __import_r__
> self.__import__(variable.owner, reason=reason)
>   File "C:\Program Files\Anaconda3\lib\site-packages\theano\gof\fg.py", line 
> 465, in __import__
> detailed_err_msg)
> theano.gof.fg.MissingInputError: A variable that is an input to the graph was 
> neither provided as an input to the function nor given a value. A chain of 
> variables leading from this input to an output is [x, 
> Elemwise{mul,no_inplace}.0, Elemwise{true_div,no_inplace}.0, 
> Sum{acc_dtype=float64}.0, mean]. This chain may not be uniqueBacktrace when 
> the variable is created:
>   File "functions.py", line 28, in 
> x = T.dvector('x')
> 
> I do not know where to slam my head again... 
> 
> Thanks
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Why thano does not update?

[Pascal Lamblin]

When you do:

self.W -= self.rate * gz

You are not updating the value of the shared variable that was in
self.W. You are creating a new symbolic variable representing
"self.W - self.rate * gz", and re-assigning the member of Test to
point to that new variable.

Instead, you want to pass "updates" to theano.function.

For instance:
  def start(self, x, y):
new_W = self.W
for x in range(5):
  z = ...
  gz = ...
  new_W -= self.rate * gz
return z, (self.W, new_W)

...
cost, update = test.start(x, y)
...
train = theano.function(
  ...,
  updates=[update])


On Thu, May 11, 2017, Giuseppe Angora wrote:
> Hi, 
> I'm tying to resolve the follow problem: a theano function has as outputs 
> the value that a class method return after has made a while loop, within 
> which a parameter is updated:
> 
> import theanoimport theano.tensor as Timport numpy as npimport copy
> theano.config.exception_verbosity = 'high'
> class Test(object):
> def __init__(self):
> self.rate=0.01
> W_val=40.00
> self.W=theano.shared(value=W_val, borrow=True)
> def start(self, x, y):
> for i in range(5):
> z=T.mean(x*self.W/y)
> gz=T.grad(z, self.W)
> self.W-=self.rate*gz
> return z
> 
> x_set=np.array([1.,2.,1.,2.,1.,2.,1.,2.,1.,2.])
> y_set=np.array([1,2,1,2,1,2,1,2,1,2])
> x_set = theano.shared(x_set, borrow=True)
> y_set = theano.shared(y_set, borrow=True)
> y_set=T.cast(y_set, 'int32')
> batch_size=2
> 
> x = T.dvector('x')
> y = T.ivector('y')
> index = T.lscalar()
> 
> test = Test()
> cost=test.start(x,y)
> 
> train = theano.function(
> inputs=[index],
> outputs=cost,
> givens={
> x: x_set[index * batch_size: (index + 1) * batch_size],
> y: y_set[index * batch_size: (index + 1) * batch_size]
> })
> for i in range(5):
> result=train(i)
> print(result)
> 
> this is the result of the print:
> 
> 39.960008940739.960008940739.960008940739.960008940739.9600089407
> 
> Now the gradient of mean(x*W/y) is equal to 1 (because x and y always have 
> the same value). So the first time i should have 39.95, than 39.90 and so 
> on... Why i always have the same result??
> 
> Thanks
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to theano-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Pascal

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Why thano does not update?

[Giuseppe Angora]

@Alezander Botev Thank for the the answer, the probelm is that if i use the 
theano.function, adding updates, i have an MissignInputError. Working with 
theano function was my first option, as yuo can see from my previous post 
.
So you tell me that is impossible to update in this way? 

Il giorno giovedì 11 maggio 2017 14:28:46 UTC+2, Alexander Botev ha scritto:
>
> That's because Theano shared variables are symbolic and still do not 
> update in the usual way as in python code. You will need to use the 
> `updates` arguments when creating the theano function. I suggest read more 
> in the intro here: 
> http://deeplearning.net/software/theano/tutorial/examples.html
>
> On Thursday, 11 May 2017 12:40:13 UTC+1, Giuseppe Angora wrote:
>>
>> Hi, 
>> I'm tying to resolve the follow problem: a theano function has as outputs 
>> the value that a class method return after has made a while loop, within 
>> which a parameter is updated:
>>
>> import theanoimport theano.tensor as Timport numpy as npimport copy
>> theano.config.exception_verbosity = 'high'
>> class Test(object):
>> def __init__(self):
>> self.rate=0.01
>> W_val=40.00
>> self.W=theano.shared(value=W_val, borrow=True)
>> def start(self, x, y):
>> for i in range(5):
>> z=T.mean(x*self.W/y)
>> gz=T.grad(z, self.W)
>> self.W-=self.rate*gz
>> return z
>>
>> x_set=np.array([1.,2.,1.,2.,1.,2.,1.,2.,1.,2.])
>> y_set=np.array([1,2,1,2,1,2,1,2,1,2])
>> x_set = theano.shared(x_set, borrow=True)
>> y_set = theano.shared(y_set, borrow=True)
>> y_set=T.cast(y_set, 'int32')
>> batch_size=2
>>
>> x = T.dvector('x')
>> y = T.ivector('y')
>> index = T.lscalar()
>>
>> test = Test()
>> cost=test.start(x,y)
>>
>> train = theano.function(
>> inputs=[index],
>> outputs=cost,
>> givens={
>> x: x_set[index * batch_size: (index + 1) * batch_size],
>> y: y_set[index * batch_size: (index + 1) * batch_size]
>> })
>> for i in range(5):
>> result=train(i)
>> print(result)
>>
>> this is the result of the print:
>>
>> 39.960008940739.960008940739.960008940739.960008940739.9600089407
>>
>> Now the gradient of mean(x*W/y) is equal to 1 (because x and y always have 
>> the same value). So the first time i should have 39.95, than 39.90 and so 
>> on... Why i always have the same result??
>>
>> Thanks
>>
>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Why thano does not update?

[Alexander Botev]

That's because Theano shared variables are symbolic and still do not update 
in the usual way as in python code. You will need to use the `updates` 
arguments when creating the theano function. I suggest read more in the 
intro here: http://deeplearning.net/software/theano/tutorial/examples.html

On Thursday, 11 May 2017 12:40:13 UTC+1, Giuseppe Angora wrote:
>
> Hi, 
> I'm tying to resolve the follow problem: a theano function has as outputs 
> the value that a class method return after has made a while loop, within 
> which a parameter is updated:
>
> import theanoimport theano.tensor as Timport numpy as npimport copy
> theano.config.exception_verbosity = 'high'
> class Test(object):
> def __init__(self):
> self.rate=0.01
> W_val=40.00
> self.W=theano.shared(value=W_val, borrow=True)
> def start(self, x, y):
> for i in range(5):
> z=T.mean(x*self.W/y)
> gz=T.grad(z, self.W)
> self.W-=self.rate*gz
> return z
>
> x_set=np.array([1.,2.,1.,2.,1.,2.,1.,2.,1.,2.])
> y_set=np.array([1,2,1,2,1,2,1,2,1,2])
> x_set = theano.shared(x_set, borrow=True)
> y_set = theano.shared(y_set, borrow=True)
> y_set=T.cast(y_set, 'int32')
> batch_size=2
>
> x = T.dvector('x')
> y = T.ivector('y')
> index = T.lscalar()
>
> test = Test()
> cost=test.start(x,y)
>
> train = theano.function(
> inputs=[index],
> outputs=cost,
> givens={
> x: x_set[index * batch_size: (index + 1) * batch_size],
> y: y_set[index * batch_size: (index + 1) * batch_size]
> })
> for i in range(5):
> result=train(i)
> print(result)
>
> this is the result of the print:
>
> 39.960008940739.960008940739.960008940739.960008940739.9600089407
>
> Now the gradient of mean(x*W/y) is equal to 1 (because x and y always have 
> the same value). So the first time i should have 39.95, than 39.90 and so 
> on... Why i always have the same result??
>
> Thanks
>
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Why thano does not update?

[Giuseppe Angora]

Hi, 
I'm tying to resolve the follow problem: a theano function has as outputs 
the value that a class method return after has made a while loop, within 
which a parameter is updated:

import theanoimport theano.tensor as Timport numpy as npimport copy
theano.config.exception_verbosity = 'high'
class Test(object):
def __init__(self):
self.rate=0.01
W_val=40.00
self.W=theano.shared(value=W_val, borrow=True)
def start(self, x, y):
for i in range(5):
z=T.mean(x*self.W/y)
gz=T.grad(z, self.W)
self.W-=self.rate*gz
return z

x_set=np.array([1.,2.,1.,2.,1.,2.,1.,2.,1.,2.])
y_set=np.array([1,2,1,2,1,2,1,2,1,2])
x_set = theano.shared(x_set, borrow=True)
y_set = theano.shared(y_set, borrow=True)
y_set=T.cast(y_set, 'int32')
batch_size=2

x = T.dvector('x')
y = T.ivector('y')
index = T.lscalar()

test = Test()
cost=test.start(x,y)

train = theano.function(
inputs=[index],
outputs=cost,
givens={
x: x_set[index * batch_size: (index + 1) * batch_size],
y: y_set[index * batch_size: (index + 1) * batch_size]
})
for i in range(5):
result=train(i)
print(result)

this is the result of the print:

39.960008940739.960008940739.960008940739.960008940739.9600089407

Now the gradient of mean(x*W/y) is equal to 1 (because x and y always have the 
same value). So the first time i should have 39.95, than 39.90 and so on... Why 
i always have the same result??

Thanks

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Re: Variable number of arguments in scan

[Gilles Degottex]

It seems it does!

Cheers,

On Thursday, 11 May 2017 00:31:48 UTC+1, Pascal Lamblin wrote:
>
> On Fri, May 05, 2017, Gilles Degottex wrote: 
> > Sorry, slight confusion, by "chose the order arbitrarily" I didn't mean 
> on 
> > the fly. I meant be able to selection what ever order, but once for all 
> > once running. 
> > I'm not actually changing anything once compiled. 
>
> Then it should work fine. 
>
> > 
> > 
> > 
> > On Friday, 5 May 2017 02:56:03 UTC+1, Adam Becker wrote: 
> > > 
> > > I don't think this works. The inner function of scan will be converted 
> to 
> > > a graph, then getting compiled inside ScanOp. If you change nonlocal 
> > > variable "order" on the fly, the change won't be reflected on the 
> compiled 
> > > function. 
> > > 
> > > If the inner loop itself can be written as scan, you can just make a 
> > > nested scan instead. Compile the innermost graph with scan first (by 
> hand), 
> > > then pass it as fn to the outer scan. 
> > > 
> > > On Wednesday, May 3, 2017 at 8:04:46 PM UTC+8, Gilles Degottex wrote: 
> > >> 
> > >> Hi, 
> > >> 
> > >> I'm new on this list, so I hope I'm not duplicating a subject that 
> has 
> > >> already been covered and answered. 
> > >> 
> > >> I'm trying to implement High-Order RNN (
> https://arxiv.org/abs/1605.00064) 
> > >> and I would like to let me chose the order arbitrarily, which changes 
> the 
> > >> number parameters to pass to the inner scan function. 
> > >> For this purpose, I'm doing something like: 
> > >> 
> > >> def step(in_t, *args): 
> > >> 
> > >> W_xi_ = args[-2] 
> > >> b_i_ = args[-1] 
> > >> args = args[:-2] 
> > >> 
> > >> h_t = T.dot(in_t, W_xi_) 
> > >> 
> > >> for p in xrange(order): 
> > >> h_t += T.dot(args[p], args[order+p]) 
> > >> 
> > >> h_t = nonlinearity(h_t + b_i_) 
> > >> 
> > >> return h_t 
> > >> 
> > >> h, _ = theano.scan(step, sequences=[invalues], 
> > >> outputs_info=[dict(initial=hid_init, 
> taps=range(-order,0))], 
> > >> non_sequences=W_hip+[W_xi, b_i], strict=True) 
> > >> 
> > >> W_hip being a list of shared matrices, one for each tap. 
> > >> 
> > >> Basically, it compiles, I can train such a model, so it looks like it 
> > >> works. 
> > >> 
> > >> However, I've strictly no clue if the code is doing what I'm 
> expecting it 
> > >> to do. 
> > >> Is is okay to use a for loop in th inner scan function? 
> > >> Does scan behave nicely with a variable number of argument? 
> > >> 
> > >> Any tips to help verifying/checking that a code is doing what it is 
> > >> supposed to do is also very welcome. 
> > >> 
> > >> Cheers, 
> > >> 
> > >> 
> > >> 
> > 
> > -- 
> > 
> > --- 
> > You received this message because you are subscribed to the Google 
> Groups "theano-users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to theano-users...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
> -- 
> Pascal 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [theano-users] Error "pygpu.gpuarray.GpuArrayException: invalid resource handle" when using multiple GPUs with new backend

[T Qri]

Thank you.

On Thursday, May 11, 2017 at 9:22:48 AM UTC+8, Pascal Lamblin wrote:
>
> Unfortunately, model parallelism using different GPUs in the same 
> process is still experimental, and not a current focus for development. 
>
>
> On Wed, May 10, 2017, T Qri wrote: 
> > Hi everyone, 
> > 
> > In brief, I'm implementing a model with 2 seq2seq components which share 
> an 
> > encoder. Each of the 2 components are distributed across different GPU 
> > devices and the shared encoder is also placed on one of these 2 devices. 
> > 
> > When config.optimizer is set to None, the error doesn't exist anymore 
> but 
> > the model runs on multiple CPUs quite slowly. 
> > 
> > The detailed Theano flags are described in `theano-flags` file. And the 
> > corresponding error message is in `error-message` file. 
> > 
> > I tried to disable constant_folding and elemwise optimization with 
> > `optimizer_excluding=constant_folding elemwise` set but it didn't work. 
> > 
> > I'm using  Theano 0.9.0 version on Ubuntu 14.04. There are 4 GPUs 
> available 
> > and 2 of them are involved in my model. 
> > 
> > Thanks for your help. 
> > 
> > 
> > -- 
> > 
> > --- 
> > You received this message because you are subscribed to the Google 
> Groups "theano-users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to theano-users...@googlegroups.com . 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
>
>
> -- 
> Pascal 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.