Dear John,

Sorry to have disturb you with that problem actually it was linked to a local fact. We had in the past a problem with the api authentification so as a work around, we have 2 galaxy servers. one for galaxy and another for the api. Actually it is quite new (a few weeks), so i was not very familiar with it.

The problem was because i only reboot galaxy and not galaxy api... so my changes were not taken into account. So your patch now works !! Thank you again, we can go on with our project using Bioblend.

Have a nice day !
-
Olivia

Le 10/01/14 15:35, John Chilton a écrit :
I am stumped, I am not sure how this would happen. How did you apply
the patch against the -dist stable code? Can you send me the full
contents of the patched ..api/library_contents.py file?

-John





On Tue, Jan 7, 2014 at 9:48 AM, Olivia Doppelt
<olivia.dopp...@pasteur.fr> wrote:
Dear John,

Unfortunatly, I have another problem.

The galaxy version in use for our institute was last updated on December
10th.

Weirdly, when i put the patch in the library_contents.py file, I get another
error (which i don't have on my local version)

Actually, it never goes in the code i patched... and it doesn't underdand
the string which is the role.

the error in the following:
_____

157.99.60.187 - - [07/Jan/2014:16:36:31 +0200] "GET
/api/libraries/79449a84684b4136/contents?key=API_key HTTP/1.1" 200 - "-"
"python-requests/2.0.1 CPython/2.7.5+ Linux/3.11.0-12-generic"
sqlalchemy.pool.QueuePool WARNING 2014-01-07 16:36:32,212 Error closing
cursor: current transaction is aborted, commands ignored until end of
transaction block

galaxy.web.framework ERROR 2014-01-07 16:36:32,212 Uncaught exception in
exposed API method:

Traceback (most recent call last):
   File "/xxxx/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py", line
197, in decorator
     rval = func( self, trans, *args, **kwargs)
   File
"/xxxx/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/library_contents.py",
line 183, in create

     role = str( trans.security.decode_id( role ) )
   File
"/xxxx/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/library_common.py",
line 843, in upload_library_dataset

     trans.app.security_agent.derive_roles_from_access( trans, library.id,
cntrller, library=True, **vars )
   File "/xxxx/galaxy/galaxy-dist/lib/galaxy/security/__init__.py", line 939,
in derive_roles_from_access
     in_roles = [ self.sa_session.query( self.model.Role ).get( x ) for x in
listify( kwd.get( k + '_in', [] ) ) ]
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py",
line 775, in get
     return self._load_on_ident(key)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py",
line 2512, in _load_on_ident
     return q.one()
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py",
line 2184, in one
     ret = list(self)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py",
line 2227, in __iter__
     return self._execute_and_instances(context)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py",
line 2242, in _execute_and_instances
     result = conn.execute(querycontext.statement, self._params)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
line 1449, in execute
     params)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
line 1584, in _execute_clauseelement
     compiled_sql, distilled_params
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
line 1698, in _execute_context
     context)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py",
line 1691, in _execute_context
     context)
   File
"/xxxx/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py",
line 331, in do_execute
     cursor.execute(statement, parameters)
DataError: (DataError) invalid input syntax for integer:
"odoppelt__at__pasteur.fr"
LINE 3: WHERE role.id = E'odoppelt__at__pasteur.fr'
                         ^
  'SELECT role.id AS role_id, role.create_time AS role_create_time,
role.update_time AS role_update_time, role.name AS role_name,
role.description AS role_description, role.type AS role_type, role.deleted
AS role_deleted \nFROM role \nWHERE role.id = %(param_1)s' {'param_1':
'odoppelt__at__pasteur.fr'}




_____


Le 06/01/14 18:34, Olivia Doppelt a écrit :

Hello John,

Thank you very much for the patch. It works now with the argument 'roles'.

I tested with the wrong role in order to be sure that it filters when the
role is not the right one and it does get filtered :)

However, I need to add a few lines to the patch in order to avoid the use of
this method without a defined role.

basically, the complete patch is:


+        roles = payload.get("roles", None)
+        if roles:
+            roles = util.listify(roles)
+
+            def to_id_as_str(role):
+                role = role.replace("__at__", "@")
+                if "@" in role:
+                    role = str( trans.sa_session.query(
trans.app.model.Role ).filter( trans.app.model.Role.table.c.name == role
).first().id )
+                else:
+                    role = str( trans.security.decode_id( role ) )
+                return role
+
+            payload["roles"] = map( to_id_as_str, roles )
+
+        else:
+            trans.response.status = 400
+            return "Missing required roles definition"



I don't know if you (could/want to) add this to your commit.

Thank you very much for your help.
--
Olivia

Le 06/01/14 16:53, John Chilton a écrit :

Hello Olivia,

Great, thanks for the clarification! It took me a little bit, but I
have caught up. First - you don't need to justify using bioblend over
the provided scripts in galaxy distribution. Anything non-trivial
should be using bioblend (if it weren't going to make me very
unpopular I would just delete scripts/api out of Galaxy).

There are a couple problems here - but the main one is simply that the
roles param is taking in something odd as an argument - namely the
unencoded integer id of the role as a string - it probably makes
perfect sense for the UI to deal with things this way but it doesn't
fit well with the API. Here
(https://github.com/jmchilton/galaxy-central/commit/11377929a679fb3391481e35227bcccd7e96a497.patch)
is a patch you can apply to Galaxy to make it work the way you are
trying to get it to work (allowing roles to take in a single role
name). Just apply that to Galaxy and I imagine your example will work
as is.

However, this is not "the right" way for it to work however so I am
not going to apply it Galaxy directly but it is a very isolated change
you can apply to your Galaxy instance if your comfortable with that.
In an ideal implementation - there are many different permission types
that should be exposed and the API should be consuming encoded role
ids (as obtained from /api/roles) - but the backend isn't implemented
for this first part and bioblend doesn't have a roles client for the
second.

I have created a Trello card for this feature:
https://trello.com/c/qsQHnFlG.

Let me know if you have any additional questions or if this is unclear
in any way.

-John

On Sun, Jan 5, 2014 at 3:35 AM, Olivia Doppelt
<olivia.dopp...@pasteur.fr> wrote:

Hello again,
I just reminded a capital thing concerning the api script
library_upload_from_import_dir.py. Actually it only works for admin users.
simple users don't have the authorization to upload anything in their galaxy
library using this script. It is the main reason why we choose to user
bioblend.
--
Olivia


Le 04/01/14 22:10, Olivia Doppelt a écrit :


Le 03/01/14 15:44, John Chilton a écrit :

I am not sure I understand the problem. Can you clarify a few points:

Is bioblend letting users associate roles with data libraries that
they shouldn't be able to and the operation is succeeding -or- are you
trying to add roles and the operation is failing?

Dear John,

In our Galaxy instance, each user connecting to Galaxy has a library
associated to its user role which is created automatically as well as two
directories; one in the links directory corresponding to our
"user_library_import_dir" and one in an "output" directory to enable the
users to export their results from the instance.

The user directories are readable by Galaxy and by themselves (unix rights
are set using a root cron script)

For a project, we need to do basic galaxy stuff using either the API scripts
or bioblend api. We have tried both and bioblend is a little simpler to use.

Using the api script library_upload_from_import_dir.py, using a user api
key, only files from the user directory are "uploadable".

However, using bioblend function from libraries class
"upload_file_from_server", a user with its key can upload within its galaxy
library, the data of anyone.

#giOlivia.libraries.upload_file_from_server("OLIVIA_API_key","/xxxxxxx/links/otheru...@pasteur.fr/","F0045a9ffd77d1226")


To disable this, i'm trying to add an associate role when I launch the
upload_file_from_server command but I get an error.

giOlivia.libraries.upload_file_from_server("OLIVIA_API_key","/xxxxxxxx/links/oli...@pasteur.fr/",
"F0045a9ffd77d1226", roles='oli...@pasteur.fr')

and I get the following error:

Traceback (most recent call last):   File "<input>", line 1, in <module>
File
"/usr/local/lib/python2.7/dist-packages/bioblend-0.4.2_dev-py2.7.egg/bioblend/galaxy/libraries/__init__.py",
line 242, in upload_file_from_server     return self._do_upload(**vars)
File
"/usr/local/lib/python2.7/dist-packages/bioblend-0.4.2_dev-py2.7.egg/bioblend/galaxy/libraries/__init__.py",
line 190, in _do_upload     files_attached=files_attached)   File
"/usr/local/lib/python2.7/dist-packages/bioblend-0.4.2_dev-py2.7.egg/bioblend/galaxy/client.py",
line 71, in _post     r = self.gi.make_post_request(url, payload=payload,
files_attached=files_attached)   File
"/usr/local/lib/python2.7/dist-packages/bioblend-0.4.2_dev-py2.7.egg/bioblend/galaxyclient.py",
line 94, in make_post_request     r.status_code, body=r.text)     # @see
self.body for HTTP response body ConnectionError: Unexpected response from
galaxy: 500: <html>   <head><title>Internal Server Error</title></head>
<body>     <h1>Internal Server Error</h1>     <p>The server has either erred
or is incapable of performing the requested operation.
<br/> <!--  --></p>     <hr noshade>     <div align="right">WSGI
Server</div>   </body> </html>


If the API is producing an error message can you share that: in
particular this code is probably resulting in an error or some kind?

             elif roles:
                 # Check to see if the user selected roles to associate
with the DATASET_ACCESS permission
                 # on the dataset that would cause accessibility issues.
                 vars = dict( DATASET_ACCESS_in=roles )
                 permissions, in_roles, error, message = \
                     trans.app.security_agent.derive_roles_from_access(
trans, library.id, cntrller, library=True, **vars )
             if error:
                 if cntrller == 'api':
                     return 400, message


the api does not produce any error when the api key corresponds to the user
which tries to import its own data.


Has your institute made modifications Galaxy to enforce this
constraint - ("it can't be possible that any user may upload any other
user's data in their Galaxy Data library")? Also is the thing you are
trying to do something that is working through the web ui but not
through the API, or have you only tried through the API?


I did a few tests yesterday, and to resume, it works as it should using the
api script but there is a problem using bioblend.
I'm not sure that roles are really needed here, i only thought that it could
guide galaxy (and bioblend) to not upload data that does not belong to user
(corresponding to the API key)

I looked at the sources, and my major doubt was that bioblend didn't pass
all the arguments but it does, everything is transmitted to galaxy do it
should work.

Tell me if you need more information. I'll look further on monday but i'm
kind of stuck.

Thank you for your help,

Cheers,

--
Olivia Doppelt-Azeroual, PhD
CIB - Institut Pasteur



On Fri, Jan 3, 2014 at 5:36 AM, Olivia Doppelt
<olivia.dopp...@pasteur.fr> wrote:

Hello to all,

I'm trying to use the function  "upload_file_from_server" from the galaxy
libraries api of Bioblend.

It's working correctly when i don't associate any role to the arguments.
However, in our architecture (Institut Pasteur, Paris) it can't be possible
that any user may upload any other user's data in their Galaxy Data library.

Thus, it is how ""upload_file_from_server" is working at the moment.

As I understand the problem, it doesn't come from Bioblend itself, as all
arguments (as well as roles) are well transmitted on the server side.

could somebody help me with that ?

Happy new Year !!! and Have a nice day,

--
Olivia Doppelt-Azeroual, PhD
Tel: 92 15
CIB - Institut Pasteur


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/



--
Olivia Doppelt-Azeroual, PhD
Tel: 92 15
CIB - Institut Pasteur



--
Olivia Doppelt-Azeroual, PhD
Tel: 92 15
CIB - Institut Pasteur



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/



--
Olivia Doppelt-Azeroual, PhD
Tel: 92 15
CIB - Institut Pasteur


--
Olivia Doppelt-Azeroual, PhD
Tel: 92 15
CIB - Institut Pasteur

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/

Reply via email to