During IT housekeeping, we have decided that we must know the filenames
of all of the galaxy datasets in our local deployment. Searching the docs
the web yielded no reasonable answer, so we figured we'll just write our
own tool.
At first, we thought about a Galaxy tool, but found no way to extract all
datasets of all histories. So we went deeper. To the SQL itself.

Without further ado, I give you the query:

    SELECT substring(to_char(history_dataset_association.dataset_id,
'000000') from 2 for 3) ||  '/dataset_' ||
history_dataset_association.dataset_id || '.dat' ,
history_dataset_association.create_time, history_dataset_association.name,
history_dataset_association.extension, history.name AS
    FROM history_dataset_association
    JOIN history ON history_dataset_association.history_id = history.id
    JOIN galaxy_user ON history.user_id = galaxy_user.id
    WHERE history_dataset_association.deleted = false AND
history_dataset_association.purged = false
    ORDER BY  username, history_name, create_time;

We use PostgreSQL, so to run this you save the query as
`get_filenames.sql`, and run:

    psql -U galaxy galaxy_prod -f get_filenames.sql -A >

This prints out a line structured as:
003/dataset_3748.dat|2012-10-09 12:07:37.786751| index.fastq|fastq|nan|sean

Which means:
  folder/filename  |  time of creation  |  name as appears in history  |
format  |  history name |  history owner

Only files that were not deleted in galaxy are printed (you can remove the
line if you want them all).

By the way, one of the main struggles was finding the folder/filename. We
have searched
the database repeatedly for a filename field to no avail, until someone
the fact that they can be derived from the dataset_id. If you zero pad the
id to six chars,
the first three chars would be your folder name. The whole id is the file
name. e.g.:
dataset_id = 12345
folder = 012
filename = dataset_12345.dat


Search engine candies:   export migrate files datasets filenames on disk
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

To search Galaxy mailing lists use the unified search at:

Reply via email to