RE: mountSharedResource() on huge amount of images
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Tuesday, March 15, 2011 7:10 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
Now you're talking about rendering them, which is a different
topic than mounting a resource which serves said images.
You'll only need to mount a single shared resource which
serves all the images. However, given the amount of images
you can consider allowing your front-end (e.g. apache httd)
or a dedicated webserver serve the images.
Is there example, please?
Since you mention that the amount of images can be
potentially large (250), I'd suggest removing the show all
option, or using an ajax viewport (max 20-50 images on-screen
at a time, when the user scrolls you load new images over ajax).
Someone else just suggested storing images in the database.
While there is usually heated debate about this topic (files
on disk vs in the database), let me just say that simply
having lots of images is no reason to move images into the
database. For starters, you can always store your files in a
hashed folder structure, e.g. when the ID is 1234, store the
image in a file/folder called /1/2/3/4.jpg
Bas
Op 15-3-2011 18:00, Ladislav DANKO schreef:
Imagine this situation: users have accounts on photo album
where they
upload images. System from uploaded images create thumbnails. Users
can browse their photo - there is combo show 25, show
50... show
all images. On page I render thumbnails on a page which
are shadowbox
clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more
than 3.000
images in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes the
filename from the url.
There is no reason to mount all images by such a handler
one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
RE: mountSharedResource() on huge amount of images
-Original Message-
From: mzem...@osc.state.ny.us [mailto:mzem...@osc.state.ny.us]
Sent: Tuesday, March 15, 2011 7:28 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
No offense Bas but that seems like a major hassle, especially
considering with an ORM its a simple update/delete/find.
When it comes time to build out new servers now you have to
shuffle around 300k photos instead of simply replicating a
database. Also sounds like it would make debugging more
difficult when your images are three or more folders deep.
And what if you want to store attributes along with the
photos? Like say user comments, or flag them for
inappropriate content, copyright infringement, etc. What if
there is an open file handle when you try to delete the
image? Just seems to me a much smoother solution to put in a
db. But with that I'm bowing out because as you said it's a
heated debate and comes down to personal preference.
Next version will have images in db. Arguments you serve I know,
from point of another atributes database is better.
From: Bas Gooren b...@iswd.nl
To: users@wicket.apache.org
Date: 03/15/2011 02:10 PM
Subject:Re: mountSharedResource() on huge amount of images
Now you're talking about rendering them, which is a different
topic than mounting a resource which serves said images.
You'll only need to mount a single shared resource which
serves all the images. However, given the amount of images
you can consider allowing your front-end (e.g. apache httd)
or a dedicated webserver serve the images.
Since you mention that the amount of images can be
potentially large (250), I'd suggest removing the show all
option, or using an ajax viewport (max 20-50 images on-screen
at a time, when the user scrolls you load new images over ajax).
Someone else just suggested storing images in the database.
While there is usually heated debate about this topic (files
on disk vs in the database), let me just say that simply
having lots of images is no reason to move images into the
database. For starters, you can always store your files in a
hashed folder structure, e.g. when the ID is 1234, store the
image in a file/folder called /1/2/3/4.jpg
Bas
Op 15-3-2011 18:00, Ladislav DANKO schreef:
Imagine this situation: users have accounts on photo album
where they
upload
images. System from uploaded images create thumbnails. Users can
browse their photo - there is combo show 25, show 50...
show all images.
On
page
I render thumbnails on a page which are shadowbox clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more
than 3.000
images in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes the
filename from the url.
There is no reason to mount all images by such a handler
one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
Notice: This communication, including any attachments, is
intended solely for the use of the individual or entity to
which it is addressed. This communication may contain
information that is protected from disclosure under State
and/or Federal law. Please notify the sender immediately if
you have received this communication in error and delete this
email from your system. If you are not the intended
recipient, you are requested not to disclose, copy
RE: mountSharedResource() on huge amount of images
Imagine this situation: users have accounts on photo album where they upload
images. System from uploaded images create thumbnails. Users can browse
their photo - there is combo show 25, show 50... show all images. On
page
I render thumbnails on a page which are shadowbox clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more than 3.000
images
in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes
the filename from the url.
There is no reason to mount all images by such a handler one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
RE: mountSharedResource() on huge amount of images
For that many photos I would suggest storing them in a database. Storing
that many images on the file system is cumbersome and inefficient in my
opinion. You might run into many headaches especially around backups,
deploys, upgrades, performance, file names, storage space, etc...
From: Ladislav DANKO em...@1ac0.net
To: users@wicket.apache.org
Date: 03/15/2011 01:01 PM
Subject:RE: mountSharedResource() on huge amount of images
Imagine this situation: users have accounts on photo album where they
upload
images. System from uploaded images create thumbnails. Users can browse
their photo - there is combo show 25, show 50... show all images. On
page
I render thumbnails on a page which are shadowbox clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more than 3.000
images
in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes
the filename from the url.
There is no reason to mount all images by such a handler one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
Notice: This communication, including any attachments, is intended solely
for the use of the individual or entity to which it is addressed. This
communication may contain information that is protected from disclosure
under State and/or Federal law. Please notify the sender immediately if
you have received this communication in error and delete this email from
your system. If you are not the intended recipient, you are requested not
to disclose, copy, distribute or take any action in reliance on the
contents of this information.
Re: mountSharedResource() on huge amount of images
Now you're talking about rendering them, which is a different topic than
mounting a resource which serves said images.
You'll only need to mount a single shared resource which serves all the
images. However, given the amount of images you can consider allowing
your front-end (e.g. apache httd) or a dedicated webserver serve the images.
Since you mention that the amount of images can be potentially large
(250), I'd suggest removing the show all option, or using an ajax
viewport (max 20-50 images on-screen at a time, when the user scrolls
you load new images over ajax).
Someone else just suggested storing images in the database. While there
is usually heated debate about this topic (files on disk vs in the
database), let me just say that simply having lots of images is no
reason to move images into the database. For starters, you can always
store your files in a hashed folder structure, e.g. when the ID is 1234,
store the image in a file/folder called /1/2/3/4.jpg
Bas
Op 15-3-2011 18:00, Ladislav DANKO schreef:
Imagine this situation: users have accounts on photo album where they upload
images. System from uploaded images create thumbnails. Users can browse
their photo - there is combo show 25, show 50... show all images. On
page
I render thumbnails on a page which are shadowbox clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more than 3.000
images
in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes
the filename from the url.
There is no reason to mount all images by such a handler one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
Re: mountSharedResource() on huge amount of images
No offense Bas but that seems like a major hassle, especially considering
with an ORM its a simple update/delete/find. When it comes time to build
out new servers now you have to shuffle around 300k photos instead of
simply replicating a database. Also sounds like it would make debugging
more difficult when your images are three or more folders deep. And what
if you want to store attributes along with the photos? Like say user
comments, or flag them for inappropriate content, copyright infringement,
etc. What if there is an open file handle when you try to delete the
image? Just seems to me a much smoother solution to put in a db. But
with that I'm bowing out because as you said it's a heated debate and
comes down to personal preference.
From: Bas Gooren b...@iswd.nl
To: users@wicket.apache.org
Date: 03/15/2011 02:10 PM
Subject:Re: mountSharedResource() on huge amount of images
Now you're talking about rendering them, which is a different topic than
mounting a resource which serves said images.
You'll only need to mount a single shared resource which serves all the
images. However, given the amount of images you can consider allowing
your front-end (e.g. apache httd) or a dedicated webserver serve the
images.
Since you mention that the amount of images can be potentially large
(250), I'd suggest removing the show all option, or using an ajax
viewport (max 20-50 images on-screen at a time, when the user scrolls
you load new images over ajax).
Someone else just suggested storing images in the database. While there
is usually heated debate about this topic (files on disk vs in the
database), let me just say that simply having lots of images is no
reason to move images into the database. For starters, you can always
store your files in a hashed folder structure, e.g. when the ID is 1234,
store the image in a file/folder called /1/2/3/4.jpg
Bas
Op 15-3-2011 18:00, Ladislav DANKO schreef:
Imagine this situation: users have accounts on photo album where they
upload
images. System from uploaded images create thumbnails. Users can browse
their photo - there is combo show 25, show 50... show all images.
On
page
I render thumbnails on a page which are shadowbox clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more than 3.000
images
in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes
the filename from the url.
There is no reason to mount all images by such a handler one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
Notice: This communication, including any attachments, is intended solely
for the use of the individual or entity to which it is addressed. This
communication may contain information that is protected from disclosure
under State and/or Federal law. Please notify the sender immediately if
you have received this communication in error and delete this email from
your system. If you are not the intended recipient, you are requested not
to disclose, copy, distribute or take any action in reliance on the
contents of this information.
Re: mountSharedResource() on huge amount of images
Well, exactly. It's usually a debate where both parties have solid
arguments and it boils down to preference.
I work with both options in medium-sized installations and in the end
see the benefits of both options, but prefer disk-based file storage.
For most applications though, a CDN or something like Amazon S3 is a
great option if you don't want the storage hassle. I say hassle
because managing hash-based storage is (imho) easy. It's only a matter
of adding a layer of abstraction between file access in the app and
actual file storage. Which is something I would always recommend to be
able to switch storage if required/desired.
This layer of abstraction can then either store files in the db, on
disk, both. It can also perform de-duplication, indexing, calculate a
hash for hash-based storage, store the file on S3, etc.
So to the original author: either way works, pick what you prefer and
suits your project best.
Bas
Op 15-3-2011 19:27, mzem...@osc.state.ny.us schreef:
No offense Bas but that seems like a major hassle, especially considering
with an ORM its a simple update/delete/find. When it comes time to build
out new servers now you have to shuffle around 300k photos instead of
simply replicating a database. Also sounds like it would make debugging
more difficult when your images are three or more folders deep. And what
if you want to store attributes along with the photos? Like say user
comments, or flag them for inappropriate content, copyright infringement,
etc. What if there is an open file handle when you try to delete the
image? Just seems to me a much smoother solution to put in a db. But
with that I'm bowing out because as you said it's a heated debate and
comes down to personal preference.
From: Bas Goorenb...@iswd.nl
To: users@wicket.apache.org
Date: 03/15/2011 02:10 PM
Subject:Re: mountSharedResource() on huge amount of images
Now you're talking about rendering them, which is a different topic than
mounting a resource which serves said images.
You'll only need to mount a single shared resource which serves all the
images. However, given the amount of images you can consider allowing
your front-end (e.g. apache httd) or a dedicated webserver serve the
images.
Since you mention that the amount of images can be potentially large
(250), I'd suggest removing the show all option, or using an ajax
viewport (max 20-50 images on-screen at a time, when the user scrolls
you load new images over ajax).
Someone else just suggested storing images in the database. While there
is usually heated debate about this topic (files on disk vs in the
database), let me just say that simply having lots of images is no
reason to move images into the database. For starters, you can always
store your files in a hashed folder structure, e.g. when the ID is 1234,
store the image in a file/folder called /1/2/3/4.jpg
Bas
Op 15-3-2011 18:00, Ladislav DANKO schreef:
Imagine this situation: users have accounts on photo album where they
upload
images. System from uploaded images create thumbnails. Users can browse
their photo - there is combo show 25, show 50... show all images.
On
page
I render thumbnails on a page which are shadowbox clickable images.
All images (show all) I do in way described below.
Works fine but in extreme situation there is user with more than 3.000
images
in one photoalbum.
Or -how to do it better way?
Thanks,
Laco
-Original Message-
From: Bas Gooren [mailto:b...@iswd.nl]
Sent: Thursday, March 10, 2011 11:32 PM
To: users@wicket.apache.org
Subject: Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes
the filename from the url.
There is no reason to mount all images by such a handler one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images
(thousands) in app? Does mountSharedResource() has any bottleneck?
Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFilelList = Arrays.asList(files);
Collections.sort(lList); int
i = 0; IteratorFileiterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
Notice
Re: mountSharedResource() on huge amount of images
The general idea is to mount a single handler, which takes the filename
from the url.
There is no reason to mount all images by such a handler one-by-one.
Bas
Op 10-3-2011 23:01, Ladislav DANKO schreef:
Hi all,
what is the recommended way to mount huge amount of an images (thousands)
in app? Does mountSharedResource() has any bottleneck? Simplified code look
like:
Folder folder = ((Start) Application.get()).getUploadFolder();
File[] files = folder.getFiles();
ListFile lList = Arrays.asList(files);
Collections.sort(lList);
int i = 0;
IteratorFile iterator = lList.iterator();
while(iterator.hasNext())
{
iterator.next();
String fileName = lList.get(i).getName();
mountSharedResource(/images/ + fileName, new
ResourceReference(Images.class, fileName).getSharedResourceKey());
i++;
}
But what if in folder is for example 100.000 photos?
Thanks for pointing,
--
Ladislav DANKO
-
To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org
For additional commands, e-mail: users-h...@wicket.apache.org
