Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Chathura Ekanayake
As Charitha and Waruna mentioned, we will have to implement the storage method as an extension, where it is possible to switch among local file system (with rsync), FTP, S3, etc based on deployment requirements. @ChathuraD: We will not be exposing the actual storage to outside and all accesses

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Chathura Dilan
I'm +1 with option 2 since we can have a big storage for binary files and small storage for image files. The method of serving images and binary files are for two different things. We can optimize image storage for serving images because images are accessed frequently. Also in terms of security

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Megala Uthayakumar
Hi All, Please fine my responses below, *@Nuwan * Is there a problem is storing this file on the DB instead of the FS? As explained by Chathura and Charitha main reasoning behind not selecting the database is size of the binary file and yes we will consider other possible ways for supporting

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Waruna Jayaweera
Hi Megala, AFAIU There are two problems as where we store the file contents(supporting clustering cases) and file structure of application content. Storage Location - This should be written as extension where you can store in local directory, FTP, NFS or someone can customize based on

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Charitha Goonetilleke
Hi Nuwan, As our previous experience with similar approach we have used to store binaries in DB, it cause for scallability issues in later when growing the size of the DB in GBs. So storing files as binaries in DB is highly depending on the DB engine used and the growth rate of the data. IMO,

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Chathura Dilan
Hi Megala, If you go with the first approach, could you please mention what are the parameters you use to get the image? It is always better to get the image with a random parameters so it prevents hackers from guessing image names. @Nuwan. Some binary files could be large as 500MB in size. So

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Nuwan Dias
If you require NFS anyway (for other usecases), then I think its fine. But if you're introducing the complexity of NFS just for this use case, then I think it would be an overkill. BTW, not every one is ok with setting up a NFS. So if you need to share files between nodes you may want to have

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Megala Uthayakumar
Hi Nuwan, As mentioned by Harshan user needs to configure NFS when configuring this in HA setup. We do accept local file path or NFS url in the configuration file for the path. Thanks. On Wed, Aug 16, 2017 at 5:01 PM, Nuwan Dias wrote: > When you have two nodes of the IoT

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Harshan Liyanage
Hi, @Megala : I think having too many folders (lets say if there are 100's of apps) won't be a good idea. So I'm +1 with the option 2. As Nuwan has mentioned we need to think of serving this on HA mode. So if we are to go with filesystem based approach we might need to have NFS in-between IoT

Re: [Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Nuwan Dias
When you have two nodes of the IoT server running for HA, and if a given image/file is stored on the file system of one server, how do you make that image/file available for a request that's served via the other server? On Wed, Aug 16, 2017 at 4:54 PM, Megala Uthayakumar wrote:

[Dev] [IOT]Concerns regarding saving the image and binary files in the File System

2017-08-16 Thread Megala Uthayakumar
Hi All, Currently we are working on writing a mobile application store for IOT server. For that we have decided to save the images and binary files related with applications in the file system(This decision was taken considering the file size). File location will be derived from a configuration.