You can store blobs as much as you want in Warp 10 (up to the configured max size), but the actual access to that data will not be as efficient as if you were storing the same blobs in a more traditional K/V store. The reason being that when reading the blobs back, you will create a GTS with STRING values that will need further conversion to byte arrays.
In the standalone version LevelDB files are kept at 2Mb, storing anything larger will create larger file which will only contain a single value. In the distributed version, the maximum size you can store is limited by the maximum size of Kafka messages since all stored data will transit via Kafka. So while not optimal, it is still feasible. On Monday, February 17, 2020 at 5:46:45 PM UTC+1, henri laurent wrote: > > Thanks Fabien, > > I added > standalone.value.maxsize = 400000 > to the 00-warp.conf file and it now works. > > You mention that it is "far from being optimal" to save images in Warp10. > Is that a real issue knowing that the images will be very infrequent, once > an hour at most for time series recorded at 1Hz, and not that big, limited > to 200KB in size. > > best, > > Henri > > > On Monday, February 17, 2020 at 4:31:13 PM UTC+1, Fabien Tencé wrote: >> >> Hi Henri, >> >> I think there's missing an information in the doc: you also need to >> change standalone.value.maxsize for standalone or ingress.value.maxsize for >> distributed. >> Keep in mind that storing images in a time series database is far from >> being optimal. >> >> Regards, >> Fabien >> >> On Monday, February 17, 2020 at 3:06:26 PM UTC+1, henri laurent wrote: >>> >>> Hi, >>> >>> I am trying to save images in a standalone Warp10-2.4.0 database. >>> I have no problem saving images with a base64 encoded size less than >>> 100000 bytes (see attached photo_loaded_successfully.txt file). >>> I cannot save larger images, I am getting a (Value too large for GTS ) >>> error message while uploading my image with curl (see attached >>> photo_failing_to_load.txt file). >>> >>> I saw in the documentation ( >>> https://www.warp10.io/content/03_Documentation/03_Interacting_with_Warp_10/03_Ingesting_data/02_GTS_input_format) >>> >>> that the "Maximum size is linked to the max.encoder.size" parameter. >>> I modified standalone.max.encoder.size in the >>> /warp10-2.4.0/etc/conf.d/00-warp.conf file. from the default 100000 to >>> standalone.max.encoder.size = 400000 >>> and restarted Warp10. >>> >>> I still can't save pictures. >>> >>> Can you help? >>> >>> Thanks, >>> >>> Henri >>> >>> -- You received this message because you are subscribed to the Google Groups "Warp 10 users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/warp10-users/d8a0cb9e-1c44-468e-84b2-3b64905a0dae%40googlegroups.com.
