That's good to hear Mark. I wasn't looking forward to having to debug this if it was still an issue with the newer code.
I'm still actively working on this, in fact I put in a little time on it even today, so there isn't any documentation as things are apt to change as I explore different implementation approaches. One caveat to be aware of: The way the code now works is that it saves the imported image as a normal base 64 tiddler in the wiki tiddler store in memory, while uploading it to the files folder in the background. Once it has been uploaded and successfully saved to the files folder, the base 64 image tiddler is converted into a _canonical_uri tiddler. All of this happens without any user interaction and doesn't seem to disrupt normal usage in any manner. This workflow guards against losing data because of any issues writing the file to the files folder, and also works around the fact that import process in TW is designed to be synchronous. Working on a local node.js server it is all almost instantaneous and you don't notice that the tiddler is first saved as base 64, but just a heads up that this is what happens in the background. On a remote server over a slow connection, or with very large media files, this could be an issue as the normal sync mechanism may start saving the base 64 tiddler while the file is uploading to the files folder. However, best dealing with that may require some core modifications and support in the file saving adaptor. The long term goal is to identify what changes would be needed and hopefully get support for saving to the files folder in the core. Not present in the version I uploaded for you this morning but now implemented, is a toolbar button and a tm-externalise-tiddler message that can be used to convert an existing base 64 image into a _canonical_uri tiddler with the actual image saved in the files folder. If that's something you need, I can publish that. Saq On Friday, April 2, 2021 at 7:09:06 PM UTC+2 Mark S. wrote: > Ok! That seems to fix it -- it creates zero byte stub files! (Linux, > Chromium). > > Thank you! > On Friday, April 2, 2021 at 12:11:47 AM UTC-7 [email protected] wrote: > >> Hi Mark, >> >> Very peculiar! This suggests that there is something about my environment >> that makes this work, whereas it otherwise fails. I've been using this as >> part of a highly customized TW, where the image import is via a custom UI >> and part of a larger workflow, so it is possible that there is some other >> part of it that is needed for this to work properly. >> >> However, I did realize that when I was testing yesterday to recreate your >> problem in a fresh wiki, I made the mistake of using the version of the >> plugin that I have been working on the last week. Since this is the code >> base I will be working on going forwards, it would be very helpful to know >> if this version still gives you the same issue on 5.1.23, before I try to >> do a deeper dive and identify and resolve the issue. >> >> https://saqimtiaz.github.io/sq-tw/temp/import-to-external-file.html >> >> Thanks again for the feedback and time invested in testing this. >> Regards, >> >> Saq >> >> On Friday, April 2, 2021 at 4:00:40 AM UTC+2 Mark S. wrote: >> >>> I just booted into Windows 10 and experienced the same error. In the >>> case of one of the image files, it corrected itself after I temporarily >>> changed the text in the canonical_uri tiddler. But the other two images >>> that I imported continued to have a non-zero byte file (1kib) in the >>> tiddlers directory. In addition, there was an oddity that the imported >>> image files have no mod dates, and so don't appear in the "recent" list. >>> >>> So I've tried it now on 3 file systems, 3 operating systems and two >>> different physical machines. >>> >>> Thanks! >>> >>> On Thursday, April 1, 2021 at 2:49:14 PM UTC-7 [email protected] wrote: >>> >>>> Hi Mark, appreciate the additional testing and feedback. It at least >>>> helps rule out some potential causes. >>>> >>>> I'm still quite perplexed as I have been using this for several months >>>> now, with no issues across several server restarts. I consistently get 0 >>>> bytes image files in the tiddlers folder, which I presume is the expected >>>> behaviour. >>>> >>>> Anyway, I've actually worked on this some the last week, including >>>> rewriting portions and adding better redundancy and error handling in case >>>> of write/upload errors. I'll publish that once I get the chance, it will >>>> be >>>> interesting to see if the problem you are encountering still persists. >>>> >>>> On Thursday, April 1, 2021 at 11:38:39 PM UTC+2 Mark S. wrote: >>>> >>>>> Just to be clear, the 6 byte images were created using the kit from >>>>> your sandbox. >>>>> >>>>> Just in case it was an OS thing, I fired up my old Windows 7 machine, >>>>> created a new node.js instance (5.1.21 is what I have there), dragged and >>>>> dropped the items from your sandbox including the plugin, restarted the >>>>> instance. Then dropped in an image. It made the _canonical_uri tiddler. I >>>>> was able to insert the image as img[[ wikitext. Then I restarted the >>>>> instance. Now the img[] image is replaced by the "missing" icon. On the >>>>> disk, the image file in the tiddlers directory is 1k rather than 6bytes. >>>>> (1k is too small to be the image size, BTW). >>>>> >>>>> Then, just in case it related to TW version, I upgraded to 5.1.23 and >>>>> tested again. Once again it makes 1k files. But they won't display via >>>>> img[] after restart. >>>>> >>>>> So the same problem appears to exist when making files on Windows and >>>>> on Linux. The only difference is that in one system it's making 6byte >>>>> image >>>>> files and in the other 1k byte files. Either way the image doesn't >>>>> display >>>>> with img[] after the node instance has been restarted. >>>>> >>>>> On Thursday, April 1, 2021 at 12:47:31 PM UTC-7 [email protected] >>>>> wrote: >>>>> >>>>>> Understood. As I'm unable to reproduce the problem of the 6 byte >>>>>> images being created, there isn't much I can do to further debug this >>>>>> without more information. >>>>>> >>>>>> Saq >>>>>> On Thursday, April 1, 2021 at 9:40:28 PM UTC+2 Mark S. wrote: >>>>>> >>>>>>> I explained that poorly. The way that I created the png files >>>>>>> (actually using your external files tool) generated 6 byte file images. >>>>>>> *These* images do not display correctly. >>>>>>> >>>>>>> The zero-byte images generated when making external uri's by hand >>>>>>> *do* display correctly. >>>>>>> >>>>>>> So to be clear, AFAIK, there is nothing wrong with img[] >>>>>>> nomenclature under node.js. If there was still a way to mark an issue >>>>>>> as >>>>>>> resolved, I would do that. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> On Thursday, April 1, 2021 at 11:24:56 AM UTC-7 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> It would helpful to determine what the default behaviour of TW on >>>>>>>> node.js is for _canonical_uri tiddlers and saving to file. >>>>>>>> >>>>>>>> On Windows 10, latest Chrome and using latest pre-release with the >>>>>>>> server edition and no plugins, I tried manually creating a tiddler >>>>>>>> with: >>>>>>>> - title: test.png >>>>>>>> - type: image/png, >>>>>>>> - and an arbitrary _canonical_uri value of files/test.png >>>>>>>> >>>>>>>> In the tiddlers directory I get a 0 bytes file test.png and >>>>>>>> test.png.meta. >>>>>>>> I am not sure if the 0 byte expected behaviour on node.js for >>>>>>>> canonical_uri tiddlers or something new in 5.1.23 >>>>>>>> >>>>>>>> Using [img[test.png]] in another tiddler, the img is correctly >>>>>>>> resolved to have a src attribute files/test.png, both before and after >>>>>>>> restarting the node server. >>>>>>>> >>>>>>>> So I cannot reproduce the problem. It could be OS specific. >>>>>>>> >>>>>>>> It is worth considering that 5.1.23 had significant changes to the >>>>>>>> file saving code, so there may have been a regression. >>>>>>>> Also I am using the latest pre-release, which might be relevant as >>>>>>>> at least one bugfix since 5.1.23 seems to deal with missing text >>>>>>>> fields of >>>>>>>> tiddlers, see >>>>>>>> https://github.com/Jermolene/TiddlyWiki5/pull/5416/files >>>>>>>> >>>>>>>> If you can confirm that creating a _canonical_uri tiddler by hand >>>>>>>> works well for you on node.js, that will help narrow down whether this >>>>>>>> is a >>>>>>>> core issues or a plugin issue. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Saq >>>>>>>> On Thursday, April 1, 2021 at 7:43:50 PM UTC+2 Saq Imtiaz wrote: >>>>>>>> >>>>>>>>> Hmm. That's interesting. I'm seeing 0 byte image files but the >>>>>>>>> images do otherwise load correctly. >>>>>>>>> What OS and browser are you using? >>>>>>>>> >>>>>>>>> It would also be interesting to check how TW on node.js on your >>>>>>>>> system, with no plugins saves _canonical_uri tiddlers. Are there any >>>>>>>>> image >>>>>>>>> files created at all, or just a tid file? >>>>>>>>> >>>>>>>>> I've been working on this here and there over the last week, so >>>>>>>>> with a few more details I should be able to address this. >>>>>>>>> >>>>>>>>> On Thursday, April 1, 2021 at 7:24:36 PM UTC+2 Mark S. wrote: >>>>>>>>> >>>>>>>>>> The problem turned out to be that the way the _canonical_uri >>>>>>>>>> files were created was also creating 6byte image files in the >>>>>>>>>> tiddler >>>>>>>>>> folder which apparently confuses the system into thinking that it is >>>>>>>>>> dealing with base64 image files. >>>>>>>>>> >>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/7b086599-2e51-4f04-a41b-b96b70038813n%40googlegroups.com.
