Re: Embed files into binary.
On 06/06/2016 6:45 PM, Pie? wrote: On Monday, 6 June 2016 at 06:10:35 UTC, rikki cattermole wrote: On 06/06/2016 6:06 PM, Pie? wrote: On Monday, 6 June 2016 at 05:30:12 UTC, rikki cattermole wrote: On 06/06/2016 5:07 PM, Pie? wrote: [...] Then I would say go get a new image library as that one isn't a very good one. ..snip.. Right got rid of all that text. If you want to make the filesystem appear to have files it doesn't, you'll need a virtual file system library. Welcome to the next issue, getting libraries to support it ;) My point exactly! Glad you could finally support my conclusion! ;) Finally was many many months ago[0]. I'm not happy with my implementation. Either way, somebody has to go ahead and write one for Phobos. Although good luck with getting that in. [0] https://github.com/rikkimax/alphaPhobos/tree/master/source/std/experimental/vfs Would this, say, allow an application to embed a "file system" into it's run time and then virtually dispatch file IO? If so, what about write access(the binary would grow in size when written to)? Its in the name, "virtual" file system. Write access would depend on where the "file" is. What about seemlessly shifting between a real file system and the virtual one by simple flag toggling and rebuilding? If so, then D would definitely benefit from it. This way one can create binaries and back them by a real file system for debugging purposes(easy access, no extracting from binaries or having to store them) then for release, one can turn that in to a virtual file system. Nope, virtual file system is its own representation. Can't switch all the c functions to use it. Its a almost full rewrite to use it. If this is what your vfs does or aims at then hats off! Cheerio! Where do I sign up? If not, would you mind expanding it to do such a thing? Again, I'm not happy with my one.
Re: Embed files into binary.
On Monday, 6 June 2016 at 06:10:35 UTC, rikki cattermole wrote: On 06/06/2016 6:06 PM, Pie? wrote: On Monday, 6 June 2016 at 05:30:12 UTC, rikki cattermole wrote: On 06/06/2016 5:07 PM, Pie? wrote: [...] Then I would say go get a new image library as that one isn't a very good one. ..snip.. Right got rid of all that text. If you want to make the filesystem appear to have files it doesn't, you'll need a virtual file system library. Welcome to the next issue, getting libraries to support it ;) My point exactly! Glad you could finally support my conclusion! ;) Finally was many many months ago[0]. I'm not happy with my implementation. Either way, somebody has to go ahead and write one for Phobos. Although good luck with getting that in. [0] https://github.com/rikkimax/alphaPhobos/tree/master/source/std/experimental/vfs Would this, say, allow an application to embed a "file system" into it's run time and then virtually dispatch file IO? If so, what about write access(the binary would grow in size when written to)? What about seemlessly shifting between a real file system and the virtual one by simple flag toggling and rebuilding? If so, then D would definitely benefit from it. This way one can create binaries and back them by a real file system for debugging purposes(easy access, no extracting from binaries or having to store them) then for release, one can turn that in to a virtual file system. If this is what your vfs does or aims at then hats off! Cheerio! Where do I sign up? If not, would you mind expanding it to do such a thing? I am essentially coding up something like that as we speak... Using static importing the binary and modulating all my file IO into a routine that chooses to use the physical file system when in debug mode and the imported data in release(it's quite easy, just dispatch to std.file io in debug and simply return the memory of the requested data in release).
Re: Embed files into binary.
On 06/06/2016 6:06 PM, Pie? wrote: On Monday, 6 June 2016 at 05:30:12 UTC, rikki cattermole wrote: On 06/06/2016 5:07 PM, Pie? wrote: [...] Then I would say go get a new image library as that one isn't a very good one. ..snip.. Right got rid of all that text. If you want to make the filesystem appear to have files it doesn't, you'll need a virtual file system library. Welcome to the next issue, getting libraries to support it ;) My point exactly! Glad you could finally support my conclusion! ;) Finally was many many months ago[0]. I'm not happy with my implementation. Either way, somebody has to go ahead and write one for Phobos. Although good luck with getting that in. [0] https://github.com/rikkimax/alphaPhobos/tree/master/source/std/experimental/vfs
Re: Embed files into binary.
On Monday, 6 June 2016 at 05:30:12 UTC, rikki cattermole wrote: On 06/06/2016 5:07 PM, Pie? wrote: [...] Then I would say go get a new image library as that one isn't a very good one. ..snip.. Right got rid of all that text. If you want to make the filesystem appear to have files it doesn't, you'll need a virtual file system library. Welcome to the next issue, getting libraries to support it ;) My point exactly! Glad you could finally support my conclusion! ;)
Re: Embed files into binary.
On 06/06/2016 5:07 PM, Pie? wrote:
On Monday, 6 June 2016 at 03:23:02 UTC, rikki cattermole wrote:
On 06/06/2016 3:11 PM, Pie? wrote:
On Monday, 6 June 2016 at 02:34:07 UTC, Adam D. Ruppe wrote:
On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote:
I believe the essentially converted the file into a ubyte or
something and then wrote that out to a temp file and read in the temp
file... this seems a bit of a kludge to me.
They might do that for certain special cases, but
import("file.whatever") just drops the file content in memory and you
can then access it as an array.
Because D allows such an embedding feature, maybe the file system
should allow working with this concept?
Why do you need it through the file system? If you're writing the
code, just use the array in memory. If it is external, see if the
library offers something like that.
DLL and exe are a special case, most things don't need to be physical
files.
Because, as I said, if I'm working with pre-exiting modules that work
with file, I have to provide a file or modify the source.
e.g., how could I do this easily with your read in your png module? It
takes a file..
/// Easily reads a png file into a MemoryImage
MemoryImage readPng(string filename) {
import std.file;
return imageFromPng(readPng(cast(ubyte[]) read(filename)));
recognize the code?
Of course, like I said, it can be modified in this case, but that means
nothing in general.
ubyte[] theArray = import("...");
MemoryImage img = imageFromPng(readPng(theArray));
And that's just from the snippet you provided.
Yay overloads!
Yes, I figured it could be done because of the MemoryImage, but that
wasn't the point. You are delving in to code that you might not
normally have access or might not exist.
Adam wrote the MemoryImage that essentially does this but that is not
the general case.
Image if all you access to was readPng(string filename), then what would
you say? That was my point of posting the code, I didn't mean for you to
go look and see if it was possible to achieve what I was saying in
arsd(Formally known as Adam Druppe) git repository ;)
Then I would say go get a new image library as that one isn't a very
good one.
..snip..
Right got rid of all that text.
If you want to make the filesystem appear to have files it doesn't,
you'll need a virtual file system library.
Welcome to the next issue, getting libraries to support it ;)
Re: Embed files into binary.
On Monday, 6 June 2016 at 03:23:02 UTC, rikki cattermole wrote:
On 06/06/2016 3:11 PM, Pie? wrote:
On Monday, 6 June 2016 at 02:34:07 UTC, Adam D. Ruppe wrote:
On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote:
I believe the essentially converted the file into a ubyte or
something and then wrote that out to a temp file and read in
the temp
file... this seems a bit of a kludge to me.
They might do that for certain special cases, but
import("file.whatever") just drops the file content in memory
and you
can then access it as an array.
Because D allows such an embedding feature, maybe the file
system
should allow working with this concept?
Why do you need it through the file system? If you're writing
the
code, just use the array in memory. If it is external, see if
the
library offers something like that.
DLL and exe are a special case, most things don't need to be
physical
files.
Because, as I said, if I'm working with pre-exiting modules
that work
with file, I have to provide a file or modify the source.
e.g., how could I do this easily with your read in your png
module? It
takes a file..
/// Easily reads a png file into a MemoryImage
MemoryImage readPng(string filename) {
import std.file;
return imageFromPng(readPng(cast(ubyte[]) read(filename)));
recognize the code?
Of course, like I said, it can be modified in this case, but
that means
nothing in general.
ubyte[] theArray = import("...");
MemoryImage img = imageFromPng(readPng(theArray));
And that's just from the snippet you provided.
Yay overloads!
Yes, I figured it could be done because of the MemoryImage, but
that wasn't the point. You are delving in to code that you might
not normally have access or might not exist.
Adam wrote the MemoryImage that essentially does this but that is
not the general case.
Image if all you access to was readPng(string filename), then
what would you say? That was my point of posting the code, I
didn't mean for you to go look and see if it was possible to
achieve what I was saying in arsd(Formally known as Adam Druppe)
git repository ;)
If that was the case I would have titled this something similar
not but maybe not exactly equivalent(Just being pedantic since we
seem to be having a communications gap in how accurate and
precise our expressions are to be interpreted): "Possible to get
arsd(Formally known as Adam Druppe) readpng lib to read from
import(filename)?".
Since I was desiring to arrive at a more general solution to
potential future programming goals, I phrased my question in more
general terms. I assumed that this would be understood. (i.e.,
vague questions generally gets/desires vague results, intelligent
questions generally get/desires intelligent results, etc.)
In fact, I posted the most generic(and possibly vague) question I
could:
"Embed files into binary." I stated, but not exclaimed or
questioned(as these are in theory, mutually exclusive), in the
subject line!
But this is no question! It is a riddle my friend! By stating it
in such a way as to only imply the logical consequent "How to use
embedded files in the binary"[Of course, with an infinite number
of assumptions mixed in for spice]!
Why is this a necessary forgone conclusion, as Einstein would
some times say(not to me in particular, mind you)? Because there
is no point to embed files in a binary if one can not use them!!!
Naturally, this is why I further said, in a round about way, why
exporting them to temp files is not really an option. This was
further to exclude, if you are keeping track with your Venn
Diagram, something akin to the infinite monkey theorem... or a
breach in the space time continuum or just several responses
about along the lines "...save the data to a temp file and use
that".
Note my retort:
">>>> Because D allows such an embedding feature, maybe the file
system
should allow working with this concept?"
Asking about future events and possibly space-time paradoxes!
For example, what if I derive some source code from some
unexpected places and they have some complex way of using D's
std.file routines and offers no interface to use memory arrays as
an input? Suppose further that it is very contorted and only
Steven Hawkins has figured out how to unravel the complexities of
the law of import/export. We are not privy to his knowledge or
intellectual properties. Would it not be intelligent and
forthright, mimicking Steven Hawkins in a sort of motherly
daughterly bond kind of way, to see that we must write new code
to solve the problem... this timing mimicking God himself as he
bought forth the earth and heaven?
Does that make sense?
Re: Embed files into binary.
On Monday, 6 June 2016 at 03:11:32 UTC, Pie? wrote:
e.g., how could I do this easily with your read in your png
module? It takes a file..
/// Easily reads a png file into a MemoryImage
MemoryImage readPng(string filename) {
import std.file;
return imageFromPng(readPng(cast(ubyte[]) read(filename)));
recognize the code?
The string version just forwards to the byte array version. So
you can use it directly with the array by just casting to ubyte[].
rikki cattermole got it right, except I *think* the cast(ubyte[])
needs to be explicit on the import line, so
ubyte[] theArray = cast(ubyte[]) import("...");
(Actually, the string one is an afterthought, I originally wrote
it to only work with arrays (well, technically, input ranges of
bytes), leaving the file reading to an outside function, but
since loading a file from a filename is so common, I added the
overload for it.)
Re: Embed files into binary.
On 06/06/2016 3:11 PM, Pie? wrote:
On Monday, 6 June 2016 at 02:34:07 UTC, Adam D. Ruppe wrote:
On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote:
I believe the essentially converted the file into a ubyte or
something and then wrote that out to a temp file and read in the temp
file... this seems a bit of a kludge to me.
They might do that for certain special cases, but
import("file.whatever") just drops the file content in memory and you
can then access it as an array.
Because D allows such an embedding feature, maybe the file system
should allow working with this concept?
Why do you need it through the file system? If you're writing the
code, just use the array in memory. If it is external, see if the
library offers something like that.
DLL and exe are a special case, most things don't need to be physical
files.
Because, as I said, if I'm working with pre-exiting modules that work
with file, I have to provide a file or modify the source.
e.g., how could I do this easily with your read in your png module? It
takes a file..
/// Easily reads a png file into a MemoryImage
MemoryImage readPng(string filename) {
import std.file;
return imageFromPng(readPng(cast(ubyte[]) read(filename)));
recognize the code?
Of course, like I said, it can be modified in this case, but that means
nothing in general.
ubyte[] theArray = import("...");
MemoryImage img = imageFromPng(readPng(theArray));
And that's just from the snippet you provided.
Yay overloads!
Re: Embed files into binary.
On Monday, 6 June 2016 at 02:34:07 UTC, Adam D. Ruppe wrote:
On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote:
I believe the essentially converted the file into a ubyte or
something and then wrote that out to a temp file and read in
the temp file... this seems a bit of a kludge to me.
They might do that for certain special cases, but
import("file.whatever") just drops the file content in memory
and you can then access it as an array.
Because D allows such an embedding feature, maybe the file
system should allow working with this concept?
Why do you need it through the file system? If you're writing
the code, just use the array in memory. If it is external, see
if the library offers something like that.
DLL and exe are a special case, most things don't need to be
physical files.
Because, as I said, if I'm working with pre-exiting modules that
work with file, I have to provide a file or modify the source.
e.g., how could I do this easily with your read in your png
module? It takes a file..
/// Easily reads a png file into a MemoryImage
MemoryImage readPng(string filename) {
import std.file;
return imageFromPng(readPng(cast(ubyte[]) read(filename)));
recognize the code?
Of course, like I said, it can be modified in this case, but that
means nothing in general.
Re: Embed files into binary.
On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote:
I believe the essentially converted the file into a ubyte or
something and then wrote that out to a temp file and read in
the temp file... this seems a bit of a kludge to me.
They might do that for certain special cases, but
import("file.whatever") just drops the file content in memory and
you can then access it as an array.
Because D allows such an embedding feature, maybe the file
system should allow working with this concept?
Why do you need it through the file system? If you're writing the
code, just use the array in memory. If it is external, see if the
library offers something like that.
DLL and exe are a special case, most things don't need to be
physical files.
Re: Embed files into binary.
On Monday, 6 June 2016 at 02:18:48 UTC, docandrew wrote: On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote: I saw somewhere someone explaining how to embed resources into a binary using the import keyword. I believe the essentially converted the file into a ubyte or something and then wrote that out to a temp file and read in the temp file... this seems a bit of a kludge to me. Is it possible to do the same, but create a sort of "in memory" file layout? While I can modify my own routines to read the "file" from memory, I can't do that easily with outside code(I could modify the binaries). Basically reading a file has to read it to memory, and if the file data already exists in memory, there is no point to read, just get it direct. Any thoughts on this? Because D allows such an embedding feature, maybe the file system should allow working with this concept? That way, it becomes VERY easy to embed files into the binary and work with them like they wernt. Also, going between the two different versions(embedded vs not) could be done with version (Release). I'm not sure about import, but one option is to put the external files directly in your binary as part of the linker step. Check out: http://www.linuxjournal.com/content/embedding-file-executable-aka-hello-world-version-5967 On Windows I think there's a way to embed files in .DLLs and then link those into your executable, but I'm ignorant of the steps/tools required. -Jon Check out https://p0nce.github.io/d-idioms/ for embedding dll's. It's much easier than using the link you give. The problem is the same though, How to use the in memory data rather than re-reading it? It's not a problem of embedding but that the routines that use the data expect files, not memory backed virtual files.
Re: Embed files into binary.
On Monday, 6 June 2016 at 02:05:09 UTC, Pie? wrote: I saw somewhere someone explaining how to embed resources into a binary using the import keyword. I believe the essentially converted the file into a ubyte or something and then wrote that out to a temp file and read in the temp file... this seems a bit of a kludge to me. Is it possible to do the same, but create a sort of "in memory" file layout? While I can modify my own routines to read the "file" from memory, I can't do that easily with outside code(I could modify the binaries). Basically reading a file has to read it to memory, and if the file data already exists in memory, there is no point to read, just get it direct. Any thoughts on this? Because D allows such an embedding feature, maybe the file system should allow working with this concept? That way, it becomes VERY easy to embed files into the binary and work with them like they wernt. Also, going between the two different versions(embedded vs not) could be done with version (Release). I'm not sure about import, but one option is to put the external files directly in your binary as part of the linker step. Check out: http://www.linuxjournal.com/content/embedding-file-executable-aka-hello-world-version-5967 On Windows I think there's a way to embed files in .DLLs and then link those into your executable, but I'm ignorant of the steps/tools required. -Jon
Embed files into binary.
I saw somewhere someone explaining how to embed resources into a binary using the import keyword. I believe the essentially converted the file into a ubyte or something and then wrote that out to a temp file and read in the temp file... this seems a bit of a kludge to me. Is it possible to do the same, but create a sort of "in memory" file layout? While I can modify my own routines to read the "file" from memory, I can't do that easily with outside code(I could modify the binaries). Basically reading a file has to read it to memory, and if the file data already exists in memory, there is no point to read, just get it direct. Any thoughts on this? Because D allows such an embedding feature, maybe the file system should allow working with this concept? That way, it becomes VERY easy to embed files into the binary and work with them like they wernt. Also, going between the two different versions(embedded vs not) could be done with version (Release).
