[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-845633963 > Now, we don't have a method that doesn't embed assets in API binary? yes, the good old `make api-run` doesn't embed web assets and lua scripts. And newly introduced `make api-build` creates a build without embedding web assets. > There is an empty file `api/cmd/dag-to-lua/keep`. Yes, that's an ongoing discussion, please share your views, https://github.com/apache/apisix-dashboard/pull/1788#discussion_r634528665 https://github.com/apache/apisix-dashboard/pull/1788#discussion_r634248776 thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-841787048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-841856999 Hi, the tasks are done. The PR is ready to be reviewed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-841816015 Aye Aye, Captain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788:
URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-841797154
> Hi, @bisakhmondal
>
> > it will have another job that is writing the lua package into temporary
directory (`os.TempDir()`).
> > I don't get the point of this line, can you give more details about it ?
Sure. Actually because of this
https://github.com/apache/apisix-dashboard/blob/a59e8eff7ecdf38c197b4678c34d2fd15f364f72/api/internal/handler/route/route.go#L280-L296
Line 281: navigating to lua package directory
Line 296: Importing the package from the same directory.
But the issue is with import when the package is in a filesystem that is
inside memory/RAM (due to embedding).
So, as a workaround, we can do something like
1. embed the package. {provides the flexibility of a single binary}
2. when manager-api starts, it will dump the memory content of the package
into a temp directory (maybe say /tmp). Now it'll work fine.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-841787048 @nic-chen Something just come up into my mind. Let me know what you think about it. Since our main goal is to create a single binary, we can tackle the issue (I've described earlier) by this way, step-1 `during build : ` it involves downloading the lua scripts and integrate it into a single binary. [now the lua package is in memory whenever the manager api will start] step-2 `running manager-api : ` whenever the binary of manager-api will be loaded into memory, along with jobs including writing pid file, ..., it will have another job that is writing the lua package into temporary directory (`os.TempDir()`). step-3 `cleanup during stop: ` Optional. Now, this approach won't affect the current way we are translating the Lua script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-841684415 >Are there any related error logs? Hi @nic-chen, sorry I didn't get it. The snippet I mentioned earlier was the existing code that we are using for Lua to go compiler VM. Till now, I couldn't figure out the workaround as after embedding the dag-to-dua package will be an in-memory filesystem (no way I can make `os.Chdir` and `local dag_to_lua = require 'dag-to-lua'` to import the package ). All I can do is, read the content of the individual script from the directory stored in memory fs as per the current set of apis exposed through the go official `io/fs` package [link](https://golang.org/pkg/io/fs/) Another way we can tackle this by reading and joining the individual scripts into a monolithic script. what do you think? I don't know Lua (wishing to give it a try real soon ), so it would be great to have some heads up on this topic :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-840695352 Hi guys, regarding embedding the static lua codes into the binary, this is the method that handles the conversion of lua code, but I am not getting how this is done!! https://github.com/apache/apisix-dashboard/blob/a59e8eff7ecdf38c197b4678c34d2fd15f364f72/api/internal/handler/route/route.go#L271-L306 The documentation in `gopher-lua` is not that good. Could anyone give me some details? It seems pointing to the directory is enough, but after the embedding, it's going to be an **in-memory filesystem**. I haven't found such apis in gopher-lua, could you please guide me? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-828392441 > As we talked about the issue. [#1391 (comment)](https://github.com/apache/apisix-dashboard/issues/1391#issuecomment-824492523) > > > provide a flag to use the embed or the external assets in startup? > > We need a flag to use the embed or the external assets in a startup. You mean, a flag on manager-api itself (cobra commands), right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-827304425 Hii, what about `conf.yaml`, do we also have to embed this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-823779141 > hi @bisakhmondal > I think we should also embed `dag-to-lua`, see: > https://github.com/apache/apisix-dashboard/blob/master/api/build.sh#L38-L42 Okay. Let me have a look. I'll update it accordingly. Thanks :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-822427098 go1.16 has been added to the environment https://github.com/apache/apisix-dashboard/pull/1788/checks?check_run_id=2380595572#step:4:10 but still `api/test/shell/cli_test.sh` is using 1.15 https://github.com/apache/apisix-dashboard/pull/1788/checks?check_run_id=2380595572#step:6:79 any idea why樂 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [apisix-dashboard] bisakhmondal commented on pull request #1788: feat: embed assets in api binary
bisakhmondal commented on pull request #1788: URL: https://github.com/apache/apisix-dashboard/pull/1788#issuecomment-822147200 As `go:embed` and related package `io/fs` has been introduced in stable go version 1.16, we need to update the CI in the master branch. We can cherry-pick the CI update commit from this PR. cc @nic-chen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
