Oh no, seems like we have one of the legendary domino effects with npm dependencies... .
I guess we need to tackle them step by step. Currently, I only have an intel nuc running on linux for private projects. That's why it's a bit unhandy when you travel by train. For testings purposes we can use npm link. Basically, it's a two step process: 1. Go to apache-beam into the typescript folder beam/sdks/typescript and run: npm link This command creates a global npm link on your device. We can now use the locally developed code directly in a test project. 2. Go to your test project where you want to use the apache-beam package and run: npm link apache-beam --save This should overwrite/add the apache-beam dependency in your package.json and uses instead the link to your local apache-beam code. Unfortunately, npm link doesn't behave like a real dependency. That's why you need to install linked dependencies as well: npm install --install-links As I said, I'll test it on the weekend and will give some feedback. Am Do., 12. Sept. 2024 um 02:57 Uhr schrieb Robert Bradshaw via dev < dev@beam.apache.org>: > On Mon, Sep 9, 2024 at 12:00 AM Robert Weber < > mister.robert.we...@gmail.com> wrote: > >> Good morning Roberts! >> >> This is such a conversation full of bright fame :-D --> Robert is an old >> German name that means “bright fame*.” *Just needed to celebrate >> somehow this coincidence. >> > > :) > > >> This week I am on a business travel. >> > > I've been traveling as well. > > >> But I can prepare something next week and document it as well. >> >> I'll send you an update. >> > > So, I took a stab at this. I tried to move to typescript 5, ts-patch, and > update a bunch of packages, see https://github.com/apache/beam/pull/32439 > . It compiles but is giving import errors (e.g. "Cannot use import > statement outside a module") I can read and write typescript itself, but > when it comes to the plethora of modules and packaging structure I am out > of my depth... > > > https://github.com/apache/beam/pull/32439/commits/b2560be4519a88873e36ce77b084063396c3f896 > was the only interesting (beam-specific) bit. I also had to do > https://github.com/apache/beam/pull/32439/commits/bfb180d7fdc0ee25d8cd427b77c5ab40deff659c > Hopefully that should be a good starting point. > > (My dev environment is just cloning the repo and running "npm i" in the > sdks/typescript directory. To test, e.g. the quickstart I run "npm pack" > and then in the quickstart directory "npm i > /path/to/created/apache-beam-2.60.0-SNAPSHOT.tgz" Just getting "npm test" > to work from within sdks/typescript inside the beam repo would be a big > step forward though.) > > >> Am Sa., 7. Sept. 2024 um 18:55 Uhr schrieb Robert Burke < >> rob...@frantil.com>: >> >>> Hello Robert, Robert. >>> >>> Since we don't have a great deal of experience with Typescript any >>> assistance in documenting (or pointing to standard typescript >>> documentation) on how to hook up a development module to a separate project >>> that uses the module would be valuable. >>> >>> Aside, I should also do the same for the Go side, where the standard is >>> a "go.work" file that allows the development repo to overlay the >>> modules usage in a different project. >>> >>> Robert >>> >>> On Sat, Sep 7, 2024, 12:34 AM Robert Weber < >>> mister.robert.we...@gmail.com> wrote: >>> >>>> I investigated a bit more on the typescript patch compilation topic. It >>>> seems that the ttypescript module is not well maintained anymore. The last >>>> version is more than 2 years old. An alternative is ts-patch. This one >>>> seems to be maintained and working with typescript version >5.0.0. Hence, I >>>> suggest a migration to ts-patch. >>>> ts-patch migration steps: >>>> >>>> 1. install ts-patch >>>> 2. use tspc instead of ttsc to compile during build (build.sh) >>>> [image: image.png] >>>> 3. uninstall ttypescript >>>> >>>> I think we don't need to adapt the transformer plugins in >>>> tsconfig.json, since, the plugin options are the same as for ttypescript. >>>> Hope this works! >>>> >>>> I definitely need to setup my own dev environment. So I can build and >>>> link apache-beam to the starter project locally and test it on my own. I >>>> will do this as my next step :-). >>>> >>>> Am Fr., 6. Sept. 2024 um 18:24 Uhr schrieb Robert Bradshaw via dev < >>>> dev@beam.apache.org>: >>>> >>>>> On Fri, Sep 6, 2024 at 4:41 AM Robert Weber >>>>> <mister.robert.we...@gmail.com> wrote: >>>>> > >>>>> > Hi everyone, >>>>> > >>>>> > I am very interested in apache beam and watched a video on youtube >>>>> about the typescript SDK and that you might need some community support. >>>>> >>>>> Yes, definitely. >>>>> >>>>> > That's why I am reaching out. >>>>> >>>>> That'd be great! >>>>> >>>>> > I have a proven experience of 10 years in data engineering and data >>>>> analytics, as well as python and javascript/typescript experience. I am >>>>> interested to support from time to time. Is this still relevant? >>>>> > >>>>> > First of all I tested the starter project for typescript. >>>>> Unfortunately, it is broken. I'd like to fix it ( >>>>> https://beam.apache.org/get-started/quickstart/typescript/). So far I >>>>> have some findings regarding the beam-starter-typescript: >>>>> > >>>>> > The long module is breaking the build: >>>>> > >>>>> > >>>>> node_modules/google-gax/node_modules/@grpc/grpc-js/node_modules/long/umd/index.d.ts:1:18 >>>>> - error TS1479: The current file is a CommonJS module whose imports will >>>>> produce 'require' calls; however, the referenced file is an ECMAScript >>>>> module and cannot be imported with 'require'. Consider writing a dynamic >>>>> 'import("../index.js")' call instead. >>>>> > >>>>> > 1 import Long from "../index.js"; >>>>> > >>>>> > To fix this you can either set skibLibCheck to true in >>>>> tsconfig.json, which is rather a dirty one, or make sure that an already >>>>> existing fix ( >>>>> https://github.com/dcodeIO/long.js/pull/1https://github.com/dcodeIO/long.js/pull/124 >>>>> 24) is implemented in all dependecies. Therefore, we can simply upgrade >>>>> all >>>>> dependecies. >>>>> > >>>>> > After upgrading to typescript to a higher version then >5.0.0, e.g. >>>>> version 5.5.4 the ttypescript module is breaking the build: >>>>> > >>>>> > $ npm run build >>>>> > >>>>> > > apache-beam-starter-project@0.1.0 build >>>>> > > ttsc >>>>> > >>>>> > >>>>> /home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84 >>>>> > tsm.createProgram = createProgram; >>>>> > ^ >>>>> > >>>>> > TypeError: Cannot set property createProgram of #<Object> which has >>>>> only a getter >>>>> > at patchCreateProgram >>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/patchCreateProgram.js:84:23) >>>>> > at loadTypeScript >>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/loadTypescript.js:21:56) >>>>> > at Object.<anonymous> >>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/lib/tsc.js:8:46) >>>>> > at Module._compile (node:internal/modules/cjs/loader:1364:14) >>>>> > at Module._extensions..js >>>>> (node:internal/modules/cjs/loader:1422:10) >>>>> > at Module.load (node:internal/modules/cjs/loader:1203:32) >>>>> > at Module._load (node:internal/modules/cjs/loader:1019:12) >>>>> > at Module.require (node:internal/modules/cjs/loader:1231:19) >>>>> > at require (node:internal/modules/helpers:177:18) >>>>> > at Object.<anonymous> >>>>> (/home/webnuke/Desktop/Projects/beam-starter-typescript/node_modules/ttypescript/bin/tsc:2:1) >>>>> > >>>>> > There is already a fix available. Thus, when we would move to >>>>> ttypescript version 1.1.15 it should be fixed. >>>>> > https://github.com/nonara/ts-patch/issues/93#issuecomment-1500795063 >>>>> > >>>>> > I just checked the version on github. In version 2.60-Snapshot the >>>>> ttypescript module is already on version 1.1.15. Could you just release a >>>>> new npm package. >>>>> >>>>> I tried upgrading typescript to 5.5.4, >>>>> >>>>> https://github.com/apache/beam/pull/32405/files >>>>> >>>>> Despite ttypescript being 1.1.15, I'm stiil getting the error above >>>>> when trying to build. >>>>> >>>>> I'll admit this is the area that I am the least comfortable with, >>>>> difficulties with packaging and versioning (and the various kinds of >>>>> modules) is what prevented me from releasing new mpms in the first >>>>> place (and then I got busy with other stuff and never got back to >>>>> this). Would greatly appreciate any help here! >>>>> >>>>> > Currently, the newest version on npm is 2.49. Afterwards we can >>>>> adapt the apache-beam version in the starter project. I assume then the >>>>> starter project would work again. >>>>> > >>>>> > I am looking forward to hearing from you! >>>>> > >>>>> > Best, >>>>> > Robert >>>>> > >>>>> >>>>
