Thanks!
On Tue, Dec 8, 2020 at 6:54 AM Pablo Estrada wrote:
> Hi Chad!
> I've been meaning to review this, I've just not carved up the time. I'll
> try to get back to you this week with some thoughts!
> Thanks!
> -P.
>
> On Wed, Dec 2, 2020 at 10:31 AM Chad Dombrova wr
Hi everyone,
Beam's niche is low latency, high throughput workloads, but Beam has
incredible promise as an orchestrator of long running work that gets sent
to a scheduler. We've created a modified version of Beam that allows the
python SDK worker to outsource tasks to a scheduler, like Kubernetes
>
>
> Another example of an optional annotation is marking a transform to run on
> secure hardware, or to give hints to profiling/dynamic analysis tools.
>
There seems to be a lot of overlap between this idea and Environments. Can
you talk about how you feel they may be different or related?
It’s unfortunate that those instructions don’t include pre-commit, which is
by far the easiest way to do this.
To set it up:
pip install pre-commit
pre-commit install
Install sets up git pre-commit hooks so that it will run yapf and pylint on
changed files every time you commit (you’ll need
I would like to edit it! I have an apache account and I am a committed but
IIRC I could not edit it with my normal credentials.
On Wed, Oct 28, 2020 at 8:02 PM Robert Burke wrote:
> (it's a wiki, so anyone who requests and account can improve it)
>
> On Wed, Oct 28, 2020, 7:4
> All of these are great suggestions. I think what I really need though is
> some way to figure out how to cleanly install (perhaps reinstalling)
> everything I need to run all these commands. tox, yapf,
>
tox should be the only thing you need to install. After that, tox will
install whatever
>> ERROR: py36-lint: could not install deps [-rbuild-requirements.txt]; v
>> =
>> InvocationError('/usr/local/google/home/ajamato/beam/sdks/python/target/.tox/py36-lint/bin/python
>> target/.tox/py36-lint/bin/pip install --retr
This is a random idea, but the whole file IO system inside Beam would
actually be awesome to extract into its own project. IIRC, it’s not
particularly tied to Beam.
I’m not saying this should be done now, but it’s be nice to keep it mind
for a future goal.
-chad
On Wed, May 19, 2021 at 10:23
> Brian
>
> On Wed, May 19, 2021 at 8:38 PM Chad Dombrova wrote:
>
>> This is a random idea, but the whole file IO system inside Beam would
>> actually be awesome to extract into its own project. IIRC, it’s not
>> particularly tied to Beam.
>>
>&g
>
>
Hi,
First of all, this is an area that could use a lot of help, so thank you
Kyle for digging through the trove of tickets to understand all of the user
stories.
> I should have led with this. Someone wanted to mount credentials into
> the SDK harness [1]. So in this particular case
Hi Brian,
We implemented a feature that's similar to this, but with a different
motivation: scheduled tasks. We had the same need of creating batches of
logical elements, but rather than perform SIMD-optimized computations, we
want to produce remotely scheduled tasks. It's my hope that the
Hi all,
At work, I recently started playing around with mypyc[1] as a means to
compile our python code to C extensions, and I'm pretty impressed so far.
Pros
- write normal python code with annotations: we're already doing this!
- no need for cython-specific header files that can get out
>
> Thanks Chad I'll take a look at your talk and design to see if there's any
> ideas we can merge.
>
Thanks Brian. My hope is that even if you don't add the complete
scheduling framework, we'll get all the features and hooks we need to build
our toolset without needing to modify beam code
>
> - What does the new prototype code look like (hopefully much cleaner)?
>
Instead of a separate pxd file, you just have the existing .py file with
standard typing annotations.
> - How does performance compare to the Cython approach?
>
Good question. I've not been able to find any posts
I’m also interested in the answer to this. This is essential for reading
from many types of data sources.
On Tue, Jun 20, 2023 at 2:57 PM Sam Bourne wrote:
> +dev to see if anyone has any suggestions.
>
> On Fri, Jun 16, 2023 at 5:46 PM Sam Bourne wrote:
>
>> Hello beam community!
>>
>> I’m
101 - 115 of 115 matches
Mail list logo
