Sorry for posting on a separate thread. Lets continue the discussion here.
+1 for having URN to identify environment type. I think URN is better than
'oneof' structure as its more flexible and forward compatible.
On Fri, Aug 31, 2018 at 9:14 AM Thomas Weise wrote:
> FYI the first part of
Hi Ankur,
There is a separate related thread: "Process JobBundleFactory for portable
runner". It also contains a concrete suggestion for Environment proto
changes to accommodate docker and process based on a urn/payload structure
and a but of discussion.
We should either continue the discussion
Hi,
We recently added the ProceessEnvironment which uses forked process instead
of Docker container to run SDKHarness.
But the current Environment proto does not have a well defined structure to
represent a process.
Current Proto:
message Environment {
// (Required) The URL of a container
My mistake. I thought you needed to appear as "member" on apache/beam-site
as well.
On Fri, Aug 31, 2018 at 5:06 PM Thomas Weise wrote:
> All committers should be able to merge the site changes and wasn't that
> the case till the Jenkins outage?
>
>
> On Fri, Aug 31, 2018 at 3:19 PM Udi Meiri
All committers should be able to merge the site changes and wasn't that the
case till the Jenkins outage?
On Fri, Aug 31, 2018 at 3:19 PM Udi Meiri wrote:
> I believe the bot only listens to members.
>
> On Fri, Aug 31, 2018 at 2:56 PM Thomas Weise wrote:
>
>> Any idea why the beam-site merge
I believe the bot only listens to members.
On Fri, Aug 31, 2018 at 2:56 PM Thomas Weise wrote:
> Any idea why the beam-site merge bot doesn't work?
>
> The PRs are showing a "Merging is blocked" check that I don't remember
> seeing before.
>
>
> On Fri, Aug 31, 2018 at 2:06 AM Maximilian
Any idea why the beam-site merge bot doesn't work?
The PRs are showing a "Merging is blocked" check that I don't remember
seeing before.
On Fri, Aug 31, 2018 at 2:06 AM Maximilian Michels wrote:
> Jenkins is up again! (woho!)
>
> On 30.08.18 20:23, Thomas Weise wrote:
> > I would be concerned
In addition, JdbcIO is another source that could integrate with schemas.
Another point of integration could be with shared schema registries (such
as Kafka Schema Registry.). Any source can integrate with an external
registry and use it to set the schema on the output.
Reuven
On Fri, Aug 31,
On Fri, Aug 31, 2018 at 5:01 PM Alexey Romanenko
wrote:
> Thanks Reuven for updating community with this, great work!
>
> One small question about IO integration. What kind of integration this is
> supposed to be?
>
Two IOs that I would love to see benefit from schemas are BigQuery and Avro
I wanted to give a heads up that I granted Henning Rohde, Rafael Fernandez
and Connell O'Callaghan permissions on the Apache Beam JIRA project so they
would be able to create dashboards and sprints. If there are any concerns
or others need permissions, feel free to reach out to me or another PMC.
+1
Regards
JB
Le 31 août 2018 à 18:22, à 18:22, Lukasz Cwik a écrit:
>That is possible, I'll take people's date/time suggestions and create a
>simple online poll with them.
>
>On Fri, Aug 31, 2018 at 2:22 AM Robert Bradshaw
>wrote:
>
>> Thanks for taking this up. I added some comments to the
That is possible, I'll take people's date/time suggestions and create a
simple online poll with them.
On Fri, Aug 31, 2018 at 2:22 AM Robert Bradshaw wrote:
> Thanks for taking this up. I added some comments to the doc. A
> European-friendly time for discussion would be great.
>
> On Fri, Aug
FYI the first part of support for (direct) process based job bundle factory
was merged (thanks Max!)
https://github.com/apache/beam/pull/6287
On top of that I have built a customization that runs the Python SDK worker
directly:
Thanks Reuven for updating community with this, great work!
One small question about IO integration. What kind of integration this is
supposed to be? Are there any IOs that already benefit from Schemas support?
> On 31 Aug 2018, at 16:46, Reuven Lax wrote:
>
>
>
> On Fri, Aug 31, 2018 at
On Fri, Aug 31, 2018 at 2:22 AM Maximilian Michels wrote:
> Thanks Reuven. That's an OK restriction. Apache Flink also requires
> non-final fields to be able to generate TypeInformation (~=Schema) from
> PoJos.
>
> I agree that it's not very intuitive for Users.
>
> I suppose it would work to
Good point with identical types. You definitely want to avoid the following:
class Pojo {
final String param1;
final String param2;
Pojo(String param2, String param1) {
this.param1 = param1;
this.param2 = param2;
}
}
This would change the Pojo after deserialization. So this
On Thu, Aug 30, 2018 at 5:15 PM Reuven Lax wrote:
> Some answer inline:
> On Thu, Aug 30, 2018 at 7:56 AM Ismaël Mejía wrote:
>
>> Thanks Reuven for the excellent summary and thanks to all the guys who
>> worked in the Schema/SQL improvements. This is great for usability. I
>> really like the
On Fri, Aug 31, 2018 at 11:22 AM Maximilian Michels wrote:
> Thanks Reuven. That's an OK restriction. Apache Flink also requires
> non-final fields to be able to generate TypeInformation (~=Schema) from
> PoJos.
>
> I agree that it's not very intuitive for Users.
>
> I suppose it would work to
Thanks for taking this up. I added some comments to the doc. A
European-friendly time for discussion would be great.
On Fri, Aug 31, 2018 at 3:14 AM Lukasz Cwik wrote:
> I came up with a proposal[1] for a progress model solely based off of the
> backlog and that splits should be based upon the
Thanks Reuven. That's an OK restriction. Apache Flink also requires
non-final fields to be able to generate TypeInformation (~=Schema) from
PoJos.
I agree that it's not very intuitive for Users.
I suppose it would work to assume a constructor with the same parameter
order as the fields in
Jenkins is up again! (woho!)
On 30.08.18 20:23, Thomas Weise wrote:
I would be concerned with multiple folks running the Jekyll build
locally to end up with inconsistent results. But if Jenkins stays down
for longer, then maybe one of us can be the Jenkins substitute :)
On Thu, Aug 30, 2018
21 matches
Mail list logo
