This is an automated email from the ASF dual-hosted git repository.
echauchot pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 6418fb6 Move sequence diagrams to PlantUML SVG.
new 98eb81d Merge pull request #9870 from
RyanSkraba/rskraba/svg-sequence-diags
6418fb6 is described below
commit 6418fb6c4852ba4e07257c0b04c36d6fb9b28815
Author: Ryan Skraba <[email protected]>
AuthorDate: Thu Oct 24 16:23:52 2019 +0200
Move sequence diagrams to PlantUML SVG.
The plantuml source used to generate the diagrams is embedded in the SVG
resources and can be modified and regenerated (by hand, at this time).
Remove annotation on Reader.
---
.../src/documentation/io/developing-io-overview.md | 6 +-
website/src/documentation/programming-guide.md | 4 +-
website/src/images/dofn-sequence-diagram.png | Bin 126448 -> 0 bytes
website/src/images/dofn-sequence-diagram.svg | 94 ++++++++++++++++++
website/src/images/source-sequence-diagram.png | Bin 73657 -> 0 bytes
website/src/images/source-sequence-diagram.svg | 106 +++++++++++++++++++++
6 files changed, 205 insertions(+), 5 deletions(-)
diff --git a/website/src/documentation/io/developing-io-overview.md
b/website/src/documentation/io/developing-io-overview.md
index 56f2755..b17a710 100644
--- a/website/src/documentation/io/developing-io-overview.md
+++ b/website/src/documentation/io/developing-io-overview.md
@@ -107,9 +107,9 @@ Here is a sequence diagram that shows the lifecycle of the
Source during
information to IO developers such as the constraints that
apply to the objects or particular cases such as streaming mode.
-
- ![This is a sequence diagram that shows the lifecycle of the Source](
- {{ "/images/source-sequence-diagram.png" | prepend: site.baseurl }})
+ <!-- The source for the sequence diagram can be found in the the SVG
resource. -->
+![This is a sequence diagram that shows the lifecycle of the Source](
+ {{ "/images/source-sequence-diagram.svg" | prepend: site.baseurl }})
### Using ParDo and GroupByKey
diff --git a/website/src/documentation/programming-guide.md
b/website/src/documentation/programming-guide.md
index afb1163..d78b609 100644
--- a/website/src/documentation/programming-guide.md
+++ b/website/src/documentation/programming-guide.md
@@ -809,9 +809,9 @@ Here is a sequence diagram that shows the lifecycle of the
DoFn during
apply to the objects or particular cases such as failover or
instance reuse. They also give instanciation use cases.
-
+<!-- The source for the sequence diagram can be found in the the SVG resource.
-->
![This is a sequence diagram that shows the lifecycle of the DoFn](
- {{ "/images/dofn-sequence-diagram.png" | prepend: site.baseurl }})
+ {{ "/images/dofn-sequence-diagram.svg" | prepend: site.baseurl }})
#### 4.2.2. GroupByKey {#groupbykey}
diff --git a/website/src/images/dofn-sequence-diagram.png
b/website/src/images/dofn-sequence-diagram.png
deleted file mode 100644
index cf6570d..0000000
Binary files a/website/src/images/dofn-sequence-diagram.png and /dev/null differ
diff --git a/website/src/images/dofn-sequence-diagram.svg
b/website/src/images/dofn-sequence-diagram.svg
new file mode 100644
index 0000000..898b1ae
--- /dev/null
+++ b/website/src/images/dofn-sequence-diagram.svg
@@ -0,0 +1,94 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink";
contentScriptType="application/ecmascript" contentStyleType="text/css"
height="863px" preserveAspectRatio="none" style="width:740px;height:863px;"
version="1.1" viewBox="0 0 740 863" width="740px"
zoomAndPan="magnify"><defs/><g><rect fill="#6B9FE6" height="781.5625"
style="stroke: #6B9FE6; stroke-width: 1.0;" width="10" x="49"
y="60.8125"/><rect fill="#6B9FE6" height="425.2813" style="stroke: #6B9FE6;
stro [...]
+@startuml
+
+hide footbox
+skinparam backgroundColor transparent
+skinparam shadowing false
+skinparam defaultFontName "Roboto"
+
+skinparam sequenceArrowThickness 2
+
+skinparam note {
+ BackgroundColor #cee2f2
+ BorderColor #cee2f2
+}
+
+skinparam sequence {
+ ArrowColor #67666a
+
+ LifeLineBorderColor #6b9fe6
+ LifeLineBackgroundColor #6b9fe6
+
+ GroupBackgroundColor #8ac483
+ GroupBorderColor #8ac483
+
+ ParticipantBackgroundColor #8ac483
+ ParticipantBorderColor #8ac483
+}
+
+participant "User pipeline" as Pipeline
+participant DoFn << Serializable >>
+note right of DoFn: can have non-transient instance\nvariable state that will
be deserialized
+note right of DoFn: do not include enclosing class serializable state; use
static\nnested DoFn or define as anonymous class in static method
+note right of DoFn: no shared (global) static variable access (no sync
mechanism) but a beam\nstate (based on engine mechanisms) can be injected to
processElement
+note right of DoFn: keep as pure function as possible or idempotent side
effects\nbecause DoFns can be retried on failed bundles
+
+participant Runner
+
+activate Pipeline
+Pipeline -> DoFn: **create DoFn **
+DoFn -> Runner: **passed instance or deserialized on workers**
+
+note right Pipeline: If state variables are known at pipeline construction
step\ninitialize state variables by constructor
+
+group DoFn Lifecycle
+ Runner -> DoFn: **call setup**
+ activate Runner
+ activate DoFn
+ note right DoFn: reused instance to process other bundles on the same worker
+ note right DoFn: If state variables do not depend on the main pipeline
program and are the\nsame for all DoFn instances initialize them in setup
+ group For each bundle
+ Runner -> DoFn: **call startBundle**
+ group For each element
+ Runner -> DoFn: **call processElement**
+ note right DoFn: If state variables are computed by the pipeline\npass
it in a PcollectionView as a side input
+ DoFn - -> Runner: output
+ end
+ DoFn <- Runner: call onTimer
+ DoFn <- Runner: **call finishBundle**
+ end
+ DoFn -> Runner: **If DoFn is no more needed: call tearDown**
+ note right DoFn: Call of teardown is best effort; do not use for side effects
+end
+
+@enduml
+
+PlantUML version 1.2019.11(Sun Sep 22 12:02:15 CEST 2019)
+(GPL source distribution)
+Java Runtime: OpenJDK Runtime Environment
+JVM: OpenJDK 64-Bit Server VM
+Java Version: 1.8.0_222-b10
+Operating System: Linux
+Default Encoding: UTF-8
+Language: en
+Country: CA
+--></g></svg>
diff --git a/website/src/images/source-sequence-diagram.png
b/website/src/images/source-sequence-diagram.png
deleted file mode 100644
index d544eea..0000000
Binary files a/website/src/images/source-sequence-diagram.png and /dev/null
differ
diff --git a/website/src/images/source-sequence-diagram.svg
b/website/src/images/source-sequence-diagram.svg
new file mode 100644
index 0000000..02facd6
--- /dev/null
+++ b/website/src/images/source-sequence-diagram.svg
@@ -0,0 +1,106 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink";
contentScriptType="application/ecmascript" contentStyleType="text/css"
height="538px" preserveAspectRatio="none" style="width:574px;height:538px;"
version="1.1" viewBox="0 0 574 538" width="574px"
zoomAndPan="magnify"><defs/><g><rect fill="#6B9FE6" height="435.2813"
style="stroke: #6B9FE6; stroke-width: 1.0;" width="10" x="250.5"
y="82.0469"/><rect fill="#6B9FE6" height="152.1719" style="stroke: #6B9FE6; s
[...]
+@startuml
+
+hide footbox
+skinparam backgroundColor transparent
+skinparam shadowing false
+skinparam defaultFontName "Roboto"
+
+skinparam sequenceArrowThickness 2
+
+skinparam note {
+ BackgroundColor #cee2f2
+ BorderColor #cee2f2
+}
+
+skinparam sequence {
+ ArrowColor #67666a
+
+ LifeLineBorderColor #6b9fe6
+ LifeLineBackgroundColor #6b9fe6
+
+ GroupBackgroundColor #8ac483
+ GroupBorderColor #8ac483
+
+ ParticipantBackgroundColor #8ac483
+ ParticipantBorderColor #8ac483
+}
+
+participant Runner
+participant "Source" << Serializable >>
+participant "Reader"
+participant Middleware
+
+Runner -> Source: **create source**
+activate Source
+
+Runner -> Source: get estimated size
+
+Source -> Middleware: **estimate size**
+activate Middleware
+
+Source - -> Runner: size of data
+
+Runner -> Runner: compute size / number of executors\n = desired bundle size
+
+Runner -> Source: split source (desired bundle size)
+note right
+ streaming: split based on number of executors
+end note
+
+Source - -> Runner: list<source>
+note right
+ streaming: runner asks the source for watermark
+end note
+
+Runner -> Source: **for each source create a reader**
+
+Source -> Reader: **create a reader**
+activate Reader
+
+Reader - -> Runner: **readers**
+note right
+ streaming: + checkpoint
+end note
+
+Runner -> Reader: **for each reader : start reader**
+
+Runner -> Reader: **read elements until none to read**
+
+Reader -> Middleware: **get element**
+
+deactivate Middleware
+
+Runner -> Reader: **close reader**
+deactivate Reader
+@enduml
+
+PlantUML version 1.2019.11(Sun Sep 22 12:02:15 CEST 2019)
+(GPL source distribution)
+Java Runtime: OpenJDK Runtime Environment
+JVM: OpenJDK 64-Bit Server VM
+Java Version: 1.8.0_222-b10
+Operating System: Linux
+Default Encoding: UTF-8
+Language: en
+Country: CA
+--></g></svg>
