Author: Armin Rigo <[email protected]>
Branch: extradoc
Changeset: r5357:af502143bac9
Date: 2014-07-16 20:08 +0200
http://bitbucket.org/pypy/extradoc/changeset/af502143bac9/
Log: Now I'm finally progressing :-) Thanks Remi for the blueprint of
the ICOOOLPS slides
diff --git a/talk/ep2014/stm/Makefile b/talk/ep2014/stm/Makefile
new file mode 100644
--- /dev/null
+++ b/talk/ep2014/stm/Makefile
@@ -0,0 +1,2 @@
+talk.html: talk.rst stylesheet.css
+ rst2s5 talk.rst --stylesheet-path=stylesheet.css > talk.html
diff --git a/talk/ep2014/stm/stylesheet.css b/talk/ep2014/stm/stylesheet.css
new file mode 100644
--- /dev/null
+++ b/talk/ep2014/stm/stylesheet.css
@@ -0,0 +1,330 @@
+/*
+:Author: David Goodger ([email protected])
+:Id: $Id: html4css1.css 7614 2013-02-21 15:55:51Z milde $
+:Copyright: This stylesheet has been placed in the public domain.
+
+Default cascading style sheet for the HTML output of Docutils.
+
+See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
+customize this style sheet.
+*/
+
+/* used to remove borders from tables and images */
+.borderless, table.borderless td, table.borderless th {
+ border: 0 }
+
+table.borderless td, table.borderless th {
+ /* Override padding for "table.docutils td" with "! important".
+ The right padding separates the table cells. */
+ padding: 0 0.5em 0 0 ! important }
+
+.first {
+ /* Override more specific margin styles with "! important". */
+ margin-top: 0 ! important }
+
+.last, .with-subtitle {
+ margin-bottom: 0 ! important }
+
+.hidden {
+ display: none }
+
+a.toc-backref {
+ text-decoration: none ;
+ color: black }
+
+blockquote.epigraph {
+ margin: 2em 5em ; }
+
+dl.docutils dd {
+ margin-bottom: 0.5em }
+
+object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
+ overflow: hidden;
+}
+
+/* Uncomment (and remove this text!) to get bold-faced definition list terms
+dl.docutils dt {
+ font-weight: bold }
+*/
+
+div.abstract {
+ margin: 2em 5em }
+
+div.abstract p.topic-title {
+ font-weight: bold ;
+ text-align: center }
+
+div.admonition, div.attention, div.caution, div.danger, div.error,
+div.hint, div.important, div.note, div.tip, div.warning {
+ margin: 2em ;
+ border: medium outset ;
+ padding: 1em }
+
+div.admonition p.admonition-title, div.hint p.admonition-title,
+div.important p.admonition-title, div.note p.admonition-title,
+div.tip p.admonition-title {
+ font-weight: bold ;
+ font-family: sans-serif }
+
+div.attention p.admonition-title, div.caution p.admonition-title,
+div.danger p.admonition-title, div.error p.admonition-title,
+div.warning p.admonition-title, .code .error {
+ color: red ;
+ font-weight: bold ;
+ font-family: sans-serif }
+
+/* Uncomment (and remove this text!) to get reduced vertical space in
+ compound paragraphs.
+div.compound .compound-first, div.compound .compound-middle {
+ margin-bottom: 0.5em }
+
+div.compound .compound-last, div.compound .compound-middle {
+ margin-top: 0.5em }
+*/
+
+div.dedication {
+ margin: 2em 5em ;
+ text-align: center ;
+ font-style: italic }
+
+div.dedication p.topic-title {
+ font-weight: bold ;
+ font-style: normal }
+
+div.figure {
+ margin-left: 2em ;
+ margin-right: 2em }
+
+div.footer, div.header {
+ clear: both;
+ font-size: smaller }
+
+div.line-block {
+ display: block ;
+ margin-top: 1em ;
+ margin-bottom: 1em }
+
+div.line-block div.line-block {
+ margin-top: 0 ;
+ margin-bottom: 0 ;
+ margin-left: 1.5em }
+
+div.sidebar {
+ margin: 0 0 0.5em 1em ;
+ border: medium outset ;
+ padding: 1em ;
+ background-color: #ffffee ;
+ width: 40% ;
+ float: right ;
+ clear: right }
+
+div.sidebar p.rubric {
+ font-family: sans-serif ;
+ font-size: medium }
+
+div.system-messages {
+ margin: 5em }
+
+div.system-messages h1 {
+ color: red }
+
+div.system-message {
+ border: medium outset ;
+ padding: 1em }
+
+div.system-message p.system-message-title {
+ color: red ;
+ font-weight: bold }
+
+div.topic {
+ margin: 2em }
+
+h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
+h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
+ margin-top: 0.4em }
+
+h1.title {
+ text-align: center }
+
+h2.subtitle {
+ text-align: center }
+
+hr.docutils {
+ width: 75% }
+
+img.align-left, .figure.align-left, object.align-left {
+ clear: left ;
+ float: left ;
+ margin-right: 1em }
+
+img.align-right, .figure.align-right, object.align-right {
+ clear: right ;
+ float: right ;
+ margin-left: 1em }
+
+img.align-center, .figure.align-center, object.align-center {
+ display: block;
+ margin-left: auto;
+ margin-right: auto;
+}
+
+.align-left {
+ text-align: left }
+
+.align-center {
+ clear: both ;
+ text-align: center }
+
+.align-right {
+ text-align: right }
+
+/* reset inner alignment in figures */
+div.align-right {
+ text-align: inherit }
+
+/* div.align-center * { */
+/* text-align: left } */
+
+ol.simple, ul.simple {
+ margin-bottom: 1em }
+
+ol.arabic {
+ list-style: decimal }
+
+ol.loweralpha {
+ list-style: lower-alpha }
+
+ol.upperalpha {
+ list-style: upper-alpha }
+
+ol.lowerroman {
+ list-style: lower-roman }
+
+ol.upperroman {
+ list-style: upper-roman }
+
+p.attribution {
+ text-align: right ;
+ margin-left: 50% }
+
+p.caption {
+ font-style: italic }
+
+p.credits {
+ font-style: italic ;
+ font-size: smaller }
+
+p.label {
+ white-space: nowrap }
+
+p.rubric {
+ font-weight: bold ;
+ font-size: larger ;
+ color: maroon ;
+ text-align: center }
+
+p.sidebar-title {
+ font-family: sans-serif ;
+ font-weight: bold ;
+ font-size: larger }
+
+p.sidebar-subtitle {
+ font-family: sans-serif ;
+ font-weight: bold }
+
+p.topic-title {
+ font-weight: bold }
+
+pre.address {
+ margin-bottom: 0 ;
+ margin-top: 0 ;
+ font: inherit }
+
+pre.literal-block, pre.doctest-block, pre.math, pre.code {
+ margin-left: 2em ;
+ margin-right: 2em }
+
+pre.code .ln { color: grey; } /* line numbers */
+pre.code, code { background-color: #eeeeee }
+pre.code .comment, code .comment { color: #5C6576 }
+pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
+pre.code .literal.string, code .literal.string { color: #0C5404 }
+pre.code .name.builtin, code .name.builtin { color: #352B84 }
+pre.code .deleted, code .deleted { background-color: #DEB0A1}
+pre.code .inserted, code .inserted { background-color: #A3D289}
+
+span.classifier {
+ font-family: sans-serif ;
+ font-style: oblique }
+
+span.classifier-delimiter {
+ font-family: sans-serif ;
+ font-weight: bold }
+
+span.interpreted {
+ font-family: sans-serif }
+
+span.option {
+ white-space: nowrap }
+
+span.pre {
+ white-space: pre }
+
+span.problematic {
+ color: red }
+
+span.section-subtitle {
+ /* font-size relative to parent (h1..h6 element) */
+ font-size: 80% }
+
+table.citation {
+ border-left: solid 1px gray;
+ margin-left: 1px }
+
+table.docinfo {
+ margin: 2em 4em }
+
+table.docutils {
+ margin-top: 0.5em ;
+ margin-bottom: 0.5em }
+
+table.footnote {
+ border-left: solid 1px black;
+ margin-left: 1px }
+
+table.docutils td, table.docutils th,
+table.docinfo td, table.docinfo th {
+ padding-left: 0.5em ;
+ padding-right: 0.5em ;
+ vertical-align: top }
+
+table.docutils th.field-name, table.docinfo th.docinfo-name {
+ font-weight: bold ;
+ text-align: left ;
+ white-space: nowrap ;
+ padding-left: 0 }
+
+/* "booktabs" style (no vertical lines) */
+table.docutils.booktabs {
+ border: 0px;
+ border-top: 2px solid;
+ border-bottom: 2px solid;
+ border-collapse: collapse;
+}
+table.docutils.booktabs * {
+ border: 0px;
+}
+table.docutils.booktabs th {
+ border-bottom: thin solid;
+ text-align: left;
+}
+
+h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
+h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
+ font-size: 100% }
+
+ul.auto-toc {
+ list-style-type: none }
+
+.slide li {
+ padding-top: 0.2em }
diff --git a/talk/ep2014/stm/talk.html b/talk/ep2014/stm/talk.html
new file mode 100644
--- /dev/null
+++ b/talk/ep2014/stm/talk.html
@@ -0,0 +1,577 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
+<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+<meta name="generator" content="Docutils 0.11:
http://docutils.sourceforge.net/"; />
+<meta name="version" content="S5 1.1" />
+<title>Using All These Cores: Transactional Memory in PyPy</title>
+<style type="text/css">
+
+/*
+:Author: David Goodger ([email protected])
+:Id: $Id: html4css1.css 7614 2013-02-21 15:55:51Z milde $
+:Copyright: This stylesheet has been placed in the public domain.
+
+Default cascading style sheet for the HTML output of Docutils.
+
+See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
+customize this style sheet.
+*/
+
+/* used to remove borders from tables and images */
+.borderless, table.borderless td, table.borderless th {
+ border: 0 }
+
+table.borderless td, table.borderless th {
+ /* Override padding for "table.docutils td" with "! important".
+ The right padding separates the table cells. */
+ padding: 0 0.5em 0 0 ! important }
+
+.first {
+ /* Override more specific margin styles with "! important". */
+ margin-top: 0 ! important }
+
+.last, .with-subtitle {
+ margin-bottom: 0 ! important }
+
+.hidden {
+ display: none }
+
+a.toc-backref {
+ text-decoration: none ;
+ color: black }
+
+blockquote.epigraph {
+ margin: 2em 5em ; }
+
+dl.docutils dd {
+ margin-bottom: 0.5em }
+
+object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
+ overflow: hidden;
+}
+
+/* Uncomment (and remove this text!) to get bold-faced definition list terms
+dl.docutils dt {
+ font-weight: bold }
+*/
+
+div.abstract {
+ margin: 2em 5em }
+
+div.abstract p.topic-title {
+ font-weight: bold ;
+ text-align: center }
+
+div.admonition, div.attention, div.caution, div.danger, div.error,
+div.hint, div.important, div.note, div.tip, div.warning {
+ margin: 2em ;
+ border: medium outset ;
+ padding: 1em }
+
+div.admonition p.admonition-title, div.hint p.admonition-title,
+div.important p.admonition-title, div.note p.admonition-title,
+div.tip p.admonition-title {
+ font-weight: bold ;
+ font-family: sans-serif }
+
+div.attention p.admonition-title, div.caution p.admonition-title,
+div.danger p.admonition-title, div.error p.admonition-title,
+div.warning p.admonition-title, .code .error {
+ color: red ;
+ font-weight: bold ;
+ font-family: sans-serif }
+
+/* Uncomment (and remove this text!) to get reduced vertical space in
+ compound paragraphs.
+div.compound .compound-first, div.compound .compound-middle {
+ margin-bottom: 0.5em }
+
+div.compound .compound-last, div.compound .compound-middle {
+ margin-top: 0.5em }
+*/
+
+div.dedication {
+ margin: 2em 5em ;
+ text-align: center ;
+ font-style: italic }
+
+div.dedication p.topic-title {
+ font-weight: bold ;
+ font-style: normal }
+
+div.figure {
+ margin-left: 2em ;
+ margin-right: 2em }
+
+div.footer, div.header {
+ clear: both;
+ font-size: smaller }
+
+div.line-block {
+ display: block ;
+ margin-top: 1em ;
+ margin-bottom: 1em }
+
+div.line-block div.line-block {
+ margin-top: 0 ;
+ margin-bottom: 0 ;
+ margin-left: 1.5em }
+
+div.sidebar {
+ margin: 0 0 0.5em 1em ;
+ border: medium outset ;
+ padding: 1em ;
+ background-color: #ffffee ;
+ width: 40% ;
+ float: right ;
+ clear: right }
+
+div.sidebar p.rubric {
+ font-family: sans-serif ;
+ font-size: medium }
+
+div.system-messages {
+ margin: 5em }
+
+div.system-messages h1 {
+ color: red }
+
+div.system-message {
+ border: medium outset ;
+ padding: 1em }
+
+div.system-message p.system-message-title {
+ color: red ;
+ font-weight: bold }
+
+div.topic {
+ margin: 2em }
+
+h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
+h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
+ margin-top: 0.4em }
+
+h1.title {
+ text-align: center }
+
+h2.subtitle {
+ text-align: center }
+
+hr.docutils {
+ width: 75% }
+
+img.align-left, .figure.align-left, object.align-left {
+ clear: left ;
+ float: left ;
+ margin-right: 1em }
+
+img.align-right, .figure.align-right, object.align-right {
+ clear: right ;
+ float: right ;
+ margin-left: 1em }
+
+img.align-center, .figure.align-center, object.align-center {
+ display: block;
+ margin-left: auto;
+ margin-right: auto;
+}
+
+.align-left {
+ text-align: left }
+
+.align-center {
+ clear: both ;
+ text-align: center }
+
+.align-right {
+ text-align: right }
+
+/* reset inner alignment in figures */
+div.align-right {
+ text-align: inherit }
+
+/* div.align-center * { */
+/* text-align: left } */
+
+ol.simple, ul.simple {
+ margin-bottom: 1em }
+
+ol.arabic {
+ list-style: decimal }
+
+ol.loweralpha {
+ list-style: lower-alpha }
+
+ol.upperalpha {
+ list-style: upper-alpha }
+
+ol.lowerroman {
+ list-style: lower-roman }
+
+ol.upperroman {
+ list-style: upper-roman }
+
+p.attribution {
+ text-align: right ;
+ margin-left: 50% }
+
+p.caption {
+ font-style: italic }
+
+p.credits {
+ font-style: italic ;
+ font-size: smaller }
+
+p.label {
+ white-space: nowrap }
+
+p.rubric {
+ font-weight: bold ;
+ font-size: larger ;
+ color: maroon ;
+ text-align: center }
+
+p.sidebar-title {
+ font-family: sans-serif ;
+ font-weight: bold ;
+ font-size: larger }
+
+p.sidebar-subtitle {
+ font-family: sans-serif ;
+ font-weight: bold }
+
+p.topic-title {
+ font-weight: bold }
+
+pre.address {
+ margin-bottom: 0 ;
+ margin-top: 0 ;
+ font: inherit }
+
+pre.literal-block, pre.doctest-block, pre.math, pre.code {
+ margin-left: 2em ;
+ margin-right: 2em }
+
+pre.code .ln { color: grey; } /* line numbers */
+pre.code, code { background-color: #eeeeee }
+pre.code .comment, code .comment { color: #5C6576 }
+pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
+pre.code .literal.string, code .literal.string { color: #0C5404 }
+pre.code .name.builtin, code .name.builtin { color: #352B84 }
+pre.code .deleted, code .deleted { background-color: #DEB0A1}
+pre.code .inserted, code .inserted { background-color: #A3D289}
+
+span.classifier {
+ font-family: sans-serif ;
+ font-style: oblique }
+
+span.classifier-delimiter {
+ font-family: sans-serif ;
+ font-weight: bold }
+
+span.interpreted {
+ font-family: sans-serif }
+
+span.option {
+ white-space: nowrap }
+
+span.pre {
+ white-space: pre }
+
+span.problematic {
+ color: red }
+
+span.section-subtitle {
+ /* font-size relative to parent (h1..h6 element) */
+ font-size: 80% }
+
+table.citation {
+ border-left: solid 1px gray;
+ margin-left: 1px }
+
+table.docinfo {
+ margin: 2em 4em }
+
+table.docutils {
+ margin-top: 0.5em ;
+ margin-bottom: 0.5em }
+
+table.footnote {
+ border-left: solid 1px black;
+ margin-left: 1px }
+
+table.docutils td, table.docutils th,
+table.docinfo td, table.docinfo th {
+ padding-left: 0.5em ;
+ padding-right: 0.5em ;
+ vertical-align: top }
+
+table.docutils th.field-name, table.docinfo th.docinfo-name {
+ font-weight: bold ;
+ text-align: left ;
+ white-space: nowrap ;
+ padding-left: 0 }
+
+/* "booktabs" style (no vertical lines) */
+table.docutils.booktabs {
+ border: 0px;
+ border-top: 2px solid;
+ border-bottom: 2px solid;
+ border-collapse: collapse;
+}
+table.docutils.booktabs * {
+ border: 0px;
+}
+table.docutils.booktabs th {
+ border-bottom: thin solid;
+ text-align: left;
+}
+
+h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
+h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
+ font-size: 100% }
+
+ul.auto-toc {
+ list-style-type: none }
+
+.slide li {
+ padding-top: 0.2em }
+
+</style>
+<!-- configuration parameters -->
+<meta name="defaultView" content="slideshow" />
+<meta name="controlVis" content="hidden" />
+<!-- style sheet links -->
+<script src="ui/default/slides.js" type="text/javascript"></script>
+<link rel="stylesheet" href="ui/default/slides.css"
+ type="text/css" media="projection" id="slideProj" />
+<link rel="stylesheet" href="ui/default/outline.css"
+ type="text/css" media="screen" id="outlineStyle" />
+<link rel="stylesheet" href="ui/default/print.css"
+ type="text/css" media="print" id="slidePrint" />
+<link rel="stylesheet" href="ui/default/opera.css"
+ type="text/css" media="projection" id="operaFix" />
+
+<style type="text/css">
+#currentSlide {display: none;}
+</style>
+</head>
+<body>
+<div class="layout">
+<div id="controls"></div>
+<div id="currentSlide"></div>
+<div id="header">
+
+</div>
+<div id="footer">
+<h1>Using All These Cores: Transactional Memory in PyPy</h1>
+
+</div>
+</div>
+<div class="presentation">
+<div class="slide" id="slide0">
+<h1 class="title">Using All These Cores: Transactional Memory in PyPy</h1>
+
+<center><p><strong>Armin Rigo - EuroPython 2014</strong></p>
+</center>
+</div>
+<div class="slide" id="part-1-intro-and-current-status">
+<h1>Part 1 - Intro and Current Status</h1>
+</div>
+<div class="slide" id="why-is-there-a-gil">
+<h1>Why is there a GIL?</h1>
+<ul class="simple">
+<li>GIL = Global Interpreter Lock</li>
+<li>initially: CPython was single threaded</li>
+<li>for concurrency (but not parallelism):<ul>
+<li>provide concurrently running threads</li>
+</ul>
+</li>
+<li>easiest way to retrofit into interpreter:<ul>
+<li>acquire GIL around the execution of bytecode instructions</li>
+<li>easy for refcounting, too</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="consequences">
+<h1>Consequences (+)</h1>
+<ul class="simple">
+<li>atomic & isolated instructions:<ul>
+<li>things like <tt class="docutils literal">list.append()</tt> are atomic</li>
+<li>tons of websites mention this</li>
+<li>latent races if Python becomes really parallel</li>
+</ul>
+</li>
+<li>sequential consistency:<ul>
+<li>less surprises; "all variables volatile"</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="id1">
+<h1>Consequences (-)</h1>
+<ul class="simple">
+<li>obviously, no parallelism</li>
+<li>GIL not available to application:<ul>
+<li>all difficulties of concurrency still there</li>
+<li>need application-level locking</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="removing-the-gil">
+<h1>Removing the GIL</h1>
+<ul class="simple">
+<li><ol class="first arabic">
+<li>Fine-grained locking</li>
+</ol>
+</li>
+<li><ol class="first arabic" start="2">
+<li>Shared-nothing</li>
+</ol>
+</li>
+<li><ol class="first arabic" start="3">
+<li>Transactional memory</li>
+</ol>
+</li>
+</ul>
+</div>
+<div class="slide" id="fine-grained-locking">
+<h1>Fine-grained locking</h1>
+<ul class="simple">
+<li>replace GIL with locks on objs / data structures</li>
+<li>accessing different objects can run in parallel</li>
+<li>harder to implement:<ul>
+<li>many locks -> deadlock risks</li>
+<li>refcounting issue</li>
+</ul>
+</li>
+<li>overhead of lock/unlock on objs:<ul>
+<li>Jython depends on JVM for good lock removal</li>
+</ul>
+</li>
+<li>still need application-level locking</li>
+</ul>
+</div>
+<div class="slide" id="shared-nothing">
+<h1>Shared-nothing</h1>
+<ul class="simple">
+<li>each independent part of the program gets its own interpreter</li>
+<li>simple implementation</li>
+<li>gives workaround instead of direct replacement</li>
+<li>not compatible to existing threaded applications, a priori</li>
+<li>explicit communication:<ul>
+<li>good: clean model, no locks</li>
+<li>bad: limitations, overhead</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="transactional-memory">
+<h1>Transactional Memory</h1>
+<ul class="simple">
+<li>like GIL, but instead of locking, each thread runs optimistically</li>
+<li>"easy" to implement:<ul>
+<li>GIL acquire -> transaction start</li>
+<li>GIL release -> transaction commit</li>
+</ul>
+</li>
+<li>overhead: cross-checking conflicting memory reads and writes,
+and if necessary, cancel and restart transactions</li>
+<li>HTM, STM, or some hybrids exist:<ul>
+<li>but mostly still research-only</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="big-point">
+<h1>Big Point</h1>
+<ul class="simple">
+<li>application-level locks still needed...</li>
+<li>but <em>can be very coarse:</em><ul>
+<li>the idea is to make sure, internally, that one transaction
+covers the whole time during which the lock was acquired</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="demo-1">
+<h1>Demo 1</h1>
+<ul class="simple">
+<li>Bottle web server</li>
+</ul>
+</div>
+<div class="slide" id="pypy-stm">
+<h1>PyPy-STM</h1>
+<ul class="simple">
+<li>implementation of a specially-tailored STM:<ul>
+<li>a reusable C library</li>
+<li>called STMGC-C7</li>
+</ul>
+</li>
+<li>used in PyPy to replace the GIL</li>
+<li>current status:<ul>
+<li>basics work</li>
+<li>tons of things to improve</li>
+<li>tons of things to improve</li>
+<li>tons of things to improve</li>
+<li>tons of things to improve</li>
+<li>tons of things to improve</li>
+<li>tons of things to improve</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="demo-2">
+<h1>Demo 2</h1>
+<ul class="simple">
+<li>counting primes</li>
+</ul>
+</div>
+<div class="slide" id="summary">
+<h1>Summary</h1>
+<ul class="simple">
+<li>Transactional Memory is still too researchy for production</li>
+<li>Has the potential to enable parallelism:<ul>
+<li>as a replacement of <tt class="docutils literal">multiprocessing</tt></li>
+<li>but also in existing applications not written for that</li>
+<li>as long as they do multiple things that are "often
independent"</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="part-2-under-the-hood">
+<h1>Part 2 - Under The Hood</h1>
+<ul class="simple">
+<li>pictures "GIL" and "no GIL"</li>
+<li>zoom with reads and writes</li>
+<li>keep boundaries, each block is a _transaction_</li>
+<li>completely the same semantics as when run with a GIL</li>
+<li>write-write conflict</li>
+<li>deadlock detection and resolution by abort-retry</li>
+<li>read-write conflict: avoids (1) crashes,
+(2) reads-from-the-past, (3) reads-from-the-future</li>
+<li>reads are more common than writes: optimize read barriers</li>
+<li>pypy-stm: write a thread-local flag "this object has been read",
+show code for read barrier and fast-path of write barrier;
+note about using the C library for CPython too</li>
+<li>reads are not synchronized at all between CPUs, but it's wrong
+to read data written by other in-progress transactions;
+so we have to write elsewhere</li>
+<li>but what if we read later an object we modified? doing any kind
+of check in the read barrier makes it much more costly</li>
+<li>a solution would be to give each thread its own "segment" of
+memory, and copy data between them only at known points</li>
+<li>mmap trick: we do that, but we use mmap sharing to view the same
+pages of memory at several addresses in memory</li>
+<li>show clang source code and assembler for %gs</li>
+<li>picture with 15/16 objects, 1/16 read markers, one page control data</li>
+<li>picture with nursery -- the GC can use the same write barrier</li>
+</ul>
+</div>
+</div>
+</body>
+</html>
diff --git a/talk/ep2014/stm/talk.rst b/talk/ep2014/stm/talk.rst
--- a/talk/ep2014/stm/talk.rst
+++ b/talk/ep2014/stm/talk.rst
@@ -2,63 +2,190 @@
Using All These Cores: Transactional Memory in PyPy
------------------------------------------------------------------------------
+.. raw:: html
-===========================================
+ <center>
+
+**Armin Rigo - EuroPython 2014**
+
+.. raw:: html
+
+ </center>
+
+
Part 1 - Intro and Current Status
-===========================================
+---------------------------------
-- stm/demo/
-- stm/bottle/
+Why is there a GIL?
+-------------------
-- transaction module; multiprocessing-like Pool(); etc.
+* GIL = Global Interpreter Lock
-- a large demo of some well-known program where
- it solves everything?... there is no such thing
- because the large program's author have already
- solved it
+* initially: CPython was single threaded
-- compare with garbage collection in C:
+* for concurrency (but not parallelism):
- - usually you do it with malloc()/free()
+ - provide concurrently running threads
- - sometimes you need more control, and e.g. you add
- some reference counts
+* easiest way to retrofit into interpreter:
- - sometimes you use more specialized versions for
- performance, e.g. allocate in a pool and throw it
- completely away at the end of some phase
+ - acquire GIL around the execution of bytecode instructions
- - Boehm GC, a GC for C: what kind of demo can you
- present for it? You take a C program, remove all
- free(), relink malloc() to Boehm, and it works
- more slowly...
+ - easy for refcounting, too
- - nevertheless, GCC did exactly that. Why?
-- so, the GIL: we already have different workarounds for
- different kinds of problems (use "multiprocessing"; or
- start N processes and have them communicate in one
- way or another)
+Consequences (+)
+----------------
-- this talk is about the GIL's equivalent of the Boehm GC
- for C: simplify your life for some problems, with a
- reasonable performance cost
+* atomic & isolated instructions:
-- this is about anything where the GIL is a blocker, obviously
+ - things like ``list.append()`` are atomic
+ - tons of websites mention this
+ - latent races if Python becomes really parallel
-- but also any program with "often-parallelizable" sections of
- code
+* sequential consistency:
-- there are actually a lot of such programs around when we think about
- it: Bottle, Twisted, Tornado, etc. serve multiple clients; many
- CPU-consuming programs at some point need to iterate over some
- large-ish data structure in some mostly-composable way; etc.
+ - less surprises; "all variables volatile"
-===========================================
+Consequences (-)
+----------------
+
+* obviously, no parallelism
+
+* GIL not available to application:
+
+ - all difficulties of concurrency still there
+ - need application-level locking
+
+
+Removing the GIL
+----------------
+
+* 1. Fine-grained locking
+
+* 2. Shared-nothing
+
+* 3. Transactional memory
+
+
+Fine-grained locking
+--------------------
+
+* replace GIL with locks on objs / data structures
+
+* accessing different objects can run in parallel
+
+* harder to implement:
+
+ - many locks -> deadlock risks
+ - refcounting issue
+
+* overhead of lock/unlock on objs:
+
+ - Jython depends on JVM for good lock removal
+
+* still need application-level locking
+
+
+Shared-nothing
+--------------
+
+* each independent part of the program gets its own interpreter
+
+* simple implementation
+
+* gives workaround instead of direct replacement
+
+* not compatible to existing threaded applications, a priori
+
+* explicit communication:
+
+ - good: clean model, no locks
+ - bad: limitations, overhead
+
+
+Transactional Memory
+--------------------
+
+* like GIL, but instead of locking, each thread runs optimistically
+
+* "easy" to implement:
+
+ - GIL acquire -> transaction start
+
+ - GIL release -> transaction commit
+
+* overhead: cross-checking conflicting memory reads and writes,
+ and if necessary, cancel and restart transactions
+
+* HTM, STM, or some hybrids exist:
+
+ - but mostly still research-only
+
+
+Big Point
+----------------------------
+
+* application-level locks still needed...
+
+* but *can be very coarse:*
+
+ - the idea is to make sure, internally, that one transaction
+ covers the whole time during which the lock was acquired
+
+
+Demo 1
+------
+
+* Bottle web server
+
+
+PyPy-STM
+--------
+
+* implementation of a specially-tailored STM:
+
+ - a reusable C library
+ - called STMGC-C7
+
+* used in PyPy to replace the GIL
+
+* current status:
+
+ - basics work
+ - tons of things to improve
+ - tons of things to improve
+ - tons of things to improve
+ - tons of things to improve
+ - tons of things to improve
+ - tons of things to improve
+
+
+Demo 2
+------
+
+* counting primes
+
+
+Summary
+-------
+
+* Transactional Memory is still too researchy for production
+
+* Has the potential to enable parallelism:
+
+ - as a replacement of ``multiprocessing``
+
+ - but also in existing applications not written for that
+
+ - as long as they do multiple things that are "often independent"
+
+
+
Part 2 - Under The Hood
-===========================================
+-----------------------
- pictures "GIL" and "no GIL"
diff --git a/talk/ep2014/stm/ui b/talk/ep2014/stm/ui
new file mode 120000
--- /dev/null
+++ b/talk/ep2014/stm/ui
@@ -0,0 +1,1 @@
+../../stanford-ee380-2011/ui
\ No newline at end of file
_______________________________________________
pypy-commit mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pypy-commit
