[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGfc410138939f: [lldb/Reproducers] Capture reproducers from 
the API test suite. (authored by JDevlieghere).
Herald added a project: LLDB.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588

Files:
  lldb/bindings/headers.swig
  lldb/bindings/interface/SBReproducer.i
  lldb/bindings/interfaces.swig
  lldb/include/lldb/API/SBReproducer.h
  lldb/packages/Python/lldbsuite/test/configuration.py
  lldb/packages/Python/lldbsuite/test/decorators.py
  lldb/packages/Python/lldbsuite/test/dotest.py
  lldb/packages/Python/lldbsuite/test/dotest_args.py
  lldb/source/API/SBReproducer.cpp
  lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
  lldb/test/API/lit.cfg.py
  lldb/test/API/lldbtest.py

Index: lldb/test/API/lldbtest.py
===
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import os
-
+import tempfile
 import subprocess
 import sys
 
@@ -67,14 +67,15 @@
 # python exe as the first parameter of the command.
 cmd = [sys.executable] + self.dotest_cmd + [testPath, '-p', testFile]
 
+builddir = getBuildDir(cmd)
+mkdir_p(builddir)
+
 # The macOS system integrity protection (SIP) doesn't allow injecting
 # libraries into system binaries, but this can be worked around by
 # copying the binary into a different location.
 if 'DYLD_INSERT_LIBRARIES' in test.config.environment and \
 (sys.executable.startswith('/System/') or \
 sys.executable.startswith('/usr/bin/')):
-builddir = getBuildDir(cmd)
-mkdir_p(builddir)
 copied_python = os.path.join(builddir, 'copied-system-python')
 if not os.path.isfile(copied_python):
 import shutil, subprocess
@@ -86,6 +87,16 @@
 shutil.copy(python, copied_python)
 cmd[0] = copied_python
 
+if 'lldb-repro-capture' in test.config.available_features or \
+   'lldb-repro-replay' in test.config.available_features:
+reproducer_root = os.path.join(builddir, 'reproducers')
+mkdir_p(reproducer_root)
+reproducer_path = os.path.join(reproducer_root, testFile)
+if 'lldb-repro-capture' in test.config.available_features:
+cmd.extend(['--capture-path', reproducer_path])
+else:
+cmd.extend(['--replay-path', reproducer_path])
+
 timeoutInfo = None
 try:
 out, err, exitCode = lit.util.executeCommand(
Index: lldb/test/API/lit.cfg.py
===
--- lldb/test/API/lit.cfg.py
+++ lldb/test/API/lit.cfg.py
@@ -60,6 +60,17 @@
   config.environment['LLDB_CAPTURE_REPRODUCER'] = os.environ[
   'LLDB_CAPTURE_REPRODUCER']
 
+# Support running the test suite under the lldb-repro wrapper. This makes it
+# possible to capture a test suite run and then rerun all the test from the
+# just captured reproducer.
+lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
+if lldb_repro_mode:
+  lit_config.note("Running API tests in {} mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':
+config.available_features.add('lldb-repro-capture')
+  elif lldb_repro_mode == 'replay':
+config.available_features.add('lldb-repro-replay')
+
 # Clean the module caches in the test build directory. This is necessary in an
 # incremental build whenever clang changes underneath, so doing it once per
 # lit.py invocation is close enough.
Index: lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
===
--- lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
+++ lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
@@ -20,6 +20,7 @@
 @skipIfWindows
 @skipIfRemote
 @skipIfiOSSimulator
+@skipIfReproducer
 def test_reproducer_attach(self):
 """Test thread creation after process attach."""
 exe = '%s_%d' % (self.testMethodName, os.getpid())
Index: lldb/source/API/SBReproducer.cpp
===
--- lldb/source/API/SBReproducer.cpp
+++ lldb/source/API/SBReproducer.cpp
@@ -124,6 +124,15 @@
   return nullptr;
 }
 
+const char *SBReproducer::PassiveReplay(const char *path) {
+  static std::string error;
+  if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
+error = llvm::toString(std::move(e));
+return error.c_str();
+  }
+  return nullptr;
+}
+
 const char *SBReproducer::Replay(const char *path) {
   return SBReproducer::Replay(path, false);
 }
Index: 

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Pavel Labath via Phabricator via lldb-commits]

labath accepted this revision.
labath added a comment.
This revision is now accepted and ready to land.

In D77588#1972418 , @JDevlieghere 
wrote:

> In D77588#1971345 , @labath wrote:
>
> > Ok, I'm starting to understand what's going on here. This patch looks 
> > mostly fine, but IIUC, it won't really do anything until the subsequent 
> > patch lands as that is what implements the inline/api/passive replay. I'm 
> > going to start looking at that other patch now.
>
>
> It's sufficient for stage one, i.e. capture the test suite and use the driver 
> (active replay), but it also contains the necessary changes for stage 2 
> (passive replay). I could've split it up in two patches but given that the 
> changes are so similar that didn't seem worth it.


Ah, ok. I can live with that.


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere updated this revision to Diff 256341.
JDevlieghere added a comment.

Update naming to passive replay.


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588

Files:
  lldb/bindings/headers.swig
  lldb/bindings/interface/SBReproducer.i
  lldb/bindings/interfaces.swig
  lldb/include/lldb/API/SBReproducer.h
  lldb/packages/Python/lldbsuite/test/configuration.py
  lldb/packages/Python/lldbsuite/test/decorators.py
  lldb/packages/Python/lldbsuite/test/dotest.py
  lldb/packages/Python/lldbsuite/test/dotest_args.py
  lldb/source/API/SBReproducer.cpp
  lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
  lldb/test/API/lit.cfg.py
  lldb/test/API/lldbtest.py

Index: lldb/test/API/lldbtest.py
===
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import os
-
+import tempfile
 import subprocess
 import sys
 
@@ -67,14 +67,15 @@
 # python exe as the first parameter of the command.
 cmd = [sys.executable] + self.dotest_cmd + [testPath, '-p', testFile]
 
+builddir = getBuildDir(cmd)
+mkdir_p(builddir)
+
 # The macOS system integrity protection (SIP) doesn't allow injecting
 # libraries into system binaries, but this can be worked around by
 # copying the binary into a different location.
 if 'DYLD_INSERT_LIBRARIES' in test.config.environment and \
 (sys.executable.startswith('/System/') or \
 sys.executable.startswith('/usr/bin/')):
-builddir = getBuildDir(cmd)
-mkdir_p(builddir)
 copied_python = os.path.join(builddir, 'copied-system-python')
 if not os.path.isfile(copied_python):
 import shutil, subprocess
@@ -86,6 +87,16 @@
 shutil.copy(python, copied_python)
 cmd[0] = copied_python
 
+if 'lldb-repro-capture' in test.config.available_features or \
+   'lldb-repro-replay' in test.config.available_features:
+reproducer_root = os.path.join(builddir, 'reproducers')
+mkdir_p(reproducer_root)
+reproducer_path = os.path.join(reproducer_root, testFile)
+if 'lldb-repro-capture' in test.config.available_features:
+cmd.extend(['--capture-path', reproducer_path])
+else:
+cmd.extend(['--replay-path', reproducer_path])
+
 timeoutInfo = None
 try:
 out, err, exitCode = lit.util.executeCommand(
Index: lldb/test/API/lit.cfg.py
===
--- lldb/test/API/lit.cfg.py
+++ lldb/test/API/lit.cfg.py
@@ -60,6 +60,17 @@
   config.environment['LLDB_CAPTURE_REPRODUCER'] = os.environ[
   'LLDB_CAPTURE_REPRODUCER']
 
+# Support running the test suite under the lldb-repro wrapper. This makes it
+# possible to capture a test suite run and then rerun all the test from the
+# just captured reproducer.
+lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
+if lldb_repro_mode:
+  lit_config.note("Running API tests in {} mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':
+config.available_features.add('lldb-repro-capture')
+  elif lldb_repro_mode == 'replay':
+config.available_features.add('lldb-repro-replay')
+
 # Clean the module caches in the test build directory. This is necessary in an
 # incremental build whenever clang changes underneath, so doing it once per
 # lit.py invocation is close enough.
Index: lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
===
--- lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
+++ lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
@@ -20,6 +20,7 @@
 @skipIfWindows
 @skipIfRemote
 @skipIfiOSSimulator
+@skipIfReproducer
 def test_reproducer_attach(self):
 """Test thread creation after process attach."""
 exe = '%s_%d' % (self.testMethodName, os.getpid())
Index: lldb/source/API/SBReproducer.cpp
===
--- lldb/source/API/SBReproducer.cpp
+++ lldb/source/API/SBReproducer.cpp
@@ -124,6 +124,15 @@
   return nullptr;
 }
 
+const char *SBReproducer::PassiveReplay(const char *path) {
+  static std::string error;
+  if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
+error = llvm::toString(std::move(e));
+return error.c_str();
+  }
+  return nullptr;
+}
+
 const char *SBReproducer::Replay(const char *path) {
   return SBReproducer::Replay(path, false);
 }
Index: lldb/packages/Python/lldbsuite/test/dotest_args.py
===
--- 

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere added a comment.

In D77588#1971345 , @labath wrote:

> Ok, I'm starting to understand what's going on here. This patch looks mostly 
> fine, but IIUC, it won't really do anything until the subsequent patch lands 
> as that is what implements the inline/api/passive replay. I'm going to start 
> looking at that other patch now.


It's sufficient for stage one, i.e. capture the test suite and use the driver 
(active replay), but it also contains the necessary changes for stage 2 
(passive replay). I could've split it up in two patches but given that the 
changes are so similar that didn't seem worth it.


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Pavel Labath via Phabricator via lldb-commits]

labath added a comment.

Ok, I'm starting to understand what's going on here. This patch looks mostly 
fine, but IIUC, it won't really do anything until the subsequent patch lands as 
that is what implements the inline/api/passive replay. I'm going to start 
looking at that other patch now.


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere added a comment.

In D77588#1968700 , @labath wrote:

> I think the code looks ok now, though I have to say I am still somewhat fuzzy 
> on what exactly will this patch enable. I think I understand the capture 
> part, but what happens during replay? Will that already be smart enough to 
> match the SB calls made by the test harness during replay to the captured 
> calls that were made during recording, or is there some more work needed 
> there?


That's implemented in https://reviews.llvm.org/D77602

> Could you add a brief documentation on this somewhere?

Sure thing: https://reviews.llvm.org/D1


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Pavel Labath via Phabricator via lldb-commits]

labath added a comment.

I think the code looks ok now, though I have to say I am still somewhat fuzzy 
on what exactly will this patch enable. I think I understand the capture part, 
but what happens during replay? Will that already be smart enough to match the 
SB calls made by the test harness during replay to the captured calls that were 
made during recording, or is there some more work needed there? Could you add a 
brief documentation on this somewhere?


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere updated this revision to Diff 255785.
JDevlieghere added a comment.

Address code review feedback


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588

Files:
  lldb/bindings/headers.swig
  lldb/bindings/interface/SBReproducer.i
  lldb/bindings/interfaces.swig
  lldb/include/lldb/API/SBReproducer.h
  lldb/packages/Python/lldbsuite/test/configuration.py
  lldb/packages/Python/lldbsuite/test/decorators.py
  lldb/packages/Python/lldbsuite/test/dotest.py
  lldb/packages/Python/lldbsuite/test/dotest_args.py
  lldb/source/API/SBReproducer.cpp
  lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
  lldb/test/API/lit.cfg.py
  lldb/test/API/lldbtest.py

Index: lldb/test/API/lldbtest.py
===
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import os
-
+import tempfile
 import subprocess
 import sys
 
@@ -67,14 +67,15 @@
 # python exe as the first parameter of the command.
 cmd = [sys.executable] + self.dotest_cmd + [testPath, '-p', testFile]
 
+builddir = getBuildDir(cmd)
+mkdir_p(builddir)
+
 # The macOS system integrity protection (SIP) doesn't allow injecting
 # libraries into system binaries, but this can be worked around by
 # copying the binary into a different location.
 if 'DYLD_INSERT_LIBRARIES' in test.config.environment and \
 (sys.executable.startswith('/System/') or \
 sys.executable.startswith('/usr/bin/')):
-builddir = getBuildDir(cmd)
-mkdir_p(builddir)
 copied_python = os.path.join(builddir, 'copied-system-python')
 if not os.path.isfile(copied_python):
 import shutil, subprocess
@@ -86,6 +87,16 @@
 shutil.copy(python, copied_python)
 cmd[0] = copied_python
 
+if 'lldb-repro-capture' in test.config.available_features or \
+   'lldb-repro-replay' in test.config.available_features:
+reproducer_root = os.path.join(builddir, 'reproducers')
+mkdir_p(reproducer_root)
+reproducer_path = os.path.join(reproducer_root, testFile)
+if 'lldb-repro-capture' in test.config.available_features:
+cmd.extend(['--capture-path', reproducer_path])
+else:
+cmd.extend(['--replay-path', reproducer_path])
+
 timeoutInfo = None
 try:
 out, err, exitCode = lit.util.executeCommand(
Index: lldb/test/API/lit.cfg.py
===
--- lldb/test/API/lit.cfg.py
+++ lldb/test/API/lit.cfg.py
@@ -60,6 +60,17 @@
   config.environment['LLDB_CAPTURE_REPRODUCER'] = os.environ[
   'LLDB_CAPTURE_REPRODUCER']
 
+# Support running the test suite under the lldb-repro wrapper. This makes it
+# possible to capture a test suite run and then rerun all the test from the
+# just captured reproducer.
+lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
+if lldb_repro_mode:
+  lit_config.note("Running API tests in {} mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':
+config.available_features.add('lldb-repro-capture')
+  elif lldb_repro_mode == 'replay':
+config.available_features.add('lldb-repro-replay')
+
 # Clean the module caches in the test build directory. This is necessary in an
 # incremental build whenever clang changes underneath, so doing it once per
 # lit.py invocation is close enough.
Index: lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
===
--- lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
+++ lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
@@ -20,6 +20,7 @@
 @skipIfWindows
 @skipIfRemote
 @skipIfiOSSimulator
+@skipIfReproducer
 def test_reproducer_attach(self):
 """Test thread creation after process attach."""
 exe = '%s_%d' % (self.testMethodName, os.getpid())
Index: lldb/source/API/SBReproducer.cpp
===
--- lldb/source/API/SBReproducer.cpp
+++ lldb/source/API/SBReproducer.cpp
@@ -124,6 +124,15 @@
   return nullptr;
 }
 
+const char *SBReproducer::APIReplay(const char *path) {
+  static std::string error;
+  if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
+error = llvm::toString(std::move(e));
+return error.c_str();
+  }
+  return nullptr;
+}
+
 const char *SBReproducer::Replay(const char *path) {
   return SBReproducer::Replay(path, false);
 }
Index: lldb/packages/Python/lldbsuite/test/dotest_args.py
===
--- 

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Pavel Labath via Phabricator via lldb-commits]

labath added inline comments.



Comment at: lldb/packages/Python/lldbsuite/test/decorators.py:861
+def is_reproducer():
+if ('LLDB_REPRODUCER_CAPTURE_PATH' in os.environ or 
'LLDB_REPRODUCER_REPLAY_PATH' in os.environ):
+return "reproducers unsupported"

Should this use variables from configuration.py now?



Comment at: lldb/test/API/lit.cfg.py:68
+if lldb_repro_mode:
+  lit_config.note("Running Shell test with lldb-repo in {} 
mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':

lldb-repro



Comment at: lldb/test/API/lldbtest.py:89
 
+reproducer_path = os.path.join(tempfile.gettempdir(), testFile)
+if 'lldb-repro-capture' in test.config.available_features:

I remember some objections to shoving everything in $TMP. Are we not able to 
put this in the build dir somewhere?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere updated this revision to Diff 255726.
JDevlieghere added a comment.

Update `lldbtest.py` too.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588

Files:
  lldb/bindings/headers.swig
  lldb/bindings/interface/SBReproducer.i
  lldb/bindings/interfaces.swig
  lldb/include/lldb/API/SBReproducer.h
  lldb/packages/Python/lldbsuite/test/configuration.py
  lldb/packages/Python/lldbsuite/test/decorators.py
  lldb/packages/Python/lldbsuite/test/dotest.py
  lldb/packages/Python/lldbsuite/test/dotest_args.py
  lldb/source/API/SBReproducer.cpp
  lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
  lldb/test/API/lit.cfg.py
  lldb/test/API/lldbtest.py

Index: lldb/test/API/lldbtest.py
===
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import os
-
+import tempfile
 import subprocess
 import sys
 
@@ -86,6 +86,12 @@
 shutil.copy(python, copied_python)
 cmd[0] = copied_python
 
+reproducer_path = os.path.join(tempfile.gettempdir(), testFile)
+if 'lldb-repro-capture' in test.config.available_features:
+cmd.extend(['--capture-path', reproducer_path])
+elif 'lldb-repro-replay' in test.config.available_features:
+cmd.extend(['--replay-path', reproducer_path])
+
 timeoutInfo = None
 try:
 out, err, exitCode = lit.util.executeCommand(
Index: lldb/test/API/lit.cfg.py
===
--- lldb/test/API/lit.cfg.py
+++ lldb/test/API/lit.cfg.py
@@ -60,6 +60,17 @@
   config.environment['LLDB_CAPTURE_REPRODUCER'] = os.environ[
   'LLDB_CAPTURE_REPRODUCER']
 
+# Support running the test suite under the lldb-repro wrapper. This makes it
+# possible to capture a test suite run and then rerun all the test from the
+# just captured reproducer.
+lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
+if lldb_repro_mode:
+  lit_config.note("Running Shell test with lldb-repo in {} mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':
+config.available_features.add('lldb-repro-capture')
+  elif lldb_repro_mode == 'replay':
+config.available_features.add('lldb-repro-replay')
+
 # Clean the module caches in the test build directory. This is necessary in an
 # incremental build whenever clang changes underneath, so doing it once per
 # lit.py invocation is close enough.
Index: lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
===
--- lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
+++ lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
@@ -20,6 +20,7 @@
 @skipIfWindows
 @skipIfRemote
 @skipIfiOSSimulator
+@skipIfReproducer
 def test_reproducer_attach(self):
 """Test thread creation after process attach."""
 exe = '%s_%d' % (self.testMethodName, os.getpid())
Index: lldb/source/API/SBReproducer.cpp
===
--- lldb/source/API/SBReproducer.cpp
+++ lldb/source/API/SBReproducer.cpp
@@ -124,6 +124,15 @@
   return nullptr;
 }
 
+const char *SBReproducer::APIReplay(const char *path) {
+  static std::string error;
+  if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
+error = llvm::toString(std::move(e));
+return error.c_str();
+  }
+  return nullptr;
+}
+
 const char *SBReproducer::Replay(const char *path) {
   return SBReproducer::Replay(path, false);
 }
Index: lldb/packages/Python/lldbsuite/test/dotest_args.py
===
--- lldb/packages/Python/lldbsuite/test/dotest_args.py
+++ lldb/packages/Python/lldbsuite/test/dotest_args.py
@@ -196,6 +196,17 @@
 metavar='platform-working-dir',
 help='The directory to use on the remote platform.')
 
+# Reproducer options
+group = parser.add_argument_group('Reproducer options')
+group.add_argument(
+'--capture-path',
+metavar='reproducer path',
+help='The reproducer capture path')
+group.add_argument(
+'--replay-path',
+metavar='reproducer path',
+help='The reproducer replay path')
+
 # Test-suite behaviour
 group = parser.add_argument_group('Runtime behaviour options')
 X('-d', 'Suspend the process after launch to wait indefinitely for a debugger to attach')
Index: lldb/packages/Python/lldbsuite/test/dotest.py
===
--- lldb/packages/Python/lldbsuite/test/dotest.py
+++ lldb/packages/Python/lldbsuite/test/dotest.py
@@ -429,6 +429,17 @@
 configuration.results_formatter_name = (
 

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere updated this revision to Diff 255718.
JDevlieghere added a comment.

Use `lldbconfig` to initialize reproducers.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588

Files:
  lldb/bindings/headers.swig
  lldb/bindings/interface/SBReproducer.i
  lldb/bindings/interfaces.swig
  lldb/include/lldb/API/SBReproducer.h
  lldb/packages/Python/lldbsuite/test/configuration.py
  lldb/packages/Python/lldbsuite/test/decorators.py
  lldb/packages/Python/lldbsuite/test/dotest.py
  lldb/packages/Python/lldbsuite/test/dotest_args.py
  lldb/source/API/SBReproducer.cpp
  lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
  lldb/test/API/lit.cfg.py
  lldb/test/API/lldbtest.py

Index: lldb/test/API/lldbtest.py
===
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import os
-
+import tempfile
 import subprocess
 import sys
 
@@ -86,6 +86,14 @@
 shutil.copy(python, copied_python)
 cmd[0] = copied_python
 
+reproducer_path = os.path.join(tempfile.gettempdir(), testFile)
+if 'lldb-repro-capture' in test.config.available_features:
+test.config.environment[
+'LLDB_REPRODUCER_CAPTURE_PATH'] = reproducer_path
+elif 'lldb-repro-replay' in test.config.available_features:
+test.config.environment[
+'LLDB_REPRODUCER_REPLAY_PATH'] = reproducer_path
+
 timeoutInfo = None
 try:
 out, err, exitCode = lit.util.executeCommand(
Index: lldb/test/API/lit.cfg.py
===
--- lldb/test/API/lit.cfg.py
+++ lldb/test/API/lit.cfg.py
@@ -60,6 +60,17 @@
   config.environment['LLDB_CAPTURE_REPRODUCER'] = os.environ[
   'LLDB_CAPTURE_REPRODUCER']
 
+# Support running the test suite under the lldb-repro wrapper. This makes it
+# possible to capture a test suite run and then rerun all the test from the
+# just captured reproducer.
+lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
+if lldb_repro_mode:
+  lit_config.note("Running Shell test with lldb-repo in {} mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':
+config.available_features.add('lldb-repro-capture')
+  elif lldb_repro_mode == 'replay':
+config.available_features.add('lldb-repro-replay')
+
 # Clean the module caches in the test build directory. This is necessary in an
 # incremental build whenever clang changes underneath, so doing it once per
 # lit.py invocation is close enough.
Index: lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
===
--- lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
+++ lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
@@ -20,6 +20,7 @@
 @skipIfWindows
 @skipIfRemote
 @skipIfiOSSimulator
+@skipIfReproducer
 def test_reproducer_attach(self):
 """Test thread creation after process attach."""
 exe = '%s_%d' % (self.testMethodName, os.getpid())
Index: lldb/source/API/SBReproducer.cpp
===
--- lldb/source/API/SBReproducer.cpp
+++ lldb/source/API/SBReproducer.cpp
@@ -124,6 +124,15 @@
   return nullptr;
 }
 
+const char *SBReproducer::APIReplay(const char *path) {
+  static std::string error;
+  if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
+error = llvm::toString(std::move(e));
+return error.c_str();
+  }
+  return nullptr;
+}
+
 const char *SBReproducer::Replay(const char *path) {
   return SBReproducer::Replay(path, false);
 }
Index: lldb/packages/Python/lldbsuite/test/dotest_args.py
===
--- lldb/packages/Python/lldbsuite/test/dotest_args.py
+++ lldb/packages/Python/lldbsuite/test/dotest_args.py
@@ -196,6 +196,17 @@
 metavar='platform-working-dir',
 help='The directory to use on the remote platform.')
 
+# Reproducer options
+group = parser.add_argument_group('Reproducer options')
+group.add_argument(
+'--capture-path',
+metavar='reproducer path',
+help='The reproducer capture path')
+group.add_argument(
+'--replay-path',
+metavar='reproducer path',
+help='The reproducer replay path')
+
 # Test-suite behaviour
 group = parser.add_argument_group('Runtime behaviour options')
 X('-d', 'Suspend the process after launch to wait indefinitely for a debugger to attach')
Index: lldb/packages/Python/lldbsuite/test/dotest.py
===
--- lldb/packages/Python/lldbsuite/test/dotest.py
+++ lldb/packages/Python/lldbsuite/test/dotest.py
@@ -429,6 +429,17 @@
 

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Pavel Labath via Phabricator via lldb-commits]

labath added inline comments.



Comment at: lldb/bindings/python.swig:132-136
+if 'LLDB_REPRODUCER_CAPTURE_PATH' in os.environ:
+   SBReproducer.Capture(os.environ['LLDB_REPRODUCER_CAPTURE_PATH'])
+   SBReproducer.SetAutoGenerate(True)
+elif 'LLDB_REPRODUCER_REPLAY_PATH' in os.environ:
+   SBReproducer.APIReplay(os.environ['LLDB_REPRODUCER_REPLAY_PATH'])

I am doubly unhappy about this. Not only does it leak testing logic into 
production code, it actually does that using environment variables... :/

I take it the problem here is that reproducers need to be set up before the 
Initialize call, but the lldb module does not let us do that as it calls 
Initialize automatically? I am not sure that was such a wise idea, but that 
ship has sailed a long time ago.

So, can we at least do something like [[ 
https://stackoverflow.com/questions/3720740/pass-variable-on-import/39360070#39360070
 | this ]] instead? I.e. have a separate configuration module which would be 
imported here and would drive this logic.

I believe it should be possible to arrange it so that the configuration module 
does not even have to get shipped -- this code could just attempt the import 
and ignore any errors.

And the configuration module itself would not need to do anything 
reproducer-related either. It could just contain a single flag to disable the 
automatic initialization. That sounds like a thing someone might conceivably 
want anyway, and it could be later extended to handle other kinds of global 
lldb configuration (ConstString pool size maybe ?).

Once the Initialize call is disabled, the normal test harness can handle the 
rest...


Repository:
  rLLDB LLDB

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D77588/new/

https://reviews.llvm.org/D77588



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

[Lldb-commits] [PATCH] D77588: [lldb/Reproducers] Make it possible to capture reproducers for the Python test suite.

[Jonas Devlieghere via Phabricator via lldb-commits]

JDevlieghere created this revision.
JDevlieghere added a reviewer: labath.

This adds the necessary boiler plate to capture and replay API tests.

http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html


Repository:
  rLLDB LLDB

https://reviews.llvm.org/D77588

Files:
  lldb/bindings/headers.swig
  lldb/bindings/interface/SBReproducer.i
  lldb/bindings/interfaces.swig
  lldb/bindings/python.swig
  lldb/include/lldb/API/SBReproducer.h
  lldb/packages/Python/lldbsuite/test/decorators.py
  lldb/source/API/SBReproducer.cpp
  lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
  lldb/test/API/lit.cfg.py
  lldb/test/API/lldbtest.py

Index: lldb/test/API/lldbtest.py
===
--- lldb/test/API/lldbtest.py
+++ lldb/test/API/lldbtest.py
@@ -1,6 +1,6 @@
 from __future__ import absolute_import
 import os
-
+import tempfile
 import subprocess
 import sys
 
@@ -86,6 +86,14 @@
 shutil.copy(python, copied_python)
 cmd[0] = copied_python
 
+reproducer_path = os.path.join(tempfile.gettempdir(), testFile)
+if 'lldb-repro-capture' in test.config.available_features:
+test.config.environment[
+'LLDB_REPRODUCER_CAPTURE_PATH'] = reproducer_path
+elif 'lldb-repro-replay' in test.config.available_features:
+test.config.environment[
+'LLDB_REPRODUCER_REPLAY_PATH'] = reproducer_path
+
 timeoutInfo = None
 try:
 out, err, exitCode = lit.util.executeCommand(
Index: lldb/test/API/lit.cfg.py
===
--- lldb/test/API/lit.cfg.py
+++ lldb/test/API/lit.cfg.py
@@ -60,6 +60,17 @@
   config.environment['LLDB_CAPTURE_REPRODUCER'] = os.environ[
   'LLDB_CAPTURE_REPRODUCER']
 
+# Support running the test suite under the lldb-repro wrapper. This makes it
+# possible to capture a test suite run and then rerun all the test from the
+# just captured reproducer.
+lldb_repro_mode = lit_config.params.get('lldb-run-with-repro', None)
+if lldb_repro_mode:
+  lit_config.note("Running Shell test with lldb-repo in {} mode.".format(lldb_repro_mode))
+  if lldb_repro_mode == 'capture':
+config.available_features.add('lldb-repro-capture')
+  elif lldb_repro_mode == 'replay':
+config.available_features.add('lldb-repro-replay')
+
 # Clean the module caches in the test build directory. This is necessary in an
 # incremental build whenever clang changes underneath, so doing it once per
 # lit.py invocation is close enough.
Index: lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
===
--- lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
+++ lldb/test/API/functionalities/reproducers/attach/TestReproducerAttach.py
@@ -20,6 +20,7 @@
 @skipIfWindows
 @skipIfRemote
 @skipIfiOSSimulator
+@skipIfReproducer
 def test_reproducer_attach(self):
 """Test thread creation after process attach."""
 exe = '%s_%d' % (self.testMethodName, os.getpid())
Index: lldb/source/API/SBReproducer.cpp
===
--- lldb/source/API/SBReproducer.cpp
+++ lldb/source/API/SBReproducer.cpp
@@ -124,6 +124,15 @@
   return nullptr;
 }
 
+const char *SBReproducer::APIReplay(const char *path) {
+  static std::string error;
+  if (auto e = Reproducer::Initialize(ReproducerMode::Replay, FileSpec(path))) {
+error = llvm::toString(std::move(e));
+return error.c_str();
+  }
+  return nullptr;
+}
+
 const char *SBReproducer::Replay(const char *path) {
   return SBReproducer::Replay(path, false);
 }
Index: lldb/packages/Python/lldbsuite/test/decorators.py
===
--- lldb/packages/Python/lldbsuite/test/decorators.py
+++ lldb/packages/Python/lldbsuite/test/decorators.py
@@ -854,3 +854,11 @@
 return "ASAN unsupported"
 return None
 return skipTestIfFn(is_asan)(func)
+
+def skipIfReproducer(func):
+"""Skip this test if the environment is set up to run LLDB with reproducers."""
+def is_reproducer():
+if ('LLDB_REPRODUCER_CAPTURE_PATH' in os.environ or 'LLDB_REPRODUCER_REPLAY_PATH' in os.environ):
+return "reproducers unsupported"
+return None
+return skipTestIfFn(is_reproducer)(func)
Index: lldb/include/lldb/API/SBReproducer.h
===
--- lldb/include/lldb/API/SBReproducer.h
+++ lldb/include/lldb/API/SBReproducer.h
@@ -22,6 +22,7 @@
   static const char *Capture(const char *path);
   static const char *Replay(const char *path);
   static const char *Replay(const char *path, bool skip_version_check);
+  static const char *APIReplay(const char *path);
   static const char *GetPath();
   static bool SetAutoGenerate(bool b);
   static