Re: meson vs. llvm bitcode files
Hi,
On Thu, 13 Mar 2025 at 13:11, Nazir Bilal Yavuz wrote:
> One thing I noticed is that gen_srcfiles['srcfiles'] seems wrong.
> gen_sources is a better name compared to gen_srcfiles. So, I changed
> it to gen_sources in v4.
Rebase is needed due to b1720fe63f, v5 is attached.
--
Regards,
Nazir Bilal Yavuz
Microsoft
From 8965a194e7449c4731c896941994500565fb6f26 Mon Sep 17 00:00:00 2001
From: Andres Freund
Date: Thu, 24 Oct 2024 07:23:05 -0400
Subject: [PATCH v5 1/7] meson: Add generated header stamps
Otherwise build commands become too long and this has visible effect on
creation time of meson build files.
Author: Andres Freund
Author: Nazir Bilal Yavuz
Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org
---
src/include/meson.build | 18 ++
src/backend/meson.build | 2 +-
src/fe_utils/meson.build | 2 +-
meson.build | 16 +---
4 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/src/include/meson.build b/src/include/meson.build
index 2e4b7aa529e..c488a5dc4c9 100644
--- a/src/include/meson.build
+++ b/src/include/meson.build
@@ -177,3 +177,21 @@ install_subdir('catalog',
# autoconf generates the file there, ensure we get a conflict
generated_sources_ac += {'src/include': ['stamp-h']}
+
+# Instead of having targets depending directly on the generated headers, have
+# them depend on a stamp files for all of them. Dependencies on headers are
+# implemented as order-only dependencies in meson (later using compiler
+# generated dependencies). The benefit of using a stamp file is that it makes
+# ninja.build smaller and meson setup faster.
+generated_headers_stamp = custom_target('generated-headers-stamp.h',
+ output: 'generated-headers-stamp.h',
+ input: generated_headers,
+ command: stamp_cmd,
+)
+
+generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h',
+ output: 'generated-backend-headers-stamp.h',
+ input: generated_backend_headers,
+ depends: generated_headers_stamp,
+ command: stamp_cmd,
+)
diff --git a/src/backend/meson.build b/src/backend/meson.build
index 2b0db214804..7fc649c3ebd 100644
--- a/src/backend/meson.build
+++ b/src/backend/meson.build
@@ -169,7 +169,7 @@ backend_mod_code = declare_dependency(
compile_args: pg_mod_c_args,
include_directories: postgres_inc,
link_args: pg_mod_link_args,
- sources: generated_headers + generated_backend_headers,
+ sources: [generated_backend_headers_stamp],
dependencies: backend_mod_deps,
)
diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build
index a18cbc939e4..5a9ddb73463 100644
--- a/src/fe_utils/meson.build
+++ b/src/fe_utils/meson.build
@@ -29,7 +29,7 @@ generated_sources += psqlscan
fe_utils_sources += psqlscan
fe_utils = static_library('libpgfeutils',
- fe_utils_sources + generated_headers,
+ fe_utils_sources,
c_pch: pch_postgres_fe_h,
include_directories: [postgres_inc, libpq_inc],
c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [],
diff --git a/meson.build b/meson.build
index a1516e54529..35cf7d8b948 100644
--- a/meson.build
+++ b/meson.build
@@ -3105,6 +3105,8 @@ gen_export_kwargs = {
'install': false,
}
+# command to create stamp files on all OSs
+stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@']
###
@@ -3222,14 +3224,14 @@ subdir('src/port')
frontend_common_code = declare_dependency(
compile_args: ['-DFRONTEND'],
include_directories: [postgres_inc],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [os_deps, zlib, zstd, lz4],
)
backend_common_code = declare_dependency(
compile_args: ['-DBUILDING_DLL'],
include_directories: [postgres_inc],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [os_deps, zlib, zstd],
)
@@ -3244,7 +3246,7 @@ shlib_code = declare_dependency(
frontend_stlib_code = declare_dependency(
include_directories: [postgres_inc],
link_with: [common_static, pgport_static],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [os_deps, libintl],
)
@@ -3252,7 +3254,7 @@ frontend_stlib_code = declare_dependency(
frontend_shlib_code = declare_dependency(
include_directories: [postgres_inc],
link_with: [common_shlib, pgport_shlib],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [shlib_code, os_deps, libintl],
)
@@ -3262,7 +3264,7 @@ frontend_shlib_code = declare_dependency(
frontend_no_fe_utils_code = declare_dependency(
include_directories: [postgres_inc],
link_with: [common_static, pgport_static],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [os_deps, libintl],
)
@@ -3288,7 +3290,7 @@ subdir('src/fe_utils')
frontend_code = declare_dependency(
include_directories: [postgres_inc],
link_with: [fe_utils, common_static, pgport_static],
- sources: generated_headers,
+ sources: generate
Re: meson vs. llvm bitcode files
Hi,
On Wed, 12 Mar 2025 at 16:39, Diego Fronza wrote:
>
> Hi,
>
> The v7 patch looks good to me, handling the bitcode modules in a uniform way
> and also avoiding the hacky code and warnings, much better now.
>
> A small note about the bitcode emission for generated sources in contrib,
> using cube as example, currently it creates two dict entries in a list:
> bc_seg_gen_sources = [{'srcfiles': [seg_scan]}]
> bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]}
>
> Then pass it to the bitcode_modules:
> bitcode_modules += {
> ...
> 'gen_srcfiles': bc_seg_gen_sources,
> }
>
> It could be passed as a list with a single dict, since both generated sources
> share the same compilation flags:
> bitcode_modules += {
> ...
> 'gen_srcfiles': [
> { 'srcfiles': [cube_scan, cube_parse[0]] }.
> ]
> }
>
> Both approaches work, the first one has the advantage of being able to pass
> separate additional_flags per generated source.
I liked the current approach as it makes bitcode_modules easier to
understand but both approaches work for me as well.
One thing I noticed is that gen_srcfiles['srcfiles'] seems wrong.
gen_sources is a better name compared to gen_srcfiles. So, I changed
it to gen_sources in v4.
--
Regards,
Nazir Bilal Yavuz
Microsoft
From 578fbb407ce89293c8c955179e61b10aadc1709f Mon Sep 17 00:00:00 2001
From: Andres Freund
Date: Thu, 24 Oct 2024 07:23:05 -0400
Subject: [PATCH v4 1/7] meson: Add generated header stamps
Otherwise build commands become too long and this has visible effect on
creation time of meson build files.
Author: Andres Freund
Author: Nazir Bilal Yavuz
Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org
---
src/include/meson.build | 18 ++
src/backend/meson.build | 2 +-
src/fe_utils/meson.build | 2 +-
meson.build | 16 +---
4 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/src/include/meson.build b/src/include/meson.build
index 2e4b7aa529e..c488a5dc4c9 100644
--- a/src/include/meson.build
+++ b/src/include/meson.build
@@ -177,3 +177,21 @@ install_subdir('catalog',
# autoconf generates the file there, ensure we get a conflict
generated_sources_ac += {'src/include': ['stamp-h']}
+
+# Instead of having targets depending directly on the generated headers, have
+# them depend on a stamp files for all of them. Dependencies on headers are
+# implemented as order-only dependencies in meson (later using compiler
+# generated dependencies). The benefit of using a stamp file is that it makes
+# ninja.build smaller and meson setup faster.
+generated_headers_stamp = custom_target('generated-headers-stamp.h',
+ output: 'generated-headers-stamp.h',
+ input: generated_headers,
+ command: stamp_cmd,
+)
+
+generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h',
+ output: 'generated-backend-headers-stamp.h',
+ input: generated_backend_headers,
+ depends: generated_headers_stamp,
+ command: stamp_cmd,
+)
diff --git a/src/backend/meson.build b/src/backend/meson.build
index 2b0db214804..7fc649c3ebd 100644
--- a/src/backend/meson.build
+++ b/src/backend/meson.build
@@ -169,7 +169,7 @@ backend_mod_code = declare_dependency(
compile_args: pg_mod_c_args,
include_directories: postgres_inc,
link_args: pg_mod_link_args,
- sources: generated_headers + generated_backend_headers,
+ sources: [generated_backend_headers_stamp],
dependencies: backend_mod_deps,
)
diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build
index a18cbc939e4..5a9ddb73463 100644
--- a/src/fe_utils/meson.build
+++ b/src/fe_utils/meson.build
@@ -29,7 +29,7 @@ generated_sources += psqlscan
fe_utils_sources += psqlscan
fe_utils = static_library('libpgfeutils',
- fe_utils_sources + generated_headers,
+ fe_utils_sources,
c_pch: pch_postgres_fe_h,
include_directories: [postgres_inc, libpq_inc],
c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [],
diff --git a/meson.build b/meson.build
index 13c13748e5d..9bfa96ad255 100644
--- a/meson.build
+++ b/meson.build
@@ -2973,6 +2973,8 @@ gen_export_kwargs = {
'install': false,
}
+# command to create stamp files on all OSs
+stamp_cmd = [python, '-c', 'import sys; open(sys.argv[1], "w")', '@OUTPUT0@']
###
@@ -3090,14 +3092,14 @@ subdir('src/port')
frontend_common_code = declare_dependency(
compile_args: ['-DFRONTEND'],
include_directories: [postgres_inc],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [os_deps, zlib, zstd, lz4],
)
backend_common_code = declare_dependency(
compile_args: ['-DBUILDING_DLL'],
include_directories: [postgres_inc],
- sources: generated_headers,
+ sources: generated_headers_stamp,
dependencies: [os_deps, zlib, zstd],
)
@@ -3112,7 +3114,7 @@ shlib_code = declare_dependency(
frontend_stlib_code = declare_dependency(
include_directories: [postgres_inc],
link_with: [common_static, pgport_static],
- sources:
Re: meson vs. llvm bitcode files
Hi,
The v7 patch looks good to me, handling the bitcode modules in a uniform
way and also avoiding the hacky code and warnings, much better now.
A small note about the bitcode emission for generated sources in contrib,
using cube as example, currently it creates two dict entries in a list:
bc_seg_gen_sources = [{'srcfiles': [seg_scan]}]
bc_seg_gen_sources += {'srcfiles': [seg_parse[0]]}
Then pass it to the bitcode_modules:
bitcode_modules += {
...
'gen_srcfiles': bc_seg_gen_sources,
}
It could be passed as a list with a single dict, since both generated
sources share the same compilation flags:
bitcode_modules += {
...
'gen_srcfiles': [
{ 'srcfiles': [cube_scan, cube_parse[0]] }.
]
}
Both approaches work, the first one has the advantage of being able to pass
separate additional_flags per generated source.
Thanks for your reply Nazir, also waiting for more opinions on this.
Regards,
Diego
On Wed, Mar 12, 2025 at 7:27 AM Nazir Bilal Yavuz
wrote:
> Hi,
>
> On Tue, 11 Mar 2025 at 01:04, Diego Fronza
> wrote:
> > I did a full review on the provided patches plus some tests, I was able
> to validate that the loading of bitcode modules is working also JIT works
> for both backend and contrib modules.
>
> Thank you!
>
> > To test JIT on contrib modules I just lowered the costs for all jit
> settings and used the intarray extension, using the data/test__int.data:
> > CREATE EXTENSION intarray;
> > CREATE TABLE test__int( a int[] );1
> > \copy test__int from 'data/test__int.data'
> >
> > For queries any from line 98+ on contrib/intarray/sql/_int.sql will work.
> >
> > Then I added extra debug messages to llvmjit_inline.cpp on
> add_module_to_inline_search_path() function, also on
> llvm_build_inline_plan(), I was able to see many functions in this module
> being successfully inlined.
> >
> > I'm attaching a new patch based on your original work which add further
> support for generating bitcode from:
>
> Thanks for doing that!
>
> > - Generated backend sources: processed by flex, bison, etc.
> > - Generated contrib module sources,
>
> I think we do not need to separate these two.
>
>foreach srcfile : bitcode_module['srcfiles']
> -if meson.version().version_compare('>=0.59')
> +srcfilename = '@0@'.format(srcfile)
> +if srcfilename.startswith(' + srcfilename = srcfile.full_path().split(meson.build_root() + '/')[1]
> +elif meson.version().version_compare('>=0.59')
>
> Also, checking if the string starts with ' hacky, and 'srcfilename = '@0@'.format(srcfile)' causes a deprecation
> warning. So, instead of this we can process all generated sources like
> how generated backend sources are processed. I updated the patch with
> that.
>
> > On this patch I just included fmgrtab.c and src/backend/parser for the
> backend generated code.
> > For contrib generated sources I added contrib/cube as an example.
>
> I applied your contrib/cube example and did the same thing for the
> contrib/seg.
>
> > All relevant details about the changes are included in the patch itself.
> >
> > As you may know already I also created a PR focused on llvm bitcode
> emission on meson, it generates bitcode for all backend and contribution
> modules, currently under review by some colleagues at Percona:
> https://github.com/percona/postgres/pull/103
> > I'm curious if we should get all or some of the generated backend
> sources compiled to bitcode, similar to contrib modules.
>
> I think we can do this. I added other backend sources like you did in
> the PR but attached it as another patch (0007) because I wanted to
> hear other people's opinions on that first.
>
> v3 is attached.
>
> --
> Regards,
> Nazir Bilal Yavuz
> Microsoft
>
Re: meson vs. llvm bitcode files
Hi,
On Tue, 11 Mar 2025 at 01:04, Diego Fronza wrote:
> I did a full review on the provided patches plus some tests, I was able to
> validate that the loading of bitcode modules is working also JIT works for
> both backend and contrib modules.
Thank you!
> To test JIT on contrib modules I just lowered the costs for all jit settings
> and used the intarray extension, using the data/test__int.data:
> CREATE EXTENSION intarray;
> CREATE TABLE test__int( a int[] );1
> \copy test__int from 'data/test__int.data'
>
> For queries any from line 98+ on contrib/intarray/sql/_int.sql will work.
>
> Then I added extra debug messages to llvmjit_inline.cpp on
> add_module_to_inline_search_path() function, also on
> llvm_build_inline_plan(), I was able to see many functions in this module
> being successfully inlined.
>
> I'm attaching a new patch based on your original work which add further
> support for generating bitcode from:
Thanks for doing that!
> - Generated backend sources: processed by flex, bison, etc.
> - Generated contrib module sources,
I think we do not need to separate these two.
foreach srcfile : bitcode_module['srcfiles']
-if meson.version().version_compare('>=0.59')
+srcfilename = '@0@'.format(srcfile)
+if srcfilename.startswith('=0.59')
Also, checking if the string starts with ' On this patch I just included fmgrtab.c and src/backend/parser for the
> backend generated code.
> For contrib generated sources I added contrib/cube as an example.
I applied your contrib/cube example and did the same thing for the contrib/seg.
> All relevant details about the changes are included in the patch itself.
>
> As you may know already I also created a PR focused on llvm bitcode emission
> on meson, it generates bitcode for all backend and contribution modules,
> currently under review by some colleagues at Percona:
> https://github.com/percona/postgres/pull/103
> I'm curious if we should get all or some of the generated backend sources
> compiled to bitcode, similar to contrib modules.
I think we can do this. I added other backend sources like you did in
the PR but attached it as another patch (0007) because I wanted to
hear other people's opinions on that first.
v3 is attached.
--
Regards,
Nazir Bilal Yavuz
Microsoft
From 62c7c05b19476939941b656a6eab43dc3661ef09 Mon Sep 17 00:00:00 2001
From: Andres Freund
Date: Thu, 24 Oct 2024 07:23:05 -0400
Subject: [PATCH v3 1/7] meson: Add generated header stamps
Otherwise build commands become too long and this has visible effect on
creation time of meson build files.
Author: Andres Freund
Author: Nazir Bilal Yavuz
Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org
---
src/include/meson.build | 18 ++
src/backend/meson.build | 2 +-
src/fe_utils/meson.build | 2 +-
meson.build | 16 +---
4 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/src/include/meson.build b/src/include/meson.build
index 2e4b7aa529e..c488a5dc4c9 100644
--- a/src/include/meson.build
+++ b/src/include/meson.build
@@ -177,3 +177,21 @@ install_subdir('catalog',
# autoconf generates the file there, ensure we get a conflict
generated_sources_ac += {'src/include': ['stamp-h']}
+
+# Instead of having targets depending directly on the generated headers, have
+# them depend on a stamp files for all of them. Dependencies on headers are
+# implemented as order-only dependencies in meson (later using compiler
+# generated dependencies). The benefit of using a stamp file is that it makes
+# ninja.build smaller and meson setup faster.
+generated_headers_stamp = custom_target('generated-headers-stamp.h',
+ output: 'generated-headers-stamp.h',
+ input: generated_headers,
+ command: stamp_cmd,
+)
+
+generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h',
+ output: 'generated-backend-headers-stamp.h',
+ input: generated_backend_headers,
+ depends: generated_headers_stamp,
+ command: stamp_cmd,
+)
diff --git a/src/backend/meson.build b/src/backend/meson.build
index 2b0db214804..7fc649c3ebd 100644
--- a/src/backend/meson.build
+++ b/src/backend/meson.build
@@ -169,7 +169,7 @@ backend_mod_code = declare_dependency(
compile_args: pg_mod_c_args,
include_directories: postgres_inc,
link_args: pg_mod_link_args,
- sources: generated_headers + generated_backend_headers,
+ sources: [generated_backend_headers_stamp],
dependencies: backend_mod_deps,
)
diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build
index a18cbc939e4..5a9ddb73463 100644
--- a/src/fe_utils/meson.build
+++ b/src/fe_utils/meson.build
@@ -29,7 +29,7 @@ generated_sources += psqlscan
fe_utils_sources += psqlscan
fe_utils = static_library('libpgfeutils',
- fe_utils_sources + generated_headers,
+ fe_utils_sources,
c_pch: pch_postgres_fe_h,
include_directories: [postgres_inc, libpq_inc],
c_args: host_system == 'windows' ? ['-DFD_SETSIZE=1024'] : [],
di
Re: meson vs. llvm bitcode files
Hello, I did a full review on the provided patches plus some tests, I was able to validate that the loading of bitcode modules is working also JIT works for both backend and contrib modules. To test JIT on contrib modules I just lowered the costs for all jit settings and used the intarray extension, using the data/test__int.data: CREATE EXTENSION intarray; CREATE TABLE test__int( a int[] );1 \copy test__int from 'data/test__int.data' For queries any from line 98+ on contrib/intarray/sql/_int.sql will work. Then I added extra debug messages to llvmjit_inline.cpp on add_module_to_inline_search_path() function, also on llvm_build_inline_plan(), I was able to see many functions in this module being successfully inlined. I'm attaching a new patch based on your original work which add further support for generating bitcode from: - Generated backend sources: processed by flex, bison, etc. - Generated contrib module sources, On this patch I just included fmgrtab.c and src/backend/parser for the backend generated code. For contrib generated sources I added contrib/cube as an example. All relevant details about the changes are included in the patch itself. As you may know already I also created a PR focused on llvm bitcode emission on meson, it generates bitcode for all backend and contribution modules, currently under review by some colleagues at Percona: https://github.com/percona/postgres/pull/103 I'm curious if we should get all or some of the generated backend sources compiled to bitcode, similar to contrib modules. Please let me know your thoughts and how we can proceed to get this feature included, thank you. Regards, Diego Fronza Percona On Fri, Mar 7, 2025 at 7:52 AM Nazir Bilal Yavuz wrote: > Hi, > > On Thu, 5 Sept 2024 at 12:24, Nazir Bilal Yavuz > wrote: > > > > I found that Andres shared a patch > > (v17-0021-meson-Add-LLVM-bitcode-emission.patch) a while ago [1]. > > Andres and I continued to work on that. I think the patches are in > sharable state now and I wanted to hear opinions before proceeding > further. After applying the patches, bitcode files should be installed > into $pkglibdir/bitcode/ directory if the llvm is found. > > There are 6 patches attached: > > v1-0001-meson-Add-generated-header-stamps: > > This patch is trivial. Instead of having targets depending directly on > the generated headers, have them depend on a stamp file. The benefit > of using a stamp file is that it makes ninja.build smaller and meson > setup faster. > -- > > v1-0002-meson-Add-postgresql-extension.pc-for-building-extension-libraries: > > This patch is for generating postgresql-extension.pc file which can be > used for building extensions libraries. > > Normally, there is no need to use this .pc file for generating bitcode > files. However, since there is no clear way to get all include paths > for building bitcode files, this .pc file is later used for this > purpose (by running pkg-config --cflags-only-I > postgresql-extension-uninstalled.pc) [1]. > -- > > v1-0003-meson-Test-building-extensions-by-using-postgresql-extension.pc: > [Not needed for generating bitcode files] > > This is a patch for testing if extensions can be built by using > postgresql-extension.pc. I added that commit as an example of using > postgresql-extension.pc to build extensions. > -- > > v1-0004-meson-WIP-Add-docs-for-postgresql-extension.pc: [Not needed > for generating bitcode files] > > I added this patch in case we recommend people to use > postgresql-extension.pc to build extension libraries. I am not sure if > we want to do that because there are still TODOs about > postgresql-extension.pc like running test suites. I just wanted to > show my plan, dividing 'Extension Building Infrastructure' into two, > 'PGXS' and 'postgresql-extension.pc'. > -- > > v1-0005-meson-Add-LLVM-bitcode-emission: > > This patch adds required infrastructure to generate bitcode files and > uses postgresql-extension-uninstalled.pc to get include paths for > generating bitcode files [1]. > -- > > v1-0006-meson-Generate-bitcode-files-of-contrib-extension.patch: > > This patch adds manually selected contrib libraries to generate their > bitcode files. These libraries are selected manually, depending on > - If they have SQL callable functions > - If the library functions are short enough (the performance gain from > bitcode files is too minimal compared to the function's run time, so > this type of libraries are omitted). > > Any kind of feedback would be appreciated. > > -- > Regards, > Nazir Bilal Yavuz > Microsoft > From be09b1f38d107789fc9ffe1cd2b2470552689a20 Mon Sep 17 00:00:00 2001 From: Diego Fronza Date: Mon, 10 Mar 2025 18:19:49 -0300 Subject: [PATCH] meson: Add LLVM bitcode emission for generated sources This commit adds suport for bitcode emission for generated source files (processed by bison, flex, etc). Since generated sources are created with custom_target, we must handle the source files dif
Re: meson vs. llvm bitcode files
Hi,
On Thu, 5 Sept 2024 at 12:24, Nazir Bilal Yavuz wrote:
>
> I found that Andres shared a patch
> (v17-0021-meson-Add-LLVM-bitcode-emission.patch) a while ago [1].
Andres and I continued to work on that. I think the patches are in
sharable state now and I wanted to hear opinions before proceeding
further. After applying the patches, bitcode files should be installed
into $pkglibdir/bitcode/ directory if the llvm is found.
There are 6 patches attached:
v1-0001-meson-Add-generated-header-stamps:
This patch is trivial. Instead of having targets depending directly on
the generated headers, have them depend on a stamp file. The benefit
of using a stamp file is that it makes ninja.build smaller and meson
setup faster.
--
v1-0002-meson-Add-postgresql-extension.pc-for-building-extension-libraries:
This patch is for generating postgresql-extension.pc file which can be
used for building extensions libraries.
Normally, there is no need to use this .pc file for generating bitcode
files. However, since there is no clear way to get all include paths
for building bitcode files, this .pc file is later used for this
purpose (by running pkg-config --cflags-only-I
postgresql-extension-uninstalled.pc) [1].
--
v1-0003-meson-Test-building-extensions-by-using-postgresql-extension.pc:
[Not needed for generating bitcode files]
This is a patch for testing if extensions can be built by using
postgresql-extension.pc. I added that commit as an example of using
postgresql-extension.pc to build extensions.
--
v1-0004-meson-WIP-Add-docs-for-postgresql-extension.pc: [Not needed
for generating bitcode files]
I added this patch in case we recommend people to use
postgresql-extension.pc to build extension libraries. I am not sure if
we want to do that because there are still TODOs about
postgresql-extension.pc like running test suites. I just wanted to
show my plan, dividing 'Extension Building Infrastructure' into two,
'PGXS' and 'postgresql-extension.pc'.
--
v1-0005-meson-Add-LLVM-bitcode-emission:
This patch adds required infrastructure to generate bitcode files and
uses postgresql-extension-uninstalled.pc to get include paths for
generating bitcode files [1].
--
v1-0006-meson-Generate-bitcode-files-of-contrib-extension.patch:
This patch adds manually selected contrib libraries to generate their
bitcode files. These libraries are selected manually, depending on
- If they have SQL callable functions
- If the library functions are short enough (the performance gain from
bitcode files is too minimal compared to the function's run time, so
this type of libraries are omitted).
Any kind of feedback would be appreciated.
--
Regards,
Nazir Bilal Yavuz
Microsoft
From 5f9a9d208bb7e308733b194b75e8b0229797ee4f Mon Sep 17 00:00:00 2001
From: Andres Freund
Date: Thu, 24 Oct 2024 07:23:05 -0400
Subject: [PATCH v1 1/6] meson: Add generated header stamps
Otherwise build commands become too long and this has visible effect on
creation time of meson build files.
Author: Andres Freund
Author: Nazir Bilal Yavuz
Discussion: https://postgr.es/m/206b001d-1884-4081-bd02-bed5c92f02ba%40eisentraut.org
---
src/include/meson.build | 18 ++
src/backend/meson.build | 2 +-
src/fe_utils/meson.build | 2 +-
meson.build | 16 +---
4 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/src/include/meson.build b/src/include/meson.build
index 2e4b7aa529e..c488a5dc4c9 100644
--- a/src/include/meson.build
+++ b/src/include/meson.build
@@ -177,3 +177,21 @@ install_subdir('catalog',
# autoconf generates the file there, ensure we get a conflict
generated_sources_ac += {'src/include': ['stamp-h']}
+
+# Instead of having targets depending directly on the generated headers, have
+# them depend on a stamp files for all of them. Dependencies on headers are
+# implemented as order-only dependencies in meson (later using compiler
+# generated dependencies). The benefit of using a stamp file is that it makes
+# ninja.build smaller and meson setup faster.
+generated_headers_stamp = custom_target('generated-headers-stamp.h',
+ output: 'generated-headers-stamp.h',
+ input: generated_headers,
+ command: stamp_cmd,
+)
+
+generated_backend_headers_stamp = custom_target('generated-backend-headers-stamp.h',
+ output: 'generated-backend-headers-stamp.h',
+ input: generated_backend_headers,
+ depends: generated_headers_stamp,
+ command: stamp_cmd,
+)
diff --git a/src/backend/meson.build b/src/backend/meson.build
index 2b0db214804..7fc649c3ebd 100644
--- a/src/backend/meson.build
+++ b/src/backend/meson.build
@@ -169,7 +169,7 @@ backend_mod_code = declare_dependency(
compile_args: pg_mod_c_args,
include_directories: postgres_inc,
link_args: pg_mod_link_args,
- sources: generated_headers + generated_backend_headers,
+ sources: [generated_backend_headers_stamp],
dependencies: backend_mod_deps,
)
diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.
Re: meson vs. llvm bitcode files
Hi, On Thu, 5 Sept 2024 at 11:56, Peter Eisentraut wrote: > > The meson build currently does not produce llvm bitcode (.bc) files. > AFAIK, this is the last major regression for using meson for production > builds. > > Is anyone working on that? I vaguely recall that some in-progress code > was shared a couple of years ago, but I haven't seen anything since. It > would be great if we could collect any existing code and notes to maybe > get this moving again. I found that Andres shared a patch (v17-0021-meson-Add-LLVM-bitcode-emission.patch) a while ago [1]. [1] https://www.postgresql.org/message-id/20220927011951.j3h4o7n6bhf7dwau%40awork3.anarazel.de -- Regards, Nazir Bilal Yavuz Microsoft
