Re: [Mono-dev] Repeat builds of core assemblies
Hello! Final update! I got all the code working with the new dependencies! The code lives in the staged-cyclic-builds branch, and I believe it is ready to merge. Will do one final human inspection before I do so. Miguel On Tue, May 13, 2014 at 11:37 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello Bryan, This is a step in the direction of having everything building with msbuild. Whether I will succeed or not, is yet to be determined :-) Miguel On Mon, May 12, 2014 at 12:07 PM, Bryan Crotaz bryan.cro...@silvercurve.co.uk wrote: Will this make building on windows possible? Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5
Re: [Mono-dev] Repeat builds of core assemblies
Hello Bryan, This is a step in the direction of having everything building with msbuild. Whether I will succeed or not, is yet to be determined :-) Miguel On Mon, May 12, 2014 at 12:07 PM, Bryan Crotaz bryan.cro...@silvercurve.co.uk wrote: Will this make building on windows possible? Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5
Re: [Mono-dev] Repeat builds of core assemblies
Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.com wrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Miguel ___ Mono-devel-list mailing list
Re: [Mono-dev] Repeat builds of core assemblies
Will this make building on windows possible? Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.com wrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP
Re: [Mono-dev] Repeat builds of core assemblies
On 12/05/2014 17:07, Bryan Crotaz wrote: Will this make building on windows possible? Bryan, I've written a walkthrough here for building 3.4.0 from the release tarball and 3.4.1 from git on Windows. http://www.codeproject.com/Articles/769292/How-to-build-Mono-on-Windows Best regards, Alex Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com mailto:mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.com mailto:mig...@xamarin.com wrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com mailto:mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: * Core Build * Secondary Build: o Core Build + o Defines: XML_DEP + SECURITY_DEP o Refs: + -r:PrebuiltSystem=../lib/Previous/System.dll + -r:System.Xml.dll + -r:MonoSecurity=Mono.Security.dll * Final Build: o Secondary Build + o defines: -d:CONFIGURATION_DEP o Refs: + System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0:
Re: [Mono-dev] Repeat builds of core assemblies
Ooooh pretty. Will give it a go. Can the build script be extended to install needed cygwin packages if they're missing? When I first tried to build I had no idea how to find the packages, or how to install them - for newbie contributors this would remove a lot (in my case 2 days) of head scratching. On 12 May 2014 17:23, Alex J Lennon ajlen...@dynamicdevices.co.uk wrote: On 12/05/2014 17:07, Bryan Crotaz wrote: Will this make building on windows possible? Bryan, I've written a walkthrough here for building 3.4.0 from the release tarball and 3.4.1 from git on Windows. http://www.codeproject.com/Articles/769292/How-to-build-Mono-on-Windows Best regards, Alex Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0
Re: [Mono-dev] Repeat builds of core assemblies
There's a lot of references in your document to build failures and potential future build failures. Can I suggest that pull-requests are only accepted if they don't break the build? It's surely the developer's responsibility to make sure that they haven't created any conflicts, or have correctly documented extra cygwin packages that are needed, or have correctly edited the Windows installer to include all the correct files. This way, newbie contributors can be guaranteed that they can download the latest default branch from git and can build it. Then they can get on with being productive instead of being put off by a whole load of errors they won't understand. Bryan On 12 May 2014 17:30, Bryan Crotaz bryan.cro...@silvercurve.co.uk wrote: Ooooh pretty. Will give it a go. Can the build script be extended to install needed cygwin packages if they're missing? When I first tried to build I had no idea how to find the packages, or how to install them - for newbie contributors this would remove a lot (in my case 2 days) of head scratching. On 12 May 2014 17:23, Alex J Lennon ajlen...@dynamicdevices.co.uk wrote: On 12/05/2014 17:07, Bryan Crotaz wrote: Will this make building on windows possible? Bryan, I've written a walkthrough here for building 3.4.0 from the release tarball and 3.4.1 from git on Windows. http://www.codeproject.com/Articles/769292/How-to-build-Mono-on-Windows Best regards, Alex Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working.
Re: [Mono-dev] Repeat builds of core assemblies
On 12/05/2014 17:30, Bryan Crotaz wrote: Ooooh pretty. Will give it a go. Can the build script be extended to install needed cygwin packages if they're missing? When I first tried to build I had no idea how to find the packages, or how to install them - for newbie contributors this would remove a lot (in my case 2 days) of head scratching. That's an interesting idea Bryan. I'd love it if the Win32 build process was a little more streamlined but I wouldn't know where to begin myself within the Mono build scripting at present. In the short-term it might be possible to do something with a script using one of the command-line package managers for Cygwin (e.g. apt-cyg) I could have a look at that when I have time maybe if the Cygwin UI setup app is proving a hurdle. On 12 May 2014 17:23, Alex J Lennon ajlen...@dynamicdevices.co.uk mailto:ajlen...@dynamicdevices.co.uk wrote: On 12/05/2014 17:07, Bryan Crotaz wrote: Will this make building on windows possible? Bryan, I've written a walkthrough here for building 3.4.0 from the release tarball and 3.4.1 from git on Windows. http://www.codeproject.com/Articles/769292/How-to-build-Mono-on-Windows Best regards, Alex Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com mailto:mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.com mailto:mig...@xamarin.com wrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com mailto:mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: * Core Build * Secondary Build: o Core Build + o Defines: XML_DEP + SECURITY_DEP o Refs: + -r:PrebuiltSystem=../lib/Previous/System.dll + -r:System.Xml.dll + -r:MonoSecurity=Mono.Security.dll * Final Build: o Secondary Build + o defines: -d:CONFIGURATION_DEP o Refs: + System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows
Re: [Mono-dev] Repeat builds of core assemblies
I'd like to get to a point where a newbie can install cygwin, check out from git, and make. C'est tout. On 12 May 2014 17:35, Alex J Lennon ajlen...@dynamicdevices.co.uk wrote: On 12/05/2014 17:30, Bryan Crotaz wrote: Ooooh pretty. Will give it a go. Can the build script be extended to install needed cygwin packages if they're missing? When I first tried to build I had no idea how to find the packages, or how to install them - for newbie contributors this would remove a lot (in my case 2 days) of head scratching. That's an interesting idea Bryan. I'd love it if the Win32 build process was a little more streamlined but I wouldn't know where to begin myself within the Mono build scripting at present. In the short-term it might be possible to do something with a script using one of the command-line package managers for Cygwin (e.g. apt-cyg) I could have a look at that when I have time maybe if the Cygwin UI setup app is proving a hurdle. On 12 May 2014 17:23, Alex J Lennon ajlen...@dynamicdevices.co.uk wrote: On 12/05/2014 17:07, Bryan Crotaz wrote: Will this make building on windows possible? Bryan, I've written a walkthrough here for building 3.4.0 from the release tarball and 3.4.1 from git on Windows. http://www.codeproject.com/Articles/769292/How-to-build-Mono-on-Windows Best regards, Alex Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile:
Re: [Mono-dev] Repeat builds of core assemblies
We've been persuading C# shops to use the Raspberry Pi platform and build scripts aren't that much of an issue for them regarding acceptance. Being able to fix bugs quickly and easily without having to fight the build system is more important to getting lots of contributors. If we could have an automated build that: 1. builds 2. tests 3. creates Windows installer 4. deploys the installer to a VM and tries some smoke tests 5. creates linux packages 6. deploys the package to a VM and tries some smoke tests 7. publishes artifacts to wiki, and to public repos (e.g. Debian package repo) and stops at first failure, emailing the results to the list... Then we'd have a system that could be hooked up to test pull requests automatically and reject any that fail, and we've have immediate availability of all build versions to newbies. The first step is a clean build script that JustWorks (TM). Sounds like you're nearly there. On 12 May 2014 17:49, Alex J Lennon ajlen...@dynamicdevices.co.uk wrote: On 12/05/2014 17:35, Bryan Crotaz wrote: There's a lot of references in your document to build failures and potential future build failures. Can I suggest that pull-requests are only accepted if they don't break the build? It's surely the developer's responsibility to make sure that they haven't created any conflicts, or have correctly documented extra cygwin packages that are needed, or have correctly edited the Windows installer to include all the correct files. This way, newbie contributors can be guaranteed that they can download the latest default branch from git and can build it. Then they can get on with being productive instead of being put off by a whole load of errors they won't understand. I'm just somebody who believes in Mono and would like to help faciliate others to be able to use it as I do. I do hope the folks behind Mono come back to your on your suggestion though, as I too believe that with a little effort each release tarball could be QA'd to be sure it builds without error on each target platform. I believe time spent that way would go a long way to drive acceptance of Mono as a viable alternative for businesses. Cheers, Alex Bryan On 12 May 2014 17:30, Bryan Crotaz bryan.cro...@silvercurve.co.uk wrote: Ooooh pretty. Will give it a go. Can the build script be extended to install needed cygwin packages if they're missing? When I first tried to build I had no idea how to find the packages, or how to install them - for newbie contributors this would remove a lot (in my case 2 days) of head scratching. On 12 May 2014 17:23, Alex J Lennon ajlen...@dynamicdevices.co.ukwrote: On 12/05/2014 17:07, Bryan Crotaz wrote: Will this make building on windows possible? Bryan, I've written a walkthrough here for building 3.4.0 from the release tarball and 3.4.1 from git on Windows. http://www.codeproject.com/Articles/769292/How-to-build-Mono-on-Windows Best regards, Alex Bryan Crotaz Silver Curve On 12 May 2014, at 16:59, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, Another update: I am almost done with the work, only one cycle left to resolve and I will be able to land the changes. All the changes are on a branch on github. MIguel On Fri, May 2, 2014 at 4:27 PM, Miguel de Icaza mig...@xamarin.comwrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.comwrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can
Re: [Mono-dev] Repeat builds of core assemblies
On 12/05/2014 18:20, Bryan Crotaz wrote: We've been persuading C# shops to use the Raspberry Pi platform and build scripts aren't that much of an issue for them regarding acceptance. Being able to fix bugs quickly and easily without having to fight the build system is more important to getting lots of contributors. If we could have an automated build that: 1. builds 2. tests 3. creates Windows installer 4. deploys the installer to a VM and tries some smoke tests 5. creates linux packages 6. deploys the package to a VM and tries some smoke tests 7. publishes artifacts to wiki, and to public repos (e.g. Debian package repo) and stops at first failure, emailing the results to the list... Then we'd have a system that could be hooked up to test pull requests automatically and reject any that fail, and we've have immediate availability of all build versions to newbies. The first step is a clean build script that JustWorks (TM). Sounds like you're nearly there. Very interesting indeed Bryan. And very much the type of fire and forget development and deployment process I am trying to work towards myself here. For what it is worth I have been building embedded Linux platforms with Yocto (and prior to that OpenEmbedded) and am finding this an excellent build environment to manage commercial, source-controlled, license-aware, cross-platform builds of embedded Linux. Most of our recent deployments have been Freescale i.MX based, but I too have been looking at the RPi because I have a strong belief in the disruptive nature of a $25-$35 price-point for a full-fat embedded Linux platform. Opening the door of the embedded arena to students and hobbyists via RPi / Arduino / Intel Galileo and so forth is going to reap huge rewards for society in terms of innovation in our future, imho. As such, I've been maintaining the meta-mono layer for the Yocto project, which builds Linux Mono 3.4.0 for various targets, including RPi, in a controllable and maintainable manner. http://git.yoctoproject.org/cgit/cgit.cgi/meta-mono/tree/README I put this walkthrough together on Mono + Yocto recently as I believe there must be a substantial number of enterprises out there with the .NET skill-sets, legacy .NET code, and the desire to deploy to Linux targets. https://wiki.yoctoproject.org/wiki/Building_and_running_embedded_Linux_.NET_applications_from_first_principles And this is a little older, but gets a Yocto build up for RPi - http://www.pimpmypi.com/blog/blogPost.php?blogPostID=7 Hopefully of interest. Cheers, Alex ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
kudos to you, Miguel! On Fri, May 2, 2014 at 5:27 PM, Miguel de Icaza mig...@xamarin.com wrote: Hello guys, Just a follow up to my previous posting on this. I have managed to untangle this mess, and now I have a clean build that does not involve overwriting assemblies. In addition to untangling this, I added dependencies on all the assemblies involved in this circular dependency mess so if you type make in any of System, System.Xml, System.Security, Mono.Security or System.Configuration, all the dependencies will be properly built. During the fixing, I noticed that our System.Xml build must have broke a few eons ago, because there was code in place to perform a 2-stage System.Xml build as well (without and with System.Configuration support), but nobody noticed that this had happened. While I fixed this, it raises the obvious point that nobody really cares (or likes) System.Configuration. While doing this review, I found a few other places that also have these ugly loops, so I am going to be fixing those as well. Miguel On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: Core Build Secondary Build: Core Build + Defines: XML_DEP + SECURITY_DEP Refs: -r:PrebuiltSystem=../lib/Previous/System.dll -r:System.Xml.dll -r:MonoSecurity=Mono.Security.dll Final Build: Secondary Build + defines: -d:CONFIGURATION_DEP Refs: System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
Why don't we extend mcs to twist netmodules compilation into multi-assembly compilation? See http://msdn.microsoft.com/en-us/library/226t7yxe(v=vs.110).aspx So that a line like below would work? mcs /out:System.dll System/System.Collections.Generic/Queue.cs ... /out:System.Xml.dll System.XML/System.Xml/XmlAttribute.cs ... Then we would have a single pass on the build for building this troika. For me it surely looks like a more sensible solution, aside from probably eating a lot more memory for this compilation. :) Another alternative would be to compile to netmodules with mcs and extend Mono's al into converting them into multiple assemblies instead of a multiple-file single assembly, code changes would be similar to the changes in mcs, but multiple assembly metadata handling in the command line would probably be more messy... Just my crazy two cents... Rafael Teixeira O..:.) On Tue, Apr 22, 2014 at 11:16 PM, Miguel de Icaza mig...@xamarin.comwrote: how are you fitting msbuild in the build process? Just to confirm if you using the generation of csharp project files and solutions for visual studio? Yes, continuing the work on mono/msvs/scripts/genproj.cs There are already a few issues solved, but ran into this broken build setup, and quickly realized that we had this hack on the Makefiles which is why it works at all. There is quite some work left to be done for genproj.cs, but this is something that would benefit both. Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
Hi, On Tue, Apr 22, 2014 at 9:53 PM, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll Another option is to take advantage of the fact that ilasm does not require referenced assemblies to exist. The idea would be do to the following: Once: * Use any existing mscorlib.dll, System.dll and System.XML.dll to create mscorlib.il, System.il and System.XML.il (possibly by using mono-cil-strip so that only the metadata remains). We commit these files to the repository. During the build: * Compile the *.il files to a set of reference assemblies (note that since ilasm does not require referenced assemblies to exist, we can compile mscorlib.il to a mscorlib.dll that references System.dll before System.dll exists). * Use those reference assemblies to compile the final mscorlib.dll, System.dll and System.XML.dll Pros: * No circular build magic required. * The build should be faster, each assembly is compiled at most twice (once the .il and once the final assembly). With the current setup we compile some assemblies three times. Cons: * It'll probably be painful to update/create new .il files when new profiles comes out from Microsoft. Rolf The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list -- Explore Xamarin University http://xamarin.com/university--unlimited, live, online,
Re: [Mono-dev] Repeat builds of core assemblies
On 23/04/14 04:14, Miguel de Icaza wrote: How about if we simplify this, by using the following strategy: - Move the interdependent types of those assemblies to corlib. - Mark them as internal. - Add InternalsVisibleTo for those assemblies towards mscorlib. - Use type forwarders to bring those types back from mscorlib to the correct places. This way the build would be much simpler, without dependency cycles. The dependency cycles are embedded in the public API contracts. So the libraries must be built with those cycles to match Microsoft's public API. I didn't propose to build binaries without dependency cycles, I just proposed to push the implementation towards a unique assembly (which BTW could be System.dll, no need to push it to mscorlib I guess). The build cycles merely encode the steps that must be taken to bring up those cycles on binaries. I'm guessing that what you mean is that by using the type forwarders to bring the types to the same assembly as they appear in MS, the metadata would be different and therefore the ABI/API would not be the same? ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Repeat builds of core assemblies
Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. To build a complete System.dll for a particular profile (net_2_0, net_4_0, etc) takes three steps: - Core Build - Secondary Build: - Core Build + - Defines: XML_DEP + SECURITY_DEP - Refs: - -r:PrebuiltSystem=../lib/Previous/System.dll - -r:System.Xml.dll - -r:MonoSecurity=Mono.Security.dll - Final Build: - Secondary Build + - defines: -d:CONFIGURATION_DEP - Refs: - System.Configuration.dll The above is what is required to bring up System. Our implementation has one major problem: it overwrites the intermediate files. So the core build output is overwritten by the secondary build, and the secondary build is overwritten by the final build. It seems that historically, instead of introducing temporary directories for each stage, instead we hacked our way out of it. We introduced a LIBRARY_USE_INTERMEDIATE_FILE whose sole purpose was to work around the case where Windows was actively telling us we were doing something wrong (we were overwriting a file that we were actively referencing!) The above is also likely going to prevent reliable parallel builds, or probably means that we introduced some gross hack to make the above work in parallel. I am going to try to fix this, but the Makefile goop is pretty dense, and I might fail. I just figured I should share my findings in case civilization comes to an end and a future archeologist tries to figure out why this was not working. These are the defines that we use to bring up System for each profile: basic Profile: basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 basic: -d:NET_1_1 -d:NET_2_0 -d:BOOTSTRAP_BASIC -d:CONFIGURATION_2_0 -d:XML_DEP Build Profile: build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 build: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP Net 2.0 profile: net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_2_0: -d:NET_1_1 -d:NET_2_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.0 profile: net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_0: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Net 4.5 profile: net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP net_4_5: -d:NET_1_1 -d:NET_2_0 -d:NET_3_0 -d:NET_3_5 -d:NET_4_0 -d:NET_4_5 -d:CONFIGURATION_2_0 -d:XML_DEP -d:SECURITY_DEP -d:CONFIGURATION_DEP Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. snip Is this a Microsoft circular dependency or a Mono implementation issue? Michael ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
Hey Miguel, On 22/04/14 21:53, Miguel de Icaza wrote: Hey guys, The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. How about if we simplify this, by using the following strategy: - Move the interdependent types of those assemblies to corlib. - Mark them as internal. - Add InternalsVisibleTo for those assemblies towards mscorlib. - Use type forwarders to bring those types back from mscorlib to the correct places. This way the build would be much simpler, without dependency cycles. HTH, Andres -- ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
Sorry, somehow I misread who was saying what; I should have addressed the mail primarily to Miguel. Happy to provide further information. I think Marek Saraf is the person that merged these updates in 2012 to the VS build process. For reference (won't provide much additional material) I have a stale branch at https://github.com/jmp75/mono/tree/msvc_build_20130604 I intended to use for better documentation. Let me know if I can contribute something. Cheers. From: Perraud, Jean-Michel (CLW, Black Mountain) Sent: Wednesday, April 23, 2014 10:23 AM To: Michael Franz; Miguel de Icaza Cc: mono-devel Subject: RE: [Mono-dev] Repeat builds of core assemblies Michael, how are you fitting msbuild in the build process? Just to confirm if you using the generation of csharp project files and solutions for visual studio? A couple of years back I worked on unwinding the circular build dependencies to build the assemblies from VS. As I recall, even then I needed to disable the parallel build option in VS to avoid some issues. The resulting build process had a few warts left, but I could built most of it from VS fine without nightmare dll swapping. If you are approaching the build another way, I'd be interested to know how. You'll find emails on it in the mono-devel-list archives (http://lists.ximian.com/pipermail/mono-devel-list/). I cannot seem to access the archives just now, but a google search with for instance the terms build visual studio site:lists.ximian.com/pipermail/mono-devel-list/ will give you prior information. I think there was a thread a few months back of people interested in contributing to Mono from Visual Studio, which may provide some help. I'll see if I can dig this up. I started a branch to document the build process using the VS toolchains, but could not find the time to follow up. Cheers, J-M From: mono-devel-list-boun...@lists.ximian.com [mono-devel-list-boun...@lists.ximian.com] on behalf of Michael Franz [mvfr...@gmail.com] Sent: Wednesday, April 23, 2014 9:46 AM To: Miguel de Icaza Cc: mono-devel Subject: Re: [Mono-dev] Repeat builds of core assemblies On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. snip Is this a Microsoft circular dependency or a Mono implementation issue? Michael ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
Michael, how are you fitting msbuild in the build process? Just to confirm if you using the generation of csharp project files and solutions for visual studio? A couple of years back I worked on unwinding the circular build dependencies to build the assemblies from VS. As I recall, even then I needed to disable the parallel build option in VS to avoid some issues. The resulting build process had a few warts left, but I could built most of it from VS fine without nightmare dll swapping. If you are approaching the build another way, I'd be interested to know how. You'll find emails on it in the mono-devel-list archives (http://lists.ximian.com/pipermail/mono-devel-list/). I cannot seem to access the archives just now, but a google search with for instance the terms build visual studio site:lists.ximian.com/pipermail/mono-devel-list/ will give you prior information. I think there was a thread a few months back of people interested in contributing to Mono from Visual Studio, which may provide some help. I'll see if I can dig this up. I started a branch to document the build process using the VS toolchains, but could not find the time to follow up. Cheers, J-M From: mono-devel-list-boun...@lists.ximian.com [mono-devel-list-boun...@lists.ximian.com] on behalf of Michael Franz [mvfr...@gmail.com] Sent: Wednesday, April 23, 2014 9:46 AM To: Miguel de Icaza Cc: mono-devel Subject: Re: [Mono-dev] Repeat builds of core assemblies On Tue, Apr 22, 2014 at 3:53 PM, Miguel de Icaza mig...@xamarin.com wrote: Hey guys, I was looking at making the MSBuild system work, and during the process I have encountered a few problems that we have in our existing build system that are problematic. The problem is that System, System.XML and System.Configuration are each defined in terms of the other assemblies. So we gradually bring up each one of those assemblies up by first compiling a stub System, which we use to build System.XML and System.Configuration. Then we rebuild System, this time referencing System.XML and System.Configuration so we can take a dependency on them, and so on. snip Is this a Microsoft circular dependency or a Mono implementation issue? Michael ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
How about if we simplify this, by using the following strategy: - Move the interdependent types of those assemblies to corlib. - Mark them as internal. - Add InternalsVisibleTo for those assemblies towards mscorlib. - Use type forwarders to bring those types back from mscorlib to the correct places. This way the build would be much simpler, without dependency cycles. The dependency cycles are embedded in the public API contracts. So the libraries must be built with those cycles to match Microsoft's public API. The build cycles merely encode the steps that must be taken to bring up those cycles on binaries. Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Repeat builds of core assemblies
how are you fitting msbuild in the build process? Just to confirm if you using the generation of csharp project files and solutions for visual studio? Yes, continuing the work on mono/msvs/scripts/genproj.cs There are already a few issues solved, but ran into this broken build setup, and quickly realized that we had this hack on the Makefiles which is why it works at all. There is quite some work left to be done for genproj.cs, but this is something that would benefit both. Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
