Re: OpenJDK successfully built with Visual Studio express edition

[Kelly O'Hair]

Andrew John Hughes wrote:

2009/12/10 Kelly O'Hair <kelly.oh...@sun.com>:

Damjan Jovanovic wrote:

Hi

I'm documenting my (unbelievably difficult but ultimately) successful
experience of building OpenJDK on Windows with Visual Studio express
edition.

Thanks for sending this out. You do deserve some kind of medal, building
on Windows is very difficult.

First off, where and how did you get the sources?
It's not exactly clear what sources we are dealing with here.

Errr... he says in the next paragraph 'the awt tree from about a week
ago' i.e. http://hg.openjdk.java.net/jdk7/awt

Ah.. You are right. When I saw "awt tree" I was thinking the awt sources
in some jdk tree.... :^(  Clearly it is the awt forest.

My congratulations too on the successful build.  I wouldn't even want
to contemplate this...

I built the awt tree from about a week ago, with Windows Vista, Cygwin
1.5.25-15, Microsoft Visual Studio 2008 Express Edition SP1, Freetype
2.3.5-1, jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar, DirectX SDK
as per README-builds.html, Ant copied from Netbeans 6.7, and make from
the cmake.org website (as per README-builds.html). The import JDK was
1.6.0.

So the devil is in the details, but first off, you have picked a few things
that differ from our recommended set of things:

 * Windows Vista - My condolences, but I don't know of anyone using Vista
   to build. I know Windows 2000 is almost 10 years old, and Windows XP is
   pretty old, but XP is the newest OS we can recommend. Anything newer is
   in the 'on your own' category. But I am glad you were ultimately
successful,
   at least we know it is possible.

 * Microsoft Visual Studio 2008 Express Edition SP1 - We recommend the
   purchased copy, but we understand why you are using this version and
   we would like to see it work, but it's the ATL issue that is the killer.
   Whatever adjustments we can make to allow it's use we will do.

 * Freetype (on windows) - A royal pain in my side. More later.

 * The jdk-7-ea-plug-b77-windows-i586-03_dec_2009.jar should not be needed
   anymore for a successful build. I would skip it. (Binary plugs are
optional).

Reminds me we need to forwardport the doc. update for this.  OpenJDK6
lists them as optional, but 7 doesn't.

I added a note to 6896978 for jdk7 to fix this. You are right.

 * DirectX SDK - Which version did you use? Just curious.

 * Ant copied from Netbeans 6.7 - If that worked, fine. But I would really
   rather people download and install the official 1.7.1 version of ant.
   NetBeans may be sifting out parts of ant in it's install, so using the
   ant from NetBeans may not be the same behavior as from an official ant
   install.  But if you got it to work, that's great.

This is especially true with the recent jaxp/jaxws changes which rely
on the regexp extension.  I know that's not in the base package on
most GNU/Linux distros.

I added a note to 6896978 for jdk7 on this. We should be clear about this.

You could spend a lifetime keeping these READMEs up-to-date. :^(

 * Was the cmake you got 3.81? Just curious.

Firstly the README-builds.html talks a lot about path problems,
without offering any good solutions. After much effort, I discovered
that you have to use /cygpath/c paths only in $PATH and use double
quote quoted C:/ style paths everywhere else (eg. export
WindowsSdkDir="C:/Program Files/Microsoft SDKs/Windows/v6.0A/").

If this isn't clear I'll try and improve it.
By the way, the Microsoft SDK should probably be v6.1, but I don't
think it will matter much. You did not list installing the Microsoft SDK
above, did you explicitly install v6.0A or did that come with the Visual
Studio compiler install?

Microsoft's quality C compilers apparently need environment variables
set before they work properly, otherwise they crash (not complain, but
actually crash). The vcvars32.bat doesn't work from inside cygwin, so
I had to run "set > a.txt", then run "vcvars32.bat", then "set >
b.txt", and finally diff a.txt with b.txt to find out what these
mysterious variables were, then convert them into C:/ and /cygpath/c
style paths that get set in cygwin. Running
jdk/make/jdk_generic_profile.sh loses the variables again, so I
avoided it.

Sigh...  I should change jdk/make/jdk_generic_profile.sh, it was never
made cygwin compliant. That file was mainly meant for documentation
purposes, not sure how I feel about people using it as part of their
build process, :^(

On vcvars32.bat, I have created a shell script called vsvars.sh
(See http://blogs.sun.com/kto/entry/windows_visual_studio_stupid_pet)
that I plan on adding to the jdk repository at some point.
It will (from a cygwin shell) effectively do the vcvars32.bat for
you and set PATH correctly. All you need to do is run
 eval `vsvars.sh -v9`

I hate vcvars32.bat. And you are right about the crashes and unfriendly
behavior, it's windows. Sorry.

Compilation misdetects the Ant version and complains it is too old,
but works fine.

"Compilation misdetects"? Can you run 'ant -version' from the command line?

The ant startup scripts are a bit crufty on Windows, better in ant 1.7.1,
but I have no idea what NetBeans has done with the Windows ant.bat or
ant shell script startup files.
On Windows, you usually need at least three things to get 'ant -version'
to work: 1. ant in PATH, 2. ANT_HOME set, and 3. JAVA_HOME set.
Unfortunately, when building the jdk, we don't allow JAVA_HOME to be
set right now due to the potential build problems it can introduce.
So getting 'ant -version' on windows has been tricky.
With the older ant versions (like 1.6.*, this ant startup was worse).

When ant is broken up and spread around, like on some Linux systems,
it's hard to set ANT_HOME, and I had that problem with NetBeans, and
on Windows, sometimes you just have to set ANT_HOME because the startup
script can't seem to find where it is living. :^(
So I always just install my own ant, makes life easier.

Indeed, my own GNU/Linux builds set ANT to /usr/bin/ant explicitly and
so does IcedTea.

The problem I've had with Linux builds is that depending on the Linux,
you might have a different version of ant. Some have newer ones than
others. Most work, but I just find it easier to have my own little
ant farm on all platforms. ;^)

I also work on JavaFX and it has a requirement of ant 1.7.1 because
it uses some NetBeans ant project files for building, and I think
NetBeans 6.7 started requiring ant 1.7.1, which is no sweat for NetBeans
itself because it ships it's own ant, but building outside of NetBeans
means you need ant 1.7.1 in your PATH. Oh well...

What a world ...  what a world... http://www.youtube.com/watch?v=qfV_ENR5IZE

-kto

Even "make sanity" rapidly died as freetype_versioncheck didn't build.
http://mail.openjdk.java.net/pipermail/jdk6-dev/2008-March/000023.html
claims you need OPENJDK=true set, an undocumented fact that doesn't
help. After much effort I discovered
http://www.mail-archive.com/build-dev@openjdk.java.net/msg02319.html
where someone got it to work with Freetype 2.3.5 by copying
bin/freetype.dll into lib/freetype6.dll and applying a patch; I also
had to copy zlib1.dll into the build/windows-i586/btbins. Please at
least document the fact Freetype 2.3.4 is the last working version?

Sigh... freetype... :^(  What a pain on windows.
I'm going to file a bug on this, see if we can't clean up this dependency
at least as far as building or setting up goes.
I have been down this path, and finally published freetype binary bundles
at http://cr.openjdk.java.net/~ohair/FreeType/

It then compiled for a few minutes further and died in the corba/
subdirectory, with "COMPILER_PATH cannot be empty here". I found
http://mail.openjdk.java.net/pipermail/build-dev/2009-March/001767.html
and edited the file in question to fix RC (I had to use the full C:/
style path, the one from that email didn't work for me), unfortunately
it still didn't help, so I just commented out lines 96-98 and it
compiled much further.

Usually when you get "COMPILER_PATH cannot be empty here", one of
 * PATH is not setup correctly (according to vcvars32.bat)
 * ALT_COMPILER_PATH (if you set it) is not set correctly
 * The system environment variable VS90CMNTOOLS is not defined
   or the Makefiles can't find the default location for the compilers.
   If you ssh into a windows system, VS90CMNTOOLS will not be set.
   If you installed in a non-standard location, that could be the issue.

So I'm a little confused as how you could continue at this point and
would like to understand that.

Can you send me the "C:/Program Files/" directory where the
cl.exe compiler file was found in your install?

The problem repeated itself under the jdk/ directory, and I had to
edit another file and fix the path to RC and RCS.

Yes, the path to RC/RCS is problematic, I think we have a bug on that.
It appears to be a moving and sometimes replicated exe that moves
with each release. :^(

Then, still in jdk/, came missing headers. The first was afxres.h
which I got from mingw and copied into Visual Studio's include
directory. This got it to compile further.

Humm... I thought we replaced a bunch of these afxres.h uses with
just windows.h or winres.h. Ah... jkernel... new stuff.

Next missing header was atlbase.h which lead me to discover an email

(http://mail.openjdk.java.net/pipermail/build-dev/2007-December/000654.html)
stating that the Visual Studio express editions are not supposed to
work (again, undocumented in README-builds.html). Indeed I found that
atlbase.h is part of ATL, which isn't part of any Visual Studio
express edition or the Platform SDK (6 or 7). Visual Studio is
expensive to buy just to compile OpenJDK. But am I really going to let
that stop me, after the previous 8 hour battle?

Well, we can't document everything that doesn't work, but I understand
your situation. We had been trying to allow for the Express compiler to
work, but we could not guarantee it.
At one point, I thought we were ATL #include free, but I see jkernel changes
were added recently and re-introduced an atl dependency.

$ grep atl * -iR | grep include
Binary file hotspot/.hg/store/data/src/share/vm/include_d_b__core.i
matches
jdk/make/jdk_generic_profile.sh:
 include4sdk="${vc7_root}/atlmfc/include"
jdk/make/jdk_generic_profile.sh:
include4sdk="${include4sdk};${platform_sdk}/Include/atl"
jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlbase.h>
jdk/src/windows/native/sun/jkernel/kernel.cpp:#include <atlwin.h>
jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlbase.h>
jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlcom.h>
jdk/src/windows/native/sun/jkernel/stdafx.h:#include <atlwin.h>
jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlbase.h>
jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlcom.h>
jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlwin.h>
jdk/src/windows/native/sun/jkernel/DownloadDialog.cpp:#include <atlhost.h>
jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlbase.h>
jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlcom.h>
jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlwin.h>
jdk/src/windows/native/sun/jkernel/DownloadHelper.cpp:#include <atlhost.h>
jdk/src/windows/native/sun/jkernel/stdafx.cpp:#include <atlimpl.cpp>

I see this bug has been filed recently on this problem:
http://bugs.sun.com/view_bug.do?bug_id=6903970

So in other words only jkernel uses the ATL for anything. Pity that it
uses it extensively, so there's no easy way to take ATL out as a
dependency. Someone should rewrite it without ATL: 4000 lines of code
shouldn't hold 7 million lines hostage.

There may be a way to skip the jkernel build. Perhaps if you
removed the jkernel from SUBDIRS in jdk/make/sun/Makefile?
I have no idea if that would work, just a guess.

But if ATL can't come out of jkernel, maybe jkernel can come out of
Java. I put in lots of #if 0 around the entire contents of all the
jkernel .cpp files, and made preJVMStartup() into a no-op.

That's another way I suppose.

It compiled further, complaining in at least one place about missing
/usr/bin/diff which isn't documented as a dependency, but luckily that
doesn't stop the build.

What? Where did it complain about /usr/bin/diff? That's strange.
You have one I assume, but Windows processes may not accept some
cygwin files as being valid. Let me know if you still have
the error message on this, I'd like to get to the bottom of this.

When it got to compiling fonts, it broke again, this time looking for
freetype.dll instead of freetype6.dll. I copied the one to the other
and typed make with crossed fingers. After 20 minutes of compiling it
got to the fonts again and they compiled.

Sigh... freetype again... I'm glad you got it to compile, it
should not be this hard. Sorry about that.

After a very long wait, it finally compiled successfully...

...and the hello world worked :-).

Glad it was finally successful.

Now I'm onto what I thought would be the hard part: the actual patch I
want to write :-).

Patches I used to build it are attached.

I'll try and make sure there are bugs filed on these issues, can't
promise your exact patches can be used.

Hope this helps someone.

I'm sure it will.

Now you can go get a little devil tatoo that says
  "I built OpenJDK on Windows". ;^)

-kto

Regards
Damjan