QEMU + MinGW-w64... working at last

OK... After finding the right combination, looks like this is possible after all.  So after looking at the gtk site, they mentioned some instabilities in gtk 2.24, and suggested backing up to 2.16, so I did.  Looking at some other posts, it is hinted at that in mingw, gcc 4.6 might be more stable than 4.7, so I down-reved that as well.  And finally, it sounded like qemu 1.4.1 did have some issues with disk IO so I upgraded my source referece to 1.5.0.  Below is the psuedo code of what I did.

msys=msys+7za+wget+svn+git+mercurial+cvs-rev13.7z
mingw=\
i686-w64-mingw32-gcc-4.6.3-2-release-win32_rubenvb.7z
gtk=gtk+-bundle_2.16.6-20100912_win32.zip
sdl=SDL-devel-1.2.15-mingw32.tar.gz
qemu=qemu-1.5.0.tar.bz2
img=linux-0.2.img.bz2
cd /tmp && wget $msys && 7za x $msys && mv msys C:/
cmd /c c:/msys/msys.bat
..
==
 copy following to .profile c:\msys\home\%USERNAME%
 or from c:\msys\msys.bat shell just do:
==
mkdir /usr/local/src; mkdir /usr/local/bld
cd /tmp && wget $mingw && 7za x $mingw && mv mingw32 C:/
cd /usr/local && wget $gtk && 7za x $gtk
cd /usr/local/src && wget $sdl && tar -xvzf $sdl
cd /usr/local/src && wget $qemu && tar -xvjf $qemu
cd /usr/local/bld && wget $img && bzip2 -d $img
echo C:/mingw32 /mingw > /etc/fstab && mount
cd /usr/local/src/SDL-1.2.15 && make native
cd /usr/local/src/qemu-1.5.0
./configure \
  --python=c:/Python26/python \
  --target-list=i386-softmmu \
  --prefix=/usr/local/bld \
  --cc=i686-w64-mingw32-gcc \
  --host-cc=i686-w64-mingw32-gcc \
  --objcc=i686-w64-mingw32-gcc \
  --extra-cflags="-I/usr/local/include" \
  --extra-ldflags="-L/usr/local/lib"
make install && cd /usr/local/bld
qemu-system-i386 linux-0.2.img

With the above settings, the test image works and seems pretty solid.  So the possible combinations and moving pieces are as follows:

GCC Versions {4.5, 4,6, 4,7}

First I tried 4.5 which is the old MinGW product.  That caused access violations, so I moved up to 4.7.  This was through the Mingw-w64 project, and I picked "Ruben's" build.  That seem to work pretty well, but I had read about some others issues with instability on 4.7 and the thought that this might be too "bleeding edge".  I finally settled on Ruben's Mingw-w64 build of 4.6 and all seems well.

GTK+ Versions {2.16, 2.24}

I may have been over-cautious here.  I started at v2.24 for win32.  This seemed to work, but I was unclear where my issues were.  As I began to research GTK stability questions, I found that for win64 version 2.16 is, in may cases, considered the most stable.  Choosing the side of caution, I back-reved my win32 versioin to 2.16 as well.

QEMU Version {1.4.1, 1.5.0}

Originally I had always been working on 1.4.1 thinking that it would be more stable.  I got good builds, but disk-io was always very slow.  After posting to the mailing list, it was revealed that slow disk-io is a known issue of 1.4.1.  Under good counsel I up-reved my source tree to 1.5.0.  This seems to produce good solid builds and will be where I baseline from.

QEMU with MinGW... Use the source Luke

Well the law of linux applies.  As Jedi Coda said... "use the source Luke".  Well... I didn't do that... I opted for pre-complied bins and that is likely the problem.  But any any case, the problem seems to be related to me using an older tool chain.  With a good deal of help from the mailing list and lots of research, I was able to get another build working.

Originally I started with the cygwin builds, but those fell apart about the time I autotools were called.  There might be a cleaver way to merge these together, but after two days at it, I fell back to a simpler approach.  So here's what we did.  BTW... I cheated a lot, I'll eventually try a cleaner process, but here's where it was today.

1) Grab the better MinGW toolchain

Somewhere, I can't fine the reference now, it was suggested to grab one of the private builds, specifically Ruben's build.

i686-w64-mingw32-gcc-4.7.4-release-win64_rubenvb.7z

2) Grab the better MSYS toolchain

Digging around the WIKI at http://sourceforge.net/apps/trac/mingw-w64/wiki I grabbed the "ultra all inclusive" MSYS build.

msys+7za+wget+svn+git+mercurial+cvs-rev13.7z

3) Grab the autotools package

I don't know if this is really needed, but it was convent to grab, so I did.

autotools-20120807.tar.xz

4) Unpack the tools (I cheated)

Now, I'm supposed to expand all of these in different directories with cleaver LIB, INCLUDE and PATH variables to get them all worked out.  After two days of fighting cygwin, I had lost most of my will to fight this good fight.  So I cheated and started hacking and slashing these toolchains.

upack mingw to c:\unpack
unpack msys to c:\unpack
unpack autotools c:\unpack\autotools

What you should see now are three directories in unpack {mingw, msys, autotools}.  Now we want to merge those directories into one directory.  Something like

cd c:\unpack
mkdir merge
move mingw32 sysroot
move sysroot merge
move msys sysroot
move sysroot merge
move autotools sysroot
move sysroot merge

Now if my memory serves me right you should have a directory {c:\unpack\merge} that should have all the stuff you want.

5) Grab gtk+ bundle and unpack

Go to the gtk+ site at http://www.gtk.org/download/win32.php and grab the bundle.  Then unpack to {c:\unpack\merge}.

get gtk+-bundle_2.24.10-20120208_win32.zipunpack to c:\unpack\merge

6) Clone ./ and ./i686-w64-mingw32 (very hackish)

There are some libraries in /lib and some in /i686-w64-mingw32/lib and resolving them got pretty gnarly.  Just for POC, I've cloned {bin, lib, include} between ./ and ./686-w64-mingw32/lib

cd c:\unpack\mergerobocopy /s .\686-w64-mingw32\bin .\binrobocopy /s .\686-w64-mingw32\lib .\librobocopy /s .\686-w64-mingw32\include .\includerobocopy /s .\bin .\686-w64-mingw32\binrobocopy /s .\lib .\686-w64-mingw32\librobocopy /s .\include .\686-w64-mingw32\include

7) Add .\lib to the Windows path and launch msys

This is required because the configure script will build and test various binaries and the DLLs have to resolve.  Once you invoke the msys shell, use that one todo all other msys work.

cd c:\unpack\mergeset PATH=c:\unpack\merge\lib;%PATH%call msys.bat{leave open you will use it in (9)}

8) Grab qemu and SDL source and unpack

We will grab the qemu and SDL source and unpack them to c:\unpack\merge\src

get qemu-1.4.1.tar.bz2unpack to c:\unpack\merge\srcget SDL-devel-1.2.15-mingw32.tar.gzunpack to c:\unpack\merge\src

9) Set the environment for the builds (hackish again)

We are using cross-compile binaries, so we need to set the compiler environment variables.  Using --host --build or --target should work for other stuff, but changing the global environment kind of catches them all.  It would be better to use --cross-prefix or something like that , but they all throw various errors.

goto the same msys windows spawned in (7)export CC=i686-w64-mingw32-gccexport CXX=i686-w64-mingw32-c++export LD=i686-w64-mingw32-ldexport AR=i686-w64-mingw32-arexport AS=i686-w64-mingw32-asexport NM=i686-w64-mingw32-nmexport STRIP=i686-w64-mingw32-stripexport RANLIB=i686-w64-mingw32-ranlibexport DLLTOOL=i686-w64-mingw32-dlltoolexport OBJDUMP=i686-w64-mingw32-objdumpexport RESCOMP=i686-w64-mingw32-windresexport WINDRES=i686-w64-mingw32-windres 

10) Build SDL and QEMU

Now we are ready to build.  Make sure you are using the same msys window from (7) and (9) since it has the environment set up.

cd /usr/src/SDL-1.2.15make nativecd /usr/src/qemu-1.4.1./configure --python=c:/Python26/python.exemakemake install

11) Test and contemplate 

So 1-10 gave us functioning binaries, but as you probably noticed, they are slow as snot.  A likely suspect is the fact that I really went to the tool chains with a meat cleaver.  Being a bit more measured will help.  Technically, I should create a clean/unhacked mingw64 / msys environment with the proper isolation and path pointers.  Then from the "clean" environment, build and install {zlib, gettext, libiconv, glib, pkg-config, SDL} from source.  Then perhaps things will work as desired.

Or it could just be that I missed some obvious ./configuration switch in (10) that would give me the normal QEMU speeds I expect.

Return to Linux, Compiling QEMU with MinGW.

I've begun my adventures with Raspberry Pi, for this summer.  I'm hopping to get the kids up to speed on some of the finer points of computer programming and computer architecture.  Something beyond the Apple app store.  So my first Raspberry Pi "project" was to get the Raspbian OS running on Windows so I could poke around at it.  This was fairly straight forward, all you need is to download the image and click.  Apple App store revised.

So playing with the pre-built Win32-Qemu binaries with Raspbian did show a few errors.  The first and foremost is that it isn't emulating a RPi at all.  It is actually emulating a Versatile board which is close to the RPi but only just.  The solution is to grab a branch of Qemu that has been patched to emulate RPi.  So now the fun begins... How to compile Qemu for Win32 without Linux?  Well the supported answer is MinGw used to compile.  Problem is... Qemu seems to have some *VERY* sloppy bounds checks and the code is absolutely trashing memory.  The Windows loader shuts it down as soon as it sees "bad" behavior.  Others seem to have the build working better, but I've yet to find their magic combo.  Here's my not so magic combo.

Abridged version for those with short attention

wget mingw-get-inst-20120426.exe
wget python-2.6.6.msi
wget gtk+-bundle_2.24.10-20120208_win32.zip
wget SDL-devel-1.2.15-mingw32.tar.gz
wget qemu-1.4.1.tar.bz2
unpack_all.sh
install_all.sh
cd SDL-lib && make native
cd qemu-src && ./configure
make && make install

Straight forward enough, and compiles, but throws exceptions on execution. Now for the deeper dive

0) Grab cygwin, for wget.exe and unzip.exe on your own

Cygwin is easy enough to get, just download setup.exe from http://www.cygwin.com .  Once the installer launches, choose the default options.  Most people just install to c:\Cygwin.  Durring setup you can search for packages.  I just make sure I have wget and unzip selected.  When the install is finished (takes a while) you should be able to call wget and unizp by simply calling c:\cygwin\bin\wget.  Don't add the bin directory to your path as it could conflict with MinGW.

1) Grab and install mingw-get-inst-20120426.exe.

This is pretty easy, just go to http://www.mingw.org and grab the binary mentioned from download.  Once grabbed, run the installer and install EVERYTHING except ADA and Fortran (unless of course you really want to.

2) Grab and install python-2.6.6.msi.

Same deal, download the 2.6 msi from http:\\www.python.org and follow the prompts

3) Grab and install the Gtk+ multipass.

Again, we start with, obvious enough http://www.gtk.org.  Fish around and grab the zip.  This is where we start to get technical.  Gtk+ has a "bundle" that you can grab that will put in all your dependencies, and there are a few.  This "multipass" means the only outstanding dependency is SDL.  In any case, when you installed (1) you should have gotten a C:\MinGW directory.  Under that directory are the lib, include and bin directories that the bundle is going to land stuff in.  So... just unzip gtk+-bundle_2.24.10-20120208_win32.zip directly into C:\MinGW and your compiler should find all the goodies.

4) Grab and install the SDL libraries.

This starts simple enough.  Download the SDL libraries from http://www.libsdl.org.  Now, unlike Gtk+, we are going to install these in the "/usr" hierarchy.  To do this properly, we are going to drop into a MinGW shell.  Back in step (1) we should have gotten a "MinGW Shell" icon that launches something called MSys.  So the way this plays out is as follows.

a) Save the .tar.gz file to C:\MinGW\msys\1.0
b) Launch the MinGW Shell from start menu / button / icon
c) Once in the shell type "mkdir /usr/src && cd /usr/src"
d) Then untar with command "tar -xzvf ../../name.tar.gz"
e) Now install with "cd libsdl && make native"

Sounds weird, but that should have done the trick.  I would have opt'ed for "make install" on (4e) but "make native" is really what they want.

5) Finally grab and expand QEMU source.

The QEMU source is located at http://wiki.qemu-project.org/.  Just as before we are going to drop this tarball in C:\MinGW\msys\1.0.

a) Save the .tar.gz file to C:\MinGW\msys\1.0
b) Launch the MinGW Shell from start menu / button / icon
c) Once in the shell type "cd /usr/src"
d) Then untar with command "tar -xzvf ../../name.tar.gz"

From here we are getting to the meat of the matter.

6) Run ./configure in QEMU

In linux, as I recall, there are configure scripts to "prep" the source for compilation.  The ./configure script has help, so it might be worth running ./configure --help to learn more.  Basically the only option I provided was the python path.  I also added a prefix so I didn't pollute my Program Files.  You may need to mkdir /usr/bld.

a) cd QEMU-src-dir
b) ./configure --python=c:/Python26/python.exe --prefix=/usr/bld

7) Now we pull the trigger... Make QEMU

This is easy enough, but takes a while.

a) cd QEMU-src-dir
b) make && make install

8) Now we need something to test

Although you may have a linux image you want to test, best start with something simple.  Turns out QEMU links to some simple linux images to standardize this process.  We will grab the simplest from http://wiki.qemu.org/Testing.

a) Download linux-0.2.img.bz2
b) Save the file to C:\MinGW\msys\1.0\bld
c) Un-bzip the image "bzip2 -d linux-0.2.img.bz2"

9) Now we test our build

Keep in mind that I actually added "--prefix=/usr/bld" to my ./configure arguments, so my qemu files are not where they are "supposed" to be.  Now I'm going to test this from a MinGW shell so that all the dependencies map out correctly, but other tests fail equally.

a) cd /usr/bld
b) ./qemu-system-i386 linux-0.2.img

Initially things look good, but fail quickly after the disk is mounted.  Looking through the errors, they all seem to be of type 0xc0000005 which is generally an access violation.  A program reading or writing from memory it is not supposed to.  On some systems it throws a DEP exception which seems right in line with a 0xc0000005.  I've tried various options like -nographic to exclude SDL.dll from consideration, and all combinations seem to fail.  I'm tempted to say that QEMU just doesn't work on Windows... except... some people are managing to build for Windows like http://www.omledom.com/.  It is possible that they are cross-compiling these binaries from Linux, but for the fact that QEMU.org seems to claim that QEMU will build (and presumably run) when built under MinGW.  I just don't see how.