[Nestedvm] Updated nestedvm toolchain!

Henry Wertz hwertz10 at gmail.com
Sun Sep 28 18:30:52 EDT 2014

      I pulled the nestedvm head from http://git.megacz.com/ and found it
didn't build.  I decided to try two different techniques for this:
    1) Patch gcc, binutils, newlib as needed.
    2) Update gcc, binutils, newlib to newer versions (that build cleanly
under gcc-4.8)
     Good news!  I got *BOTH* to work!
     First, building nestedvm "as-is".    There are  several points in the
crosscompiler build where gcc-4.8 stops due to finicky little complaints
needing one-line patches; after those are patched, MIPS binutils have a
buffer overflow, which presumably happens either way but gcc >=4.4 or so
detects the overflow and aborts binutils, while older GCC doesn't detect

     'CFLAGS="-D_FORTIFY_SOURCE=0" make'    avoids both issues -- it skips
whatever sanity check gcc-4.8 does that makes gcc stop at these points, no
patch required; and binutils completes successfully. It looks like John
Stroy's patch of August 24, 2013 disables FORTIFY so his repo at
https://github.com/jdstroy/nestedvm/ should also build AFAIK.
      Second:   I patched up nestedvm to use gcc-4.8.2 and binutils-2.24.
(I didn't update newlib, I wasn't sure how far nestedvm's patches were
customizing it...)
      Please find attached a few patches and a readme.  In the readme,
ignore the later steps if you don't have .tar.gz and classgen.zip
pre-downloaded.  I was wiping the nestedvm directory enough to want to
pre-load it with these files.

      To summarize the readme, you apply nestedvm.patch, and copy the
remaining patches into upstream/patches.  You also copy crt0-override.spec
into upstream/misc .   Then "make" to build the stuff, "make nestedvm.jar"
if you need a package, etc. just like before.
    Summary of the patches:
     binutils.patch is EXACTLY the same as the existing nestedvm binutils
patches but with updated line numbers, it just adds the mips*-unknown-elf*
target.  The original nestedvm binutils patches are not used.
     gcc.patch also just adds the mips*-unknown-elf* target.  The original
nestedvm gcc patches are not used.
     newlib.patch adds a few patches to fix build errors with gcc-4.8. It's
patched in in addition to nestedvm's patches, since stock newlib is used.
     nestedvm.patch makes a few patches; it patches the main Makefile and
upstream/Makefile, and nestedvm's Compiler.java and Interpreter.java.
     crt0-override.spec -- newer gcc no longer uses crt0.o, this has it use
it since nestedvm supplies a custom crt0.o and expects the compiler to use

     In upstream Makefile, I updated the gcc and binutils version numbers
that it grabs; changed the gcc download stuff slightly since gcc is no
longer seperated into multiple tar.gz files. I added "--disable-libssp" to
crosscompiler configure, SSP was causing build problems if I recall
correctly.  Oh, yeah!  GCC >=4.6 quit using crt0.o by default!!  So the
faff about "crt0-override.spec" is to make crt0.o be included in your
static libs and binaries, otherwise you have binaries without a proper
start point and nothing runs in nestedvm.
      In main Makefile, -freduce-all-givs is not legal in gcc-4, so I
removed that from the default CFLAGs, and I took "-Werror" of the build
flags since the examples that were I'm sure warning-free in gcc-3.3.6 now
build but shoot out loads of warnings with gcc-4.8.2.
    In Compiler.java and Interpreter.java, they used symtab.getGlobalSymbol
(for instance to look up the symbol _gp) but objdump tells me _gp is now a
*local* symbol; so I replaced uses of "getGlobalSymbol" with "GetSymbol",
which works fine.
      I built the jpeg test, and some font test.  The results came out the
same, in terms of correct execution.  gcc-4.8.2 results came out 11%
*SLOWER* than gcc-3.3.6.  D'oh!  However, I haven't messed around with LTO,
or messing with the CFLAGS (other than removing the one invalid flag in my
patch).  I figure at least this'll allow software that doesn't build under
gcc-3 to be crosscompiled if one wants to.

     Again, do read at least the top of the the readme.txt -- the patches
aren't just all applied to nestedvm, several are copied into the nestedvm
upstream/patches directory instead.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.hcoop.net/pipermail/nestedvm/attachments/20140928/3afbe1be/attachment-0001.htm 
-------------- next part --------------
Steps to prepare nestedvm:

1) unzip nestedvm.zip
2) cd nestedvm
3) patch -p0 < ../nestedvm.patch
4) cp -av ../*.patch upstream/patches/
   (Yes this copies unnecessary nestedvm.patch over too -- if this bothers
    you then seperately copy {gcc, binutils, newlib}.patch
5) cp -av ../crt0-override.spec upstream/misc/

Optional steps, if you have the .tar.gz and .zip files pre-downloaded:
steps 5-10 save several (somewhat) large downloads.
6) mkdir upstream/download
7) cp -av ../*.gz upstream/download/
8) cd upstream/tasks/
9) touch "download_binutils" "download_gcc" "download_newlib"\
     "download_bsdglob" "download_regex"
10) cd ../../
steps 11-14, classgen is small but I just have doubts on it's hosting
being as stable as gcc and binutil's
11) cd upstream/build
12) unzip ../../../classgen.zip
13) cd ../tasks/
14) touch "extract_git_classgen"
15) cd ../../

To build, run "make".  "make nestedvm.jar" builds a nice .jar file.  make
"djpegspeedtest" builds djpeg and tries to run a speed test.  It tries to
download a JPEG (which doesn't exist on the web server, so it stores a 404
error message).  If you see a message about the image not being a jpeg,
djpeg is built and working though!  Congratulations.  To actually speed test
it, put a jpeg in there to let the tests run. 

make speedtest also fails due to missing files, it wants "Arial.TTF"
(capitalized just like that) in tmp/mspack.

     What is patched?

     In nestedvm/Makefile, this removes the line "-freduce-all-givs".  This
option was removed in gcc-4.0.  This removes -Werror from MIPS_CFLAGS,
because the nestedvm code has several warnings in gcc-4 that I guess
gcc-3.3.6 didn't detect, but no reason to error out that I could see.  Also
I found gcc-4.6+ ditched crt0.o use, and gcc .specs file is built in, but a
-specs flag allows to specify a .specs file and force gcc to include crt0.o. 

     This re-implements the (very minimal) patches to add "mips-unknown-elf"
format to binutils and gcc, and fixes errors (from gcc-4 being more strict)
with ctype and a few functions for newlib.  I don't know what newlib patches
(if any) mesh with the nestedvm java syscall stuff so I decided I better
stick with newlib-1.1.3 it comes with and fix the compile errors!  The other
patches for gcc appear to be bug-fixes for older gcc, not sure about
gcc-fixes.patch and gcc-fdata-sections-bss.patch (the gcc/config/mips/mips.c
is so disimilar in gcc-4.8.2 I don't know what to patch if bss is still
missing though.)

To build: 
make nestedvm.jar

To use:  The compiler is actually just a MIPS cross-compiler, building R2000
code (-march=mips1 -O3 -mmemcpy -ffunction-sections -fdata-sections
-falign-functions=512 -fno-rename-registers -fno-schedule-insns
-fno-delayed-branch -freduce-allgivs)
     arch=mips1 specifies the simplest MIPS instruction set.
     -mmemcpy, says to use "library"-provided memcpy which actually uses
               java memcpy-like call directly with nestedvm.
     The remaining stuff ensures memory alignment to some extent (spanning a
64KB segment is particularly expensive in this Java implementation), and the
-fno-* disable optimizations that help on the pipelined R2000 MIPS but don't
help one bit on a virtual MIPS.

     To convert a MIPS binary to use this:
     java -cp $(classpath) org.ibex.nestedvm.Compiler -outformat class -d
build -o unixruntime (class name) (mips binary name)
     $(classpath) can be nestedvm.jar
     (class name) is something like test.DJPeg for build/tests/DJPeg.class
     (mips binary name) is something like build/tests/DJPeg.mips

>> I try to install gcc-3.4.6 in ubuntu 10.10 but I've got below error
>> In function ‘open’,
>> ??? inlined from ‘collect_execute’ at ./collect2.c:1537:20:
>> /usr/include/bits/fcntl2.h:51:24: error: call to ‘__open_missing_mode’
>> declared with attribute error: open with O_CREAT in second argument
>> needs 3 arguments
>> make[1]: *** [collect2.o] Error 1
>> make[1]: Leaving directory
>> `/home/esmaeil/Documents/software/gcc-3.4.6/gcc'
>> make: *** [all-gcc] Error 2
>> I did
>> ./configure
>> make
>> and the error is appear after several minutes.
> It's a bug in gcc 3.4.6 (you do know that that is a very old release,
> right?).
> You can fix it by changing line 1537 of gcc/collect2.c to this:
> ? ? ?redir_handle = open (redir, O_WRONLY | O_TRUNC | O_CREAT, 0666);
-------------- next part --------------
A non-text attachment was scrubbed...
Name: binutils.patch
Type: text/x-patch
Size: 1571 bytes
Desc: not available
Url : http://lists.hcoop.net/pipermail/nestedvm/attachments/20140928/3afbe1be/attachment-0005.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: crt0-override.spec
Type: text/x-rpm-spec
Size: 55 bytes
Desc: not available
Url : http://lists.hcoop.net/pipermail/nestedvm/attachments/20140928/3afbe1be/attachment-0006.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gcc.patch
Type: text/x-patch
Size: 922 bytes
Desc: not available
Url : http://lists.hcoop.net/pipermail/nestedvm/attachments/20140928/3afbe1be/attachment-0007.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nestedvm.patch
Type: text/x-patch
Size: 12848 bytes
Desc: not available
Url : http://lists.hcoop.net/pipermail/nestedvm/attachments/20140928/3afbe1be/attachment-0008.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: newlib.patch
Type: text/x-patch
Size: 1816 bytes
Desc: not available
Url : http://lists.hcoop.net/pipermail/nestedvm/attachments/20140928/3afbe1be/attachment-0009.bin 

More information about the Nestedvm mailing list