[PATCH] [FYI] A little different approach to building Glibc.

Aleksandar Kuktin akuktin at gmail.com
Wed Jun 9 13:03:54 PDT 2010


Hello everyone!

I am writing this in hopes that some people may find my experience
useful, as I have found the experience of others (very) useful. This
will be a long mail, so grab some coffee. :)

Some years ago, I embarked on compiling my own (B)LFS system which I
intended, and in fact do, use as my main computer. My small personal
success in this was preceded by experimental work of others.

Over time, I also started experimenting with various build set-ups and
possibilities. The one that got most of my fancy was HLFS,  with
its promise of a (mostly) secure system.
I opted not to build a full HLFS, PaX included, since I viewed it as
overkill for my needs. I chose to build a stack-protection-endowed
toolchain, and build the whole system with said stack protection and
source fortification. However..

Building as per instructions located in
http://www.linuxfromscratch.org/hlfs/view/unstable/temporary_system/
, guided by HLFS-SVN-20071012 (since navigating a collection of text
files proved to be too much of a challenge), my build progressed fine,
until it came time to build glibc.

I was (and am) building with binutils-2.19.1, gcc-4.4.3 and glibc-2.9,
on a native x86_64 system (my earlier BLFS). The build progressed until
linking of glibc's run-time linker, ld.so. As I delved into the
problem, symbol chasing showed that the problem was with the stack
protector itself. Namely, all the routines in the run-time linker called
__stack_chk_fail, which in turn called a chain of symbols, culminating
with an attempted inclusion of init-first.os into the run-time linker
which broke the build.

Realizing the problem, I chose to build the linker separately, without
stack protection, and then build the rest of glibc with full protection.
Easier said than done. :) Eventually, I came up with a method to do so.
I have made a patch (attached) which is my solution to the problem.

An explanation of the patch:
-The patch adds a new type of object files for the build process. It has
a suffix of .ortld and is a clone of .os object file in all regards
except that it is built with -fno-stack-protector. This is in
Makeconfig.
-As there are a number of .os related tweaks throughout the
Makefiles, I have made efforts to clone them all and believe I got them
all. There are several "extra-libs" variables to which various .os
objects are added. I have bypassed these.
-Also, in sysdeps/unix/make-syscall.sh, there is a part which makes
special rules for building syscalls (.ortld extension not needed for
x86_64). The functionality for .os files has been duplicated in a
minimally intrusive way.
-In elf/Makefile is the main hack. Here, the dependencies for ld.so are
changed to use .ortld objects. After the first linking of the linker,
instead of the old libc_pic.a, we make libc_pic_rtld.a and do a final
link of ld.so against that. The details of this, as well as the reason
for it is in the source (elf/Makefile).
-Finally, the build process needs some final tweaking. There are
several files which are built (in .os variant) with stack protection
off. Otherwise, the static part of the testsuite segfaults and fails
(and triggers a kernel bug in the ext3 driver, linux-2.6.32.2).

Not included in the patch: a change in function definitions of
__readlink_chk and __readlinkat_chk, in debug/readlinkat_chk.c. Two
'void *buf'-s are changed to 'char *buf', otherwise cpp complains
(with source fortification).

So, to recap: the patch enables, on x86_64, the building of glibc with
stack protection. Except the run-time linker, all components (including
libc.so.6) are build with stack protection. The difference from the
basic LFS build is in the application of the patch (and having the GCC
which can actually make the hardened build, ofcourse).

The testing:
Glibc builds fine. The testsuite has 6 failures, not counting annexc.
Error 1:
glibc-2.9-build/rt/tst-mqueue4.out
glibc-2.9-build/elf/check-localplt.out
Error 127:
glibc-2.9-build/debug/tst-chk2.out
glibc-2.9-build/debug/tst-lfschk2.out
glibc-2.9-build/debug/tst-chk5.out
glibc-2.9-build/debug/tst-lfschk5.out

These all seem to be related to source fortifications. Something along
these lines has been discussed on this list before and I am currently
ignoring these failures.

As for the real-life performance, I have not yet built a full BLFS
system on top of a library patched like this. I have, however, built a
full BLFS system on top of a library patched by an earlier patch of mine
which functioned in much the same way, but did not use .ortld. I will
start recompiling my system this week and will hopefully be finished by
the weekend and will drop a message to the list detailing any and all
quirks.

Please note that this patch currently does not work in x86, only on
x86_64. The problem this time is __stack_chk_fail_local and I am still
wrestling with it. It is becoming more interesting as time passes by. :)

The system which I currently have is working near-perfectly. Strangely,
vim can not start due to a buffer overflow (I have not investigated). A
few other packages also need to be patched in order for them to work
under the new environment (all of which I will report when this is
finished). Also, VERY interestingly, Facebook is having a rough time
working (if it works at all). You would not believe just how many
errors those Javascripts of theirs have. ;) Using Arora, on top of
Webkit, Qt-4.6.


So.. feedback is welcome. I will try to find a way to make it work on
x86 as well, but I am short on free time in which to do it. Soon.

-AKuktin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glibc-2.9-grand-unified-fsck-prtct-01.patch.bz2
Type: application/x-bzip
Size: 6803 bytes
Desc: not available
URL: <http://lists.linuxfromscratch.org/pipermail/hlfs-dev/attachments/20100609/b9a32c49/attachment.bin>


More information about the hlfs-dev mailing list