shared libraries [ was: Re: Fwd: Re: zlib-1.1.4 out - security fix ]

Matthias Benkmann matthias at winterdrache.de
Thu Mar 14 13:51:04 PST 2002


Even though it's horribly off topic for lfs-security, I'd like to shed
some light on the shared libz debate here. Before you stop reading: I
actually did measurements, so the following is not just theory.

But let's start with the theory:

1. shared libraries come with a speed penalty

a) the dynamic linker/loader needs to be executed
If it's not in the disk cache this usually requires a seek to a different
part of the hard disk. Even if it's in cache, preparing the process space
for it to run takes some time.

b) dynamic linking needs to be performed
This is a non-trivial task that involves parsing symbol tables in the
binary and the shared library/libraries. Unless they are in the disk
cache, the symbol information for the shared libraries has to be read from
disk and this usually needs a (slow) hard disk seek because the shared lib
is stored in a different area of the hard disk than the binary being run.

c) during program execution, the necessary code portions from the shared
lib need to be mmap'ed into the program's process space. Unless the
required parts of the shared library are in memory already, they need to
be read from disk when the program first accesses them. Again this usually
requires a slow hard disk seek. Furthermore, a large library like libc has
lots of code, not all of which is used by a program. The parts actually
being used may be spread out over the whole library file, so that the code
that's actually needed has to be pieced together from different hard disk
locations.

2. Shared libraries sometimes come with a RAM size penalty

Static linking usually puts only those parts of the code into the binary
that are really needed and the code is densely packed with the rest of the
code. All addresses are hardwired (in general some relocation may still be
necessary by the loader, but I think on Linux this is not the case). In
shared libraries on the other hand, addresses are not hardwired which may
require more code for indirect jumps. Furthermore the code that the binary
needs is mixed with code it doesn't need and with with symbol information
in the library file. Because the kernel only loads whole 4K blocks into
memory, it's possible that it needs to load data from the shared lib that
the binary doesn't actually use.


3. statically linked binaries often don't come with a speed penalty

A statically linked binary is larger than a dynamically linked one, so you
could be tempted to think that when the binary is loaded, more stuff has
to be loaded from disk, resulting in slower startup. This is not true.
Linux uses demand-loading, it doesn't load a page until it is needed. You
can have a 10MB binary and it will load up as fast as a 10KB one. Linux
will only load code from disk that is actually used and code that is used
is used regardless of whether the binary is statically or dynamically
linked. In the case of a dyn. linked binary, the code has to be loaded
from the shared lib and often has to be pieced together from different
parts of the shared library. In the statically linked case the code is
loaded from the binary itself where it is all densely packed, not mixed
with unneeded code. Unless the binary is fragmented, it's all in one place
and can be read with maximum hard disk speed without seeks.

4. statically linked binaries sometimes come with RAM size penalty
and with a speed penalty

If you lauch 2 statically linked programs A and B that both include libz
code, then you have the libz code in memory twice. If, however you launch
2 instances of program A, you don't have the code in memory twice, because
the code is shared among the 2 processes. If you have dynamically linked
programs A and B, the code of libz is always shared, regardless of whether
you run A and B or 2 instances of A. So statically linked binaries come
with a RAM penalty if you run multiple *different* programs that use the
same library. The above translates to a speed penalty for statically
linked programs if you run multiple programs that include the same code,
because the same code has to be loaded from disk multiple times.


5. statically linked binaries come with a hard disk size penalty

This one is obvious.


Let's summarize:
Statically linked programs need less processing at startup and fewer disk
seeks and loads at startup, so they start up faster. During runtime,
statically linked programs need fewer disk accesses (especially seeks)
than dynamically linked programs. This is true when you look at the
isolated programs. But when you run multiple different programs statically
linked with the same code, the code is loaded multiple times and these
unnecessary disk accesses can eat up the previous advantage.

A statically linked program needs less RAM than the same one dynamically
linked. However, when multiple different programs use the same library,
the sizes for the statically linked programs add up while the memory in
the dyn. linked case is always shared. This can eat up the advantage of
the statically linked case.


Now let's take the libz example. I compared running time and memory
consumption of a minigzip (part of the zlib package) dynamically linked
against libc and libz against a minigzip dynamically linked against libc
and statically linked with libz. Both programs and the libz.so.* were
stripped. 

My runtime comparison used a small file that was compressed and
decompressed a 1000 times. The times below should be dominated by startup
time because the file was so small. The hard disk didn't play a major role
because all of the test fit into the cache. However, hard disk accesses
would likely only work to the disadvantage of the shared library, because
this is the single-process case. So more hard disk influence would only
make these numbers more pronounced. 

Dynamic

real    1m23.256s
user    0m49.550s
sys     0m33.620s

Static

real    1m16.193s
user    0m44.200s
sys     0m31.990s


The memory test used a large file. I ran it several times and monitored
minigzip's growing memory consumption. The max. value was the same in all
runs. Here are the results:

Dynamic

unzip 1428/480
zip 4160/3240

Static

unzip 1412/464

zip 4148/3228


So the experiment confirms the theory. There is an observable overhead
with dynamically linked binaries regarding speed and RAM usage. However,
as said above, this overhead becomes less significant the more different
programs you run in parallel that have zlib code and there is a break-even
point beyond which shared libraries are faster and smaller. Note that
these have to be different programs. Running 10 instances of minigzip in
parallel is still faster with the statically linked version because the
code is shared. But if you copy the minigzip program to minigzip1,
minigzip2, minigzip3,...,minigzip10 and run these in parallel, the you are
punished for static linking.

So as always there is no clear winner. It depends on the circumstances and
usage patterns whether static linking or dynamic linking should be
preferred. The only thing that can be said for sure is that shipping zlib
with a program and linking it statically is easier for the maintainer.
Because zlib is such a small library, this is a viable option.

MSB

-- 
Ambition is a poor excuse for not having enough sense to be lazy.

-- 
Unsubscribe: send email to listar at linuxfromscratch.org
and put 'unsubscribe lfs-security' in the subject header of the message



More information about the lfs-security mailing list