A few notes on Malloc returning live-pointers

Kevin Day thekevinday at gmail.com
Sat Sep 2 17:59:54 PDT 2006


I have found that for the most part, when malloc is set to a more
secure mode (aka: no live pointer returns), there have been a few
programs that segfault as a result.

The following programs show serious/blatant problems when malloc does
not return a live pointer:
xterm
fontforge
wine
xine-ui

xterm refuses to even start (segfaults instantly), recompiling the
uClibc fixes xterm instantly (no xterm recompiles)

fontforge was more unique, when I had hardening enabled, fontforge
would compile through. When disabled, all parent programs up to the
getty, will get killed.  If in Xorg, the entire server stops/freezes,
killing xorg is the only way out (Control+Alt+Backspace or switching
to a linux console and running a kill command).  On a rare occasion,
the entire system was taken down by this.

wine, which has an optional dependancy on fontforge, will segfault
during compilation. (though I am still hunting down some run-time

xine-ui, will not start up, will not segfault, it locks up and becomes
defunct, recompiling uClibc does not fix this, xine-ui must be
recompiled in after uClibc is recompiled with malloc returning a live
pointer.

I have also, in the past run across a number of sources that had
compilation problems, specifically perl and python. They do not seem
to be acting up now.

Nevertheless, there seems to be a slight pattern that makes me believe
that somewhere in the massive gcc/g++ sources, there is a malloc call,
or a few calls, that expect or try to use malloc(0) returning a live
pointer.

I seem to get more problems during compilation than during runtime
when malloc does not return a live pointer.

-- 
Kevin Day



More information about the hlfs-dev mailing list