From: mbeattie@black.ox.ac.uk (Malcolm Beattie) Subject: Re: Automatic compression of binaries ? Date: 15 Feb 1993 14:15:37 GMT
In article <1l7m65INN193@life.ai.mit.edu> mycroft@hal.gnu.ai.mit.edu (Charles Hannum) writes:
>
>In article <1993Feb8.135832.4744@black.ox.ac.uk>
>mbeattie@black.ox.ac.uk (Malcolm Beattie) writes:
>>
>> [deleted]
>
>Most of what you say is irrelevant.
I'm sorry you found it irrelevant. I was trying to explain to
those who thought executing a program meant `load it all into
memory and go' that there was something more subtle than that
happening.
> If the kernel is to recognize the
>new magic number, it is trivial to have it watch for other processes
>trying to use the same image and share it. The fact that it loads the
>whole image in one fell swoop is actually a performance *win* if you're
>going to use most of the code and are not short on memory.
The philosophy behind demand paging is that, in general, one
is not going to use most of the code right away.
If one knows that the `uncompress and load and start' method has
the side-effect of disabling demand paging and, after due
consideration, considers it still worthwhile then there is no
problem. In my post, I tried to explain enough technicalities
so that those unfamiliar with demand paging could see that such
a side-effect should be considered.
>The main benefits of demand-paged executables are:
>
>* You may not actually need to read all of some executables. (For
>example, my RS/6000 KCL executable, weighing in at 12MB, doesn't all
>need to be loaded usually.)
>
>* You never have to page out the pages of text in an executable, thus
>reducing paging space requirements and making paging slightly more
>efficient. (Paging is a horrible lose anyway, and should be avoided if
>possible.)
>
>
>On the other foot, let's consider what would typically happen with a
>compressed executable:
>
>* It might be small, in which case it won't compress much, and you're
>basically just adding overhead.
>
>* It might be large, in which case it's likely that you don't need it
>all at once. If you have too little RAM, you might end up paging a bit
>when an executable loads because you're filling it up quicker. You'll
>need more paging space (though you'll probably gain more than enough
>space for it by compressing stuff).
>
>* It might be somewhere in between.
It *might* be worthwhile to increase paging space, uncompress the
compressed executable into memory and let the unwanted bits compete
to be paged out when the free memory is needed.
Then again, it *might* be worthwhile to do what was suggested in an
email to me from Anthony J. Stuckey <stuckey@mrcnext.cso.uiuc.edu>:
simply compress the executable and move it to somewhere like
/usr/tmp/binaries, and putting in its place a tiny script which
uncompresses it and exec's it. That way, you don't have to load
in most of the program only for it to be paged out again; it can
be demand paged as usual.
Which of these is better? I don't know. It would need more
discussion and/or some testing. My `irrelevant' post was intended
to explain enough to newcomers to demand paging that they could
join in with informed opinions.
>Personally, unless disk space was a critical concern, I would choose to
>not compress executables.
I agree.
--Malcolm
-- Malcolm Beattie <mbeattie@black.ox.ac.uk> | I'm not a kernel hacker Oxford University Computing Services | I'm a kernel hacker's mate 13 Banbury Road, Oxford, OX2 6NN (U.K.) | And I'm only hacking kernels Tel: +44 865 273232 Fax: +44 865 273275 | 'Cos the kernel hacker's late