From: hedrick@geneva.rutgers.edu (Charles Hedrick) Subject: Re: NT vs Linux (was: Re: truth or dare) Date: 10 Jul 1993 18:22:07 GMT
dkf@chem.ucsd.edu (David Knight French) writes:
>and disks was their concept of a memory mapped file system. No Buffer
>Cache for the disk devices... He mentioned that SunOS added the same
>thing with v4.1.2 (or maybe 4.1.3). According to him, based more on
>SunOS than AIX, the file access would kill the SCSI bus. The problem
>was in mapping all of the file BEFORE it could be accessed by the
>program and kernel. The biggest problem was the mapping would
>tie up the bus. The larger the file the bigger the problem.
This posting suggests that when you mmap a file, the entire file is
physically read from the disk. That is not what happens. At least
it's not what the TOPS-20 pmap jsys does, and so far the best evidence
is that Unix mmap implementations are based on that. (Look at the
documentation for the two -- mmap is a clear ripoff.) What pmap does
is to allocate pages in your page map for the entire file. It sets
things up so that when you reference one of those pages, a page fault
will occur, and the pager will read in the page from disk. It does
not actually read in the file. Now it is certainly possible to
preload part of the file, and in fact from a performance point of view
it may be a good idea to preload the first few pages. But this is not
inherent in the definition, and is the sort of tuning one can play
with. The advantages of doing things that way are:
- I/O is controlled by the pager, which takes a global view of
optimizing things. So it can make tradeoffs like deciding
whether there's enough free memory to leave a page in memory
in the hopes that it will be used again, or whether you
really need to keep just the pages that are being used at
this moment.
- when several processes are reading the same file, only one
copy is present in memory. Games are played so that their
page maps are indirect through a single global one for
the file. This saves memory, and also allows shared databases to
be implemented in a very neat way, because all processes see
an up to date version of the file.
I agree that there was a problem in SunOS 4. But it was not because
memory-mapped files had to be read in at once. It was because of
tuning. The policies for deciding how many pages of disk files to
keep in memory and how many of pages to use for normal program memory
were wrong. It effectively reduced the amount of memory for programs,
and pushed up the paging rate. But this is not a problem with the
concept of mapped files. It's an implementation problem. It means
that SunOS 4 worked poorly on systems with small memory. It was tuned
(if one can dignify it by that term) for large systems. Under Linux,
a similar policy of allocating memory dynamically for file buffers or
programs was implemented in order to IMPROVE performance for small
systems. And it did. One assumes this is because Linus got his
policies right.