From: pdh@netcom.com (P D H) Subject: Re: New feature for the filesystems. What do you think ? Date: Mon, 19 Apr 1993 06:42:53 GMT
keith@ksmith.com (Keith Smith) writes:
>Am I nuts here? Seems like you just need a layer of code between the
>virtual disk your filesystem see's, and the physical media. This code
>can maintain and map multiple compressed blocks (say 4K) into physical
>uncompressed block numbers used by the filesystem code. The only trick
>I see is having a filesystem that has a dynamic size. This should not
>be insurmountable either, as one could define the size of the virtual
>disk as arbitrarily 3X, and then use a non-removeable psuedo file entry
>to eat up any unavailable space, or reduce it's psuedo "size" from 2X
>the disk as successful compression kicks in. In this way you always see
>"real" space available, but space in use would be dynamic shifting from
>the pseudo file to real files and back as compression occurs or
>compressed space is free'd. For any kind of data integrity compression
>should be done on a block by block basis (4K at least), but perhaps with
>n smaller sub-blocks (256bytes?) being sqeezed into each block. That
>way a trashed block won't kill you for more than 16K of data (@4:1).
How about putting the compression layer between the applications and the
file system?
Random access of compressed data is difficult (seek to byte 16777216,
where do you end up in the compressed data?). But whatever solutions
exist; they could certainly be applied at this layer. But the SEQUENTIAL
file is going to work at this layer very well and will be able to take
advantage of more sophisticated algorithms.
One of my gripes about programs like Stacker is that I get lousy
compression in many cases; the algorithm has to find redundancy in
just a small amount of data. I still have to PKZIP my files to get
the kind of compression I get now in DOS.
>For example the filesystem wants block #20, hands off to the virtual
>translator which looks in a table and finds virtual block 20 is offset
>512 characters in physical block 3. It fetches it, uncompresses it, and
>delivers the appropriate chunk of data back.
You picked nice round numbers. Real life compression of real life data
involves whatever redundancy exists. You could get 1024 compressed to
711 bytes. Now where are you going to put it? Are you going to reuse
the remaining 313 bytes somewhere? Anywhere?
-- | Phil Howard, pdh@netcom.com, KA9WGN Spell protection? "1(911)A1" | | Right wing conservative capitalists are out to separate you from your MONEY | | Left wing liberal do gooders are out to separate you from EVERYTHING ELSE!! | +-----------------------------------------------------------------------------+