Date   : Sat, 17 Mar 1984 12:55:23-PST
From   : Sam Hahn <SHahn@SUMEX-AIM.ARPA>
Subject: Re: Download REALLY Big Files

Thanks for your reply.  In summary, there are several ways:

1.     Use DDT to break, and PIP to rebuild, the large files.  This is
       tedious and error-prone, but will work.

2.     Use PIP with the start and end options.  This requires knowing a lot
       about the insides of a (possibly binary) file.  Also, the segment
       sizes are not easily determined.

3.     Use SQ and USQ to Huffman-ize the files.  For me, there was quite
       a savings in space to SQ my file, and this worked.  Does have
       a limitation if your binary file doesn't have a good distribution
       of char's for SQ to really make a difference.  Also, still have
       a problem if the squeezed file is still too big.  SQ'ing a second
       time still reduces, but there's a limit.

4.     Use a HEXIFY program.  This is a pretty good solution which requires
       a step to break up the hexified program.  If the source file were
       nearly the size of a disk, one is pretty stuck, since the hexified
       file is larger than the original; probably want a hexify program which
       can operate in stages (could be this already exists -- in the end,
       I didn't need to do this).

5.     Custom-build a program which steps through the original file,
       and copies onto another disk until that disk is full, closes, resets,
       and continues until N disks are full and the original file is
       scanned.  Most time-consuming solution, but there will be cases in
       which this is the best solution, I imagine.

6.     Do directory twiddling to rename the extents of the big file so that
       each extent looks like a separate file.  Would require directory 
       munging of the file, so potentially dangerous, but probably doable
       in an easy and "correct" way so that the directory is not damaged.
       PIP could than be used to rebuild the file.

Thanks to all of you who responded.  For those of you who didn't see my 
original request, I needed to transfer large (>400k) files from a non-standard
(SDSystems) hidensity format to my new Compupro hardware.  My apologies for
running on at such length.
                               -- sam hahn