Date   : Sun, 21 Nov 1982 17:44:00-EST
From   : Frank J Wancho <FJW@Mit-Mc>
Subject: Large Files

As I mentioned in my previous message, there are several mainframe
implementations of MODEM, and there are two working versions that run
on TOPS-20 machines: Bill Westfield's MODEM and Max Diaz' XMODEM.
Sources to both are kept in MC:AR60:.  MODEM works on TOPS-20 only and
also works through an ARPANET connection using a TIP or TAC, NCP or
TCP.  XMODEM works on TOPS-20 direct only, and on TENEX either direct
or through an ARPANET connection using a TIP or TAC, NCP or TCP.

Downloading using either program works at any speed.  Uploading works
at any speed in direct mode, and through a TIP up to 1200.  TACs have
an inherent design flaw (a fixed-size 64-byte input buffer) which
prevents or severely inhibits uploading at speeds greater than 300.

Both uploading and downloading using either program properly handle
both text and binary files, including those binary files uploaded to
MC via LMODEM which contain a special header word as the first word of
the file to indicate that it is a binary file since ITS sites do not
have FDBs to indicate file type.  Thus, you should no longer need
files in HEX or text-only form unless you cannot FTP from MC.

Note to UNIX sites: UMODEM is available and does work with both text
and binary files in your environment.  The trick is to get FTP to
transfer the binary files correctly...someone else can tell us how
they do that.

Further comment on SQueezed vs text vs binary files: binary files are
stored as 4 bytes per PDP-10 36-bit word; text files are stored as 5
bytes per PDP-10 word.  Storing a text file as a binary file takes up
25% more disk space.  Squeezing that file first yields a typical 40%
reduction and takes up 15% less disk space.  That 15% helps, but that
alone is not quite enough justification - converting several files to
Library format is.  The reason: each file takes up directory entry
space, limiting the combined total amount of space for both the files
themselves and the directory entry space.  By packing several related
files into Library format, already a binary file at that point, and
then SQueezing the resulting file, we can make *much* more space
available for the many files which should have been uploaded by now
and haven't, due to that space crunch.

I can appreciate Bill Westfield's desire to leave text files alone so
that he can manipulate them online, but that is a luxury we can no
longer afford.  The principal intent of a CPM directory on MC is to
have a repository for public domain files of interest to the CP/M
community which they can, in turn, download to their micros by
whatever means available to them.  At first, uploading and downloading
was limited to those who had access to MC.  Since then, programs like
Bill's own MODEM have sprung up to provide similar capabilities to
other mainframes.  I suspect that it won't be too long before someone
converts SQ, USQ, TYPESQ, and LU to mainframe versions of those
utilities we have found so handy on our micros...

--Frank