Date   : Fri, 12 Feb 1988 22:52:39 EDT
From   : "John S. Fisher" <FISHER@CICGE.RPI.EDU>
Subject: Bitnet server overloaded

As many of you on the Bitnet-side of this list realize there have been
some difficulties getting files from the Bitnet file server at RPICICGE.
Three independent but simultaneous events seem to have caused serious
network congestion through the main cooridor of Bitnet.  The particular
events are not important except for the fact the RPICICGE server was one
source for the over-run of network data.

Since I cannot allow my server to be an unfair burden to the network I
have placed it into restricted service:  File requests were limited to
one per day, and directory listings were usually disabled.  Februrary
has just not been a good month.

Some analysis of the types of requests flowing into the server indicated
that most of the load was coming from people requesting directory listings
at regular intervals (probably just to see what was new).  I cannot really
blame people for their actions, the server gave them no good alternative.

At any rate, in an attempt to keep the server within acceptable traffic
loads, I have made a couple of changes to how it operates:

1.) The /PDDIR command is be reworked.  If just a directory name is
    specified (as in /PDDIR PD:<CPM>) the server will return a list
    of the subdirectory names.  Formally, it returned a complete listing
    of all files available from the server.

2.) If /PDDIR is used with more than just a directory name specified,
    it expects a new parameter, a number following the name pattern.  The
    number specifies a limit on age for entries to be listed.  If the
    number is omitted, the default is 30 meaning list no file older than
    30 days.  For example, /PDDIR PD:<CPM.*>*.* 14 would search for any
    files in the CPM directory that have been add/updated in the last two
    weeks (but see #3, next).

3.) If /PDDIR is used with an asterisk appearing in the subdirectory name
    (as in /PDDIR PD:<CPM.*>*.* and /PDDIR PD:<CPM.SYS*TL>*.*) then the
    search is unconditionally cut-off after 21 entries are found.  That
    means that /PDDIR PD:<CPM.SYSUTL>*.* 99999 would list all entries in
    the SYSUTL subdirectory, but /PDDIR PD:<CPM.SYS*TL>*.* 9999 would list
    only the first 21 entries.

4.) Both /PDGET and /PDDIR keep track of the number of requests and the
    number of bytes the command generates.  (Formally, only the /PDGET
    command keep counters, and the counters were for number of files.)
    Requests are denied with the counters reach 5 requests or 100,000
    bytes, which ever comes first, in one day.  (However, the first file
    requested during any day may exceed 100,000 bytes.)  Counters are
    also kept by network node to prevent people from defeating the
    command limits by "cycling-through" userids.

5.) The server keeps a local cache of recently requested files.  In many
    cases a file at Simtel20 would be updated, but the server on Bitnet
    would still have a cached copy of the old version.  The server now
    tries to compare date-of-last-change to determine if the cached copy
    is the most current.  Obsolete copies are discarded and the newest
    version fetched in its place.

To make matters worse, complete testing of these changes has been frustrated
by some local problems that have prevented reliable access to the Internet.
The server is presently sitting with a two-day backlog of requests.  But, be
that as it may, most of the changes are long overdue.  The limit parameters
in place are best-guesses by me and are subject to change.


Regards,
JSFisher, maintainer of many (too many) things.