@Make(Report)
@Device(LaserPrinter)

@Disable(Outline, Contents, FigureContents, TableContents)

@Use(Bibliography "UPDATE.BIB")

@Set(Page=1)

@PageHeading(Left  "The new filestores",
             Right "@Value(Page)",
             Immediate)
@PageFooting(Right "@Value(TimeStamp)",
             Immediate)

@UnNumbered(The New Filestores)

This document describes the implementation of the Computer Science Department's
new 1976-protocol filestores, based around a Advanced Personal Machine (APM)
 with a Winchester
disc drive.  The overall structure of the filestore system is described,
together with a discussion of the ways in which the functionality provided
differs from that of the original Interdata-based filestore.  This document
should be read in conjunction with @Cite[TheFilestore], which describes the
original Interdata-based implementation.


@Section(Machine Configuration)

The new filestores are based around a 1Mb APM, with the standard
RS232 (control console) and ethernet interfaces, together with a purpose-built
SMD disc interface.@Foot[A mini-filestore has also been configured, using a
small (16Mb) Winchester disc.]
Unlike the old filestore, the new ones have no departmental
link interface, so all communication with clients proceeds via the ethernet.  No
provision is made in the hardware for lineprinter support, as it is assumed
that any printers attached to a filestore will have sufficient intelligence to
communicate in the same way as would any other client (software hooks are
provided, however).  The disc space is divided into partitions in such a way
that they are no larger than can be addressed by 16 bits.


@Section(Comparison of Facilities Provided)

Although the facilities targeted towards the "normal" user of the new filestore
are substantially identical to those provided by the old filestore, or are
compatible, there are a number of changes in the more esoteric facilities. 
These are described in the following sections.

@SubSection(Passwords and Permissions)

Passwords may consist of any sequence of characters whatsoever, with the
exception of newline and comma (these are actually valid password characters,
but users may experience some difficulty in setting passwords which include
them).  The parity bit is significant at present.  Upper case and lower case
0-parity letters are deemed to be equivalent.  Passwords are limited in length
to about 245 characters.

Unlike the old filestore, a null directory password does not match anything
for login: instead a null password must be supplied.  The quoting mechanism is
unchanged, however.

Full, Read and None protections are implemented for both Owner and World.  Obey
protection is not meaningful.  Archive and Vulnerable markers are implemented. 
Owner protection is constrained to be no more restrictive than World protection.
Users quoting the system password are given either Owner or Read authority,
whichever is less restrictive.

@SubSection(Filenames, Ownernames and Directories)

Ownernames and filenames are constructed from sequences of letters, digits, 
dots, underlines
and dollars, with no restriction as to order.  Ownernames must be at least one
and no more than six characters long.  Filenames must be no more than twelve
characters long.  Null filenames and filenames beginning with the special
character '!' will be uniquified to guarantee that there are no other files of
the same name currently in the directory.

Directories are held in the same format as on the old filestore, and hence the
same limits on the number of files apply.  The datestamp records the date and
time when the file was last modified.  The special file DIRECTORY "contains" the
names of all the files in a directory; either the Finfo or the Ninfo ('N')
command should be used if more
information is required.  Transient and subliminal files are deleted when the
directory reference count goes to zero; they are not charged to the directory's
quota.

@SubSection(Spool Devices)

As already stated, there are no spool devices directly attached to the new
filestores.  Despoolers are expected to use the normal protocol to tranfer files
from the filestore.  Clients wishing to transfer files to a despooler's
directory should normally use the uniquification scheme described in the
previous section.

@SubSection(Control Terminal)

The control terminal attached to the APM performs several functions. 
@Begin(Itemise)
During booting it may be used to specify some alternative action to the default
boot sequence.

While the filestore is running significant events are logged on
the control terminal, together with a periodic report on filestore activity:
such reports include use of the system password, creation and deletion of users,
and disc and ether failure conditions.

If the keyboard is enabled it may be used to request more
detailed information on the operation of the filestore or to change its mode of
operation.  The following (single-character) commands are available:
@Begin(Description)
control-_@\(control-underline)
Toggle the enable/disable state of the keyboard.  It should normally
be kept disabled.

?@\Report on the current filestore status.

B@\Baud rates of any Qsart RS232 lines present.

O@\Alter the open state of the filestore.  A number should be given to indicate
the desired state.  The ususal values of this are 0 to close the system
entirely, 3 to close the system to anyone not using the system password, and -1
to open the system completely.

Q@\Qsart status.  This is ignored if there is no Qsart.

S@\System buffers.  This gives a list of who owns them.

T@\Set trace mode.  The normal value for this is 0.

W@\Set file system protection mode.  The normal value for this is 1.  Setting
this to 0 disables any operations which would write to the disc.
@End(Description)
Note that while these commands are waiting for input the entire filestore is
placed in a state of suspended animation, and hence they should be used with
discretion, if at all.
@End(Itemise)


@Section(Protocol)

For the most part the protocol used by the new filestore is compatible with that
of the old, in the sense that almost anything accepted by the old filestore will
also be accepted by the new one.  The new filestore is slightly stricter than
the old one in one or two cases, however.  Furthermore, the error messages
produced, while broadly similar to those of the old filestore, are not
guaranteed to be identical in all cases (or even in most cases).  Significant
changes to the protocol are noted below.

@SubSection(General enquiry)

Only case zero (date and time) is implemented.

@SubSection(Boot)

Boot is not implemented.  Clients are assumed to have sufficient intelligence
built in to be able to boot themselves.

@SubSection(File reading and writing)

There is no distinction between "SQ" files and "DA" files.  SQ and DA operations
may be freely mixed, although the user should realise that DA operations will
reset the current block pointers for the file.  Files may be extended one block
at a time by WriteSQ and WriteDA.  Files opened for reading and modification
will never be made any shorter; files opened for writing only will be truncated
to the current block position when they are closed.  Null files will be deleted
on closing.

Multi-block ReadSQs will produce the required number of blocks or the remainder
of the file, whichever is shorter, the latter condition being indicated by the
transmission of a short block.

@SubSection(Set password)

It is possible to specify a username as the second parameter to the pass
command: in this case the spcified user's password is changed, rather than the
logged-on user's.  This extension is only available to privileged users.
Note that it is not possible to quote one's own username to change its
password unless privileged.

@SubSection(Change file's datestamp)

The 'C' command may be used to alter the datestamp information associated with
a file.  This is useful when maintaining consistency between versions of a file
held on several filestores, for example.  The format of the command is
@w(C @i<U filename, date>) where @i<U> is a valid user number, @i<filename> is
the name of the file being changed and @i<date> is the new dastamp information
in the form @w[dd/mm/yy hh.mm] (standard filestore date/time format).

@SubSection(File status enquiry)

The command 'N' may be used to enquire about the status of a particular (named)
file, with the reply being in the same format as that provided by Finfo ('F'). 
The format of this command is @w(N @i<U filename>)
where @i<U> is a valid user number and
@i<filename> is the name of the file being enquired about.

@SubSection(Remote control)

This is not implemented in the same form as in the old filestore.  The
']' command is available, with the following options (@i'U' indicates
a user number, quoting the system password unless otherwise stated):
@Begin(Description)
]@i<U>1,n@\Kill Uno.  Kills user number n.  @i<U>
must be quoting the system password if
the target Uno does not belong to the same owner.

]@i<U>2,n@\Kill Xno.  Kills transaction number n.

]@i<U>3,n@\Kill port n.  Not implemented yet.

]@i<U>4,n@\Set diagnostic mode.  Same as the console 't' command.

]@i<U>5,n@\Set system availability.  Same as the console 'o' command.

]@i<U>6,pass@\Sets the system password.  This takes immediate effect.  The new
password will be written out to the disc, from whence it will be read during
subsequent reboots.

]@i<U>7@\Reboot the filestore.
@End(Description)


@Section(Special files)

A number of pseudo-files "owned" by user '$' "contain" useful information about
the state of the filestore's internal tables.  These are:
@Begin(Description)
TRACE@\The filestore's trace buffer.  This should be formatted by the
appropriate command run on the client.

BITMAP.@i<n>@\The bitmap for partition
@i<n>.  Again, this requires to be formatted to make it more digestible.

UNOS@\The user number table.

XNOS@\The transaction number table.

DIRECTORIES@\The directory cache.

PORTS@\The ether port table.

BOOTAREA@\This special file gives access to the boot area at the start of the
disc.  Note that this area is not accessible as part of any user partition. 
This file must be accessed using OpenMod, ReadDA and WriteDA.
@End(Desciption)


@Section(A Quick Tour through the System)

The filestore system is based around a number of co-operating processes:
@Begin(Itemise)
ether receiver handler

ether depacketiser

command interpreter

ether transmitter handler

console log stamp
@End(Itemise)
Apart from the last of these, requests from clients pass through each process
in turn.  Processes perform their tasks and then place themselves onto one
of the wait queues maintained by the scheduler.  The processes are removed
from their wait queues and given the CPU when the scheduler determines that
there is something for them to do, either because some other process has
requested that the first process on the queue should be awakened, or because
the scheduler itself has detected that some event has occurred which requires
a process to be resumed to handle it.  Processes are scheduled strictly
according to the priority of the wait queue on which they have placed
themselves; however higher priority processes do not pre-empt lower priority
ones, with the result that no complicated concurrency control is required.

In detail, the sequence of events triggered by the arrival of a request from
a client is as follows:
@Begin(Enumerate)
The arrival of the ether packet is detected by the ether driver in the kernel
which sets the appropriate bit in the DTX bit-vector to indicate that the
packet has arrived.  This is detected by the scheduler, which causes the ether
receiver handler to execute to read the packet into an internal buffer.  The
receiver handler then checks to see if there is another packet available, and
if so it reads that in too.  This filestore process is the only one which does
not suspend itself when it has completed the task for which it was awakened.  It
is scheduled with the highest priority in order that packets are removed
from the ether station as quickly as possible, the aim being to reduce the
load on the ether station to a minimum.  When it has read all the packets
from currently available in the station it kicks the depacketiser process and
suspends itself.

The purpose of the depacketising process is to split apart the incoming
requests from clients into their constituent text and data parts.  This is
necessitated by the filestore protocol, which considers requests as a
(variable length) textual command part separated from any data by a NL
character.  This task is performed by a separate process from the ether
receiver handler in order to keep the former as small and fast as possible.
Having been depacketised, the request is queued to the interpreting processes.

The interpreting process which dequeues the request determines what action to
take on the basis of the command letter in the first byte of the text part.  It
calls one or more of the file system routines to perform this task, putting the
resulting buffers onto the ether transmission queue.

Finally one of the ether transmitter processes dequeues the reply buffer,
transforms it into a single ether packet in a form acceptable to the client and
sends it on its way.  In order to ensure that the load imposed on the station is
minimised, the process waits until the packet is either ACKed or NAKed before
dequeueing the next buffer.
@End(Enumerate)

The purpose of the log stamp process is to generate and write out the console
log information.

@UnNumbered(References)
@Bibliography