.nr Hc 2
.nr Hs 9
.nr Hb 9
.nr Ej 0
.rm ul
.rm cu
.ND "January 28, 1980"
.TL
KSOS Input / Output User's Guide
.AF "FACC / Palo Alto"
.AU "John Nagle" JBN "" "Software Technology"
.MT 4
.SA 1
.PF "'(DRAFT)'KSOS Input / Output User Guide'9/7/80'"
.H 1 "Introduction"
The KSOS manual pages give the basics of
requesting I/O operations.  This is an application guide
for the user with specialized or complex requirements.
.H 1 "General discussion of I/O"
The Kernel calls concerned with I/O are
.BL
.LI
K_open
.LI
K_close
.LI
K_read_block
.LI
K_write_block
.LI
K_device_function
.LI
K_secure_terminal_lock
.LI
K_get_file_status
.LE
.P
The calls K_get_object_level and K_set_object_level also apply to I/O objects.
.P
All objects of I/O have seids, and possess security properties.
All objects of I/O also possess a "subtype", which is a means of restricting
access to specific approved programs.
.H 2 "Requests"
All I/O operations can be performed by K_device_function.  K_read_block
and K_write_block are simply short forms for performing K_device_function
with functions of READ and WRITE respectively.  There are a number of other
I/O functions, primarily for manipulating devices.  For objects which are
addressable, the block number argument determines which block is to be
accessed; for non_addressable objects the block number is ignored.
.H 1 "I/O Subdivisions"
The objects of I/O are files, extents, terminals, and devices.  The
name space partition field of the seid identifies the class of the object.
File seids have a special format, to allow for a large number of file
systems.
.H 2 "Files"
Files are dynamically allocated spaces on addressable devices.  Files are
organized in units of 512 bytes, and requests must be in multiples of
512 bytes.  The valid operations are READ, WRITE, ERASEFILE, and
SETFILESIZE.
ERASEFILE causes the space allocated to the file to be released.
SETFILESIZE is a convenience for UNIX emulation which allows the size of the
file, in bytes, to be stored in the file system's control information for
the file.
.P
Sparse files are permitted; unwritten blocks read as all zero.
Reasonably large
file systems are permitted; a file system may be up to
2 ** 32 bytes, in size, and one file can take up an entire file system.
.H 3 "Performance"
The overhead of reading is quite low.  The overhead associated
with sequential reading is essentially independent of file size.
Random access to files is generally slower than sequential
access.   In the best case, a file of 5 blocks or less,
there is no penalty for random access.
In the worst case, randomly distributed accesses to a
100 megabyte file can take six times longer than sequential
accesses.  This overhead cost is in proportion to the logarithm of the
file size, and the curve is relatively smooth.
.P
Writing is as fast as reading unless allocation is required.
When an allocation occurs, the time required is roughly double that of
the same operation without allocation.  As the file system fills up,
allocation slows down, but not substantially until only a few blocks
remain.  Finding the last free block in a 100 megabyte file system might
take over a second on an RP06.  A file system in which slot space is scarce
is also slow, slower than one nearly out of data space.
It is thus desirable to size file systems so that one runs out
of user space before running out of index slots.
.P,
For those interested in making specific performance calculations,
the structure used is a broad tree of 102-byte slot items, five to a disk
block.  Each slot after the first represents twelve data blocks.
The branching ratio of the tree is nine.
Sparse files are represented by a partial tree in which
subtrees not required are not created.
.P
Releasing a file takes less time than allocating and writing it.
Scrubbing and releasing a large file may take several minutes.  The file
system is not locked against other transactions during this period, but the
process making the request which initiated deletion is blocked until the
request completes.  The request can be a K_unlink, a K_close, or
a K_device_function with ERASEFILE requested.
.P
Multi-block file reads are very effective.  If a request can be completed
with one physical I/O, it will be.  Up to 32K can be transferred at one
time in this way, in little more time than that required to read one block.
Frequently-used I/O bound programs should use large buffers if at all
possible.
.P
Over time the file space may become fragmented.  This reduces the probability
of acquiring contiguous space and may slow down the file system.
Dumping the file system to tape and restoring it with the appropriate
NKSR utilities will result in a totally contiguous file system, thus
restoring maximum performance.  It is advisable to perform this process
occasionally.  At this time insufficient experience exists to predict
fragmentation rates.
.H 3 "Reliability"
The KSOS file system is designed to be quite robust in the face of error.
Although most error recovery action is never seen by the user,
special applications, primarily on-line ones, may have files which are
continously updated and must be recovered after a system fault.  For such
programs, a rather detailed discussion of the use of damaged file systems
is provided.
.H 4 "Basic Principles"
.AL
.LI
When a K_write_block call returns, the data written to the file has
become part of the file.  Stopping the system or the disk will not
alter this state.
.LI
Disk errors while reading a file do not damage the file.
.LI
No detected disk error will stop a running Kernel.
This includes
accesses to file index information.
.LI
All disk errors are reported to the user making the request.
.LI
Disk errors during allocation of new space to a file
may result in the loss of all or part of the file involved,
all or part of up to four other files per disk error, or loss of
allocatable space to the file system.
Detected disk errors will not result in security flaws or corruption of
user data, although data may be destroyed outright.
The request which encountered the disk error will return an error status
(Xerror or Xfault).
.LI
All index items of the file system contain checksums, which are checked
on each read of index information.  Items with bad checksums are treated
as if the read from disk failed; thus bad transfers from disk will not
usually stop the system.
.LE
.P
What this all means is that while KSOS file systems are not invunerable to
disk errors, the damage caused by each error is contained.  Disk errors may
cause data to become unreadable; they should not cause it to become wrong.
KSOS file systems may be damaged by a disk write error, but unless
new errors occur, the damage will not become worse due to continued
use of the file system.
.P
When a file system is involved in a system crash or write error, it is
desirable to recover the file system with the appropriate NKSR utility
programs.  Until this is done, some file space may be neither in use
or available for allocation, and files being written at the time of the
crash may show an incorrect file size.
While file recovery should be performed on every writable file system after
every abnormal system halt, a damaged file system can be mounted and used
without serious hazard if operational extingencies so require.  This
capability allows a very fast emergency restart which may be required
in some applications.
.H 2 "Terminals"
.H 3 "Terminal modes"
.P
All functions concerning modes of terminals are performed through the
K_device_function kernel call.
The functions
SETEOMCHARS, SETTERMMODES1, SETTERMMODES2, GETTERMMODES, and SETPATH
are used to set and retrieve information about the modes of the terminal.
.H 4 "SETTERMMODES1 I/O function"
.P
This function, along with SETEOMCHARS, is used to control the Kernel's 
buffering of terminal input.  There are three modes which can be turned
on and off with this function, and all three are specified in   
.I "blockNo[1]\c"
, which is treated as type
.B bits
in this case.
.VL 38 2
.LI "suppress_echo"
turns off all echoing by the Kernel when set.
.LI "break_on_all_characters"
causes every character to be considered to be a
separate logical input line; i.e. input buffering is disabled.
.LI "suppress_echo_of_break_character"
prevents echo of break characters when echo
mode is on.
.LE
.P
All of these modes are local to the path to the terminal, to prevent the use
of the terminal mode bits as an inter-level storage channel.
Contrast this with the speed and parity, which are global to all paths to 
a specific terminal but which cannot be changed by unprivileged software.
.H 4 "SETTERMMODES2 I/O function"
.P
The speed and parity of the terminal are set with the SETTERMMODES2
function.  The speeds available depend upon the hardware available.
The DH11 has 16 speeds, which are selected by specifying a number from
0 to 15 in
.I blockNo[1]
in the K_device_function call.  The DL11 does not support
computer-controlled speed selection, so a SETTERMMODES2 function will
be rejected (XnoFunct) if sent to a DL11 device.  Parity checking and
generation may be suppressed (again, for the DH11 only) by placing a
1 in
.I "blockNo[1]"
when performing the SETTERMMODES2 function.  When parity is
not suppressed, even parity is checked and generated.
.P
Because the speed and parity mode settings of the terminal could affect
recognition of the secure attention character, use of this function
requires a privilege.
.H 4 "SETEOMCHARS I/O function"
.P
When
.I break_on_all_characters
is off, an input is considered complete when
any of the following events occur:
.BL
.LI
128 characters have been accumulated
.LI
A secure attention character is received
.LI
An ASCII DEL (0177 octal) is received
.LI
A control character (0 octal to 037 octal) is received and the
corresponding bit is on in the terminal's mask of end-of-message characters.
.LE
.P
The terminal's mask of end-of-message characters is set by submitting
a 32-bit mask in the
.I blockNo
argument to
.I K_device_function
when performing a SETEOMCHARS function.
For this function, the
.I blockNo
argument is treated as type
.B bits32.
For example the constant
.DS

	bits32([13],[])

.DE
would represent a mask which caused only the ASCII CR character to be treated as
an end of input.  Only the control characters can be set as end of input
delimiters; for applications requiring more interaction the
.I break_on_all_characters
mode must be used.
No privilege is required to perform this function, but the terminal must
be opened for reading and writing.  There is a separate mask for each path
to the terminal.
.H 4 "SETPATH I/O function"
.P
The SETPATH I/O function is used to change the active path to the terminal.
The path desired is submitted in the
.I blockNo
argument.
Path 0 is reserved for use by the Secure Server, and a secure attention
from the terminal will switch the path to path 0.  The calling process is
never delayed by the SETPATH function, and the path change takes effect with
the next
.I K_write_block
for output and the next input character for input.
The modes of the terminal change when the path changes if required.
At present, the input buffer is not flushed when the path changes.
If necessary, this can be changed easily.
This function can only be performed by callers with the privilege
.I privSetPath.
.H 2 "Extents"
Extents are fixed-size contiguous areas on addressable devices.
The size, location, and security properties of extents are defined by
information in a fixed location on the disk.  This information is
read by the Kernel but maintained by the NKSR file system initializer
utility.  Extents may be used as file systems or may be used for other
purposes; the subtype mechanism prevents access to file system extents by
processes not possessing the file system subtype.   The only valid operations
for extents are READ and WRITE.  Extents are addressable in units of 512
bytes but transfers of any even size up to the limit of the underlying device
are allowed.  Writes of short blocks cause the remainder of the block to
be filled with zero.
.P
Every extent READ or WRITE generates exactly one device operation.
All referencing of the control information on disk
is performed during the K_open,
so overhead is low.   While the primary purpose of extents is to allow
multiple file systems on a disk as an administrative convenience,
extents are useful to programs with large, specially organized databases.
.H 2 "Devices"
The KSOS device system makes the hardware devices accessable to the user.
All device operations go through the device system, even those
initiated by the Kernel extent system, file system, terminal system, or
swap manager.  Devices are divided into two general categories; addressable
and non-addressable.  Addressable devices are storage media organized in
512 byte blocks.  Everything else is considered non-addressable.
While some effort is made to make addressable devices present a
uniform appearance to the user, no attempt is made to make non-addressable
devices look like anything other than what they are.  For example,
the user functions for tape units are READ, WRITE, WRITEMARK, REWIND, and
SETDENSITY, and UNLOAD.
.H 3 "RK05 cartridge disk"
The RK05 is an addressable device and may support extents and file systems.
The only supported functions are READ and WRITE.
.H 3 "RP06 removable disk"
The RP06 is an addressable device and may support extents and file systems.
The supported functions are READ, WRITE, UNLOAD, VOLUMEVALID, and
VOLUMEINVALID.  When a new pack is mounted, all functions are rejected
until a VOLUMEVALID privileged function is sent to acknowledge the pack.
VOLUMEINVALID and UNLOAD are equivalent, and will stop the drive.
.H 3 "TWE16 tape drive"
The TWE16 is not an addressable device.  The supported functions are
READ, WRITE, REWIND, UNLOAD, and SETDENSITY.  REWIND rewinds to load point;
UNLOAD rewinds to load point and takes the drive off-line.
Only 9-track drives are supported.
.P
The SETDENSITY function takes the density specified in the block number
argument (800 or 1600) and saves it for future operations.
SETDENSITY
sets the density for writing, and for older drives (usually those
equipped with TM02 formatters) also sets the density for reading.
Newer drives set read density automatically.
Density should not be changed unless the drive is known to be at load point.
.H 3 "LP11 line printer"
The LP11 is not an addressable device.  The only supported  function is
WRITE.  Characters sent to the device are not inspected by the Kernel.
Sending many characters (up to 132) with one kernel call is much more
efficient than sending one at a time.  Characters are accumulated by the
printer until a line is full or an ASCII CR, LF, or FF is sent.
