\" $Header: courier.tbl.me,v 1.5 85/03/26 06:30:58 jqj Exp $
\" $Log:	courier.tbl.me,v $
\" Revision 1.5  85/03/26  06:30:58  jqj
\" Revised public alpha-test version, released 26 March 1985
\" 
\" Revision 1.4  85/03/11  16:46:01  jqj
\" Public alpha-test version, released 11 March 1985
\" 
\" Revision 1.3  85/03/10  06:51:54  jqj
\" 
.fo ''%''
.(l C
.ps +4
.b
XNS Courier under UNIX
.ps -4
.sp 2
.r
.he $ XNS Courier $ $Date: 85/03/26 06:30:58 $
Computer Science Department
Cornell University
405 Upson Hall
Ithaca, NY  14853
.)l
.sh 1 "Introduction"
.pp
This document describes the implementation and use
of the Courier remote procedure call protocol for Berkeley UNIX
(version 4.2), with XNS.
.sp 1
.nf
.b
.(c
********************************************************
Warning:  this document is a DRAFT design specification.
It is not yet fully implemented, and the details of the
interface it specifies (particularly with respect to 
Courier servers) are very likely to change.  Send design
comments to jqj@cornell.
********************************************************
.)c
.r
.fi
.sp 1
.uh "Changes since previous revision:"
.np
Changed names of generated file names to include version number.
Added discussion on scope of names vs. DEPENDS UPON modules.
.np
Users must now link Courier program ``Foo'' with Foo1_support.o.
.np
Added some discussion of freeing dynamic storage via clear_Foo.
.uh "Earlier changes:"
.np
Changed
.i CourierOpen()
to take xn_addr structure rather than a string host name.  Routines for
converting from name to address (using Clearinghouse) have not yet been
designed. 
.np
Changed ERROR discussion to indicate that an ERROR is mapped to a long
with value > 65535.
.np
Changed description of BDT to reflect use of 
.i BDTread()
and
.i BDTwrite() .
.np
Added SPPMAXDATA.
.np
Changed discussion of BulkData constants to indicate that things now
work correctly.
.sh 2 "General Description"
.pp
Courier is both a protocol and a specification language.
The protocol describes how remote procedures are invoked and
how parameters and results of various data types are transmitted.
The specification language, somewhat reminiscent of Mesa,
provides a simple way of defining the remote interfaces of distributed
programs.
.pp
This implementation is an attempt to allow UNIX C programmers to
construct distributed Courier programs to communicate with other
Courier implementations layered on XNS.  It thus requires kernel
support for XNS Sequeneced Packet Protocol sockets.
.pp
The simplest form of distributed program using Courier
consists of three parts.
The first is the specification,
written in the Courier language.
The specification must declare all procedures of
the program that will be called remotely (the
.i "server procedures" ).
The next part is the implementation, in C, of the server procedures.
Finally, there is the client portion of the program, also in C, which calls
the server procedures.
Note that either the client or server portion of the program may be
omitted if the UNIX program is designed to act with a non-UNIX
partner; for example, a simple printing client under UNIX might send
files to a Xerox laser printer using the Printing protocol published
and supported by Xerox.
.sh 2 "Acknowledgements"
.pp
Contributors to the current implementation have included
JQ Johnson, the primary implementor of the current version of the
Courier compiler under UNIX,
Eric Cooper, the author of the original UNIX Courier/TCP compiler,
Jeff Mogul, the author of the exception signalling mechanism used
here to handle abort and reject messages,
James O'Toole, Chris Torek, and Mark Weiser,
the authors of the UNIX kernel support for XNS,
Lee Moore, and Bill Nesheim.
.pp
Some of the software is copyright 1984 by Eric Cooper, and is for
research use only; it may be freely distributed but may not be
sold as a commercial product.
.sh 2 "References"
.pp
This document assumes familiarity with the Courier language,
details of which may be found in
the Courier protocol definition,
.i "Courier: The Remote Procedure Call Protocol" ,
Xerox System Integration Standard 038112,
December 1981, and ``Appendix F, Bulk Data Transfer,'' Xerox System
Integration Standard 038112 Add. 1, October 1982.
However, for reference purposes, a grammar for the Courier language
may be found in the appendix to this document.
.pp
Users of Bulk Data Transfer will need to interact with SPP 
(Sequenced Packet Protocol)
streams, and may wish to see
.i "Internet Transport Protocols" ,
Xerox System Integration Standard 028112, December 1981.
.pp
Those interested in the workings of UNIX networking might consult
``A 4.2bsd Interprocess Communications Primer,''
(DRAFT of March 11, 1983), by Samuel J. Leffler, Robert S. Fabry, and
William N. Joy, and the 
``4.2BSD System Manuel,''
(Revised July, 1983), by William Joy 
.i "et al" .
.pp
This document is also supplemented by various UNIX manual pages, notably
.i "courier(3) " and
.i "except(3)" .
.sh 2 "Software and Hardware Dependencies"
.pp
This package requires the following software to operate:
.(l
4.2BSD on a VAX (will be ported to other 4.2BSD implementations soon)
Maryland XNS implementation
PCC (probably \- not tested with any other C compiler)
.)l
.pp
To communicate with Xerox products you will probably need an Ethernet
interface.  Supported interfaces include 3Com, Interlan, and DEC DEUNA,
among others.
.sh 1 "The Courier Specification Language"
.pp
The Courier specification language is documented (loosely) in the Xerox
``Courier: The Remote Procedure Call Protocol.''  Our compiler
implements a large subset of the specification.  Note that the embedding
of Courier constructs in a message or bit stream is usually irrelevant to the
applications programmer, who need only be concerned with the semantics
of the Courier datatypes.  The courier compiler and runtimes translate
Courier datatypes into more easily managed C constructs and handle the
protocol of communication with the remote system.
.pp
This implementation places several restrictions on the specifications
it will accept:
.np
No forward references are permitted in the courier
specification.  The user must reorder type specifications so that all
types and constants are defined before they are used.
Special provisions are made for recursive type declarations of the form
typically found in Bulk Data ``StreamOf'' declarations; see the
discussion of BDT below for details.
.np
The Courier specification is unclear as to lexical issues, 
name scope, and name overloading.
In the current compiler, names should contain only upper and lower case
letters or digits.  Also, all constants, types, and
enumeration tags should have distinct names.
.np
Constants of type CHOICE are not fully supported.  In particular, they
are allowed only if the active arm of the choice is a nill record.
.np
This implementation supports DEPENDS UPON and
referenced types and constants.  However, the semantics of referenced
enumeration constants is unclear; stay tuned.  Also, this implementation
does not support recursive dependencies.
.lp
We think we've eliminated all other restrictions, but we're probably wrong.
.sh 2 "Predefined types"
.pp
The following typedefs correspond to predefined Courier types:
.(b
.TS
c c c
l l l.
Courier type	C typedef	Implemented (on VAX) as

BOOLEAN	Boolean	char (0==TRUE, 1==FALSE)
CARDINAL	Cardinal	unsigned short
LONG CARDINAL	LongCardinal	unsigned long
INTEGER	Integer	short
LONG INTEGER	LongInteger	long
STRING	String	char*
UNSPECIFIED	Unspecified	unsigned short
LONG UNSPECIFIED	LongUnspecified	unsigned long
.TE
.)b
Note that the representations of these types as seen by the applications
programmer are quite different from those sent over a Courier SPP connection.
For example, on the VAX a Cardinal's bytes are swapped before being
sent out in network byte order.  A UNIX String corresponds to a pointer to
a null-terminated array of characters while a Courier serialized STRING
contains a count directly followed by an array of characters.  Note
in particular that this implementation does
not support Courier STRINGs containing the ascii character NUL.
.sh 2 "Constructed types"
.pp
The Courier enumeration, array, and (non-null) record types
correspond to C enumerations, arrays, and structures respectively.
The Courier null record corresponds to an int.
.pp
The Courier sequence and choice types
pose some problems when they are mapped into C.
This is because an object of one of these types
must contain run-time information (the length of the sequence
or which choice is present)
that is implicit in Courier, but must be made explicit
in C.
Furthermore, the C programmer must bear the responsibility of keeping
this information consistent.
.pp
A sequence type is mapped into a structure consisting
of a Cardinal called
.i "length" ,
and a pointer to the sequence elements called
.i "sequence" .
The consistency requirement is that
.i "length"
indicate the number of elements in the array pointed to
by
.i "sequence" .
.pp
A choice type is mapped into a structure consisting
of an enumeration element called
.i "designator" ,
and a union of all the possible choices.
The designator is of the enumeration type defined
(implicitly or explicitly)
in the declaration of the choice type.
If this enumeration consists of elements
.i A ,
.i B ,
and
.i C ,
then the choices are accessible as
.i A_case ,
.i B_case ,
and
.i C_case .
The consistency requirement is that
.i "designator"
contain the enumeration value corresponding to
which choice currently occupies the union.
.sh 2 "Constants"
.pp
The Courier specification is silent on the precise representations of
numbers and strings.  In this implementation, numbers may be represented
as an optional minus sign followed by a
sequence of decimal digits or a sequence of octal digits followed
by ``B''.  A constant of type STRING 
is represented by a sequence of characters enclosed in
double quotation marks; it has the same restrictions as a C literal string (no
embedded newlines, use ``\e'' as a prefix character, etc.) except that double
double quotes are permitted to indicate quotations inside the string.
.sp 1
.nf
minInt: INTEGER = -2147483648;		-- or 20000000000B --
lastCard: CARDINAL = 177777B;		-- or 65535 --
quotedName: STRING = "my name is ""jqj""\en";
.fi
.sp 1
.pp
Although the Courier language allows constants of
any type to be declared,
it is difficult to define constants of constructed types
in C programs.
This implementation normally handles constants by creating
static variables initialized to the values of the (possibly structured)
constants.  The technique is not in general possible for structured 
constants of type CHOICE
since they are implemented as C ``unions'', so this implementation does not
support constants which contain a non-null CHOICE.  As a special case,
constants whose selected variant is a null record are allowed, e.g.
.sp 1
	myImmediate: BulkData.Immediate = immediate [];
.sp 1
.sh 2 "Procedures"
.pp
PROCEDURE constants correspond to C functions.
Procedure declarations may include argument lists, result lists, and
error lists.  The argument list of the corresponding C function 
contains one parameter for each parameter specified in the declaration,
but is
expanded by the addition of 2 parameters (see below).  The result list
specifies a C structure to be returned by the C function containing the
results; for a procedure, Foo, this result list is of type FooResults.
Thus, a procedure declared
.sp 1
	Example: PROCEDURE [ ] RETURNS [ a: Foo, b: Baz ] = 0;
.sp 1
would be implemented by a C function with no arguments
returning a structure of type ExampleResults with definition
.sp 1
.nf
	typedef struct {
		Foo a;
		Baz b;
	} ExampleResults
.fi
.sp 1
Note that Foo and Baz might be arbitrarily complex structures;
see ``Constructed Types'' above.
.sh 2 "ERROR constants"
.pp
An ERROR constant corresponds to a defined numeric value, and possibly
to a C structure describing its argument list.  For example,
.sp 1
	OtherError: ERROR [ errorstring: STRING ] = 1;
.sp 1
corresponds to the C declarations:
.sp 1
.nf
	#define OtherError (1+ERROR_OFFSET)
	typedef struct {
		String errorstring;
	} OtherErrorArgs;
.fi
.sp 1
Note that ERROR values must be specified in the range 0 to 65535, but
that they produce a symbol which appears to the C programmer
with values of type int
in the range 65536 to 131171 (in some contexts, the user may wish to
treat C error values in the range 0 to 65535 as Unix error numbers).
The structure corresponding to the error
may be useful in creating and referencing error arguments
in C functions.
.sh 2 "Scope of Names"
.pp
Given a courier program beginning
.sp
	Example : PROGRAM 537 VERSION 1 =
.sp
types and constants declared in this program belong to the module ``Example1'';
Their full names have prefixed to them the program name and version number.
Given the declaration of ``minInt'' above, the XNS Courier compiler actually
produces a header file (here
.i Example1.h )
containing the C declaration:
.sp 1
.nf
static Integer Example1_minInt = {-2147483648};
.fi
.sp 1
.pp
This prefixing implies that a C programmer may refer to several different
remote Courier programs without risk of name collision.
For programmer convenience a second header file (here
.i Example1_defs.h )
is also created by the compiler containing macro definitions which widen
the scope to ``Example1'' and allow the programmer to refer to types and
constants without explicit qualification.  The programmer may include
whichever header file he prefers.
.pp
Note that types and constants obtained from a DEPENDS UPON inclusion must
be referenced using fully qualified form.  Note also that structure members,
procedure arguments, and enum values are not currently qualified; beware
potential name conflicts!
.sh 1 "Running the Courier Compiler"
.pp
The specification file is expected to have the extension
.i ".cr" .
Consider the following skeletal Courier program definition:
.sp
	Example : PROGRAM 537 VERSION 1 = BEGIN ... END.
.sp
The name of this Courier program is ``Example''; it is version 1.
By convention, this specification would stored in the file
.i "Example1.cr" .
The ``537'' in this example is illustrative only;
the program number is uniquely allocated from the range of LongCardinals
by Xerox.  See Appendix B of the Courier standard for program number
assignment procedures.
.pp
The first step is to use the Courier compiler on the specification,
by doing 
.sp 1
	xnscourier Example1.cr
.sp 1
Assuming there are no errors in compilation,
the following files will have been produced:
.(b
.TS
c c
l l.
File	Contents

Example1_defs.h	scope widening macros (includes Example1.h)
Example1.h	definitions, typedefs, and constant declarations
Example1_support.c	routines to map between C and Courier
Example1_server.c	server main program and support routines
Example1_client.c	client routines to support applications calling Example
.TE
.)b
.pp
The header file
.i "Example1_defs.h"
should be included via
.sp
	#include ``Example1_defs.h''
.sp
in all user-written parts of the Courier program
(i.e., the implementations of the client program and server procedures.)
.sh 2 "Calling Remote Procedures"
.pp
A remote procedure appears to the C programmer to look almost like
a local procedure (although it typically makes use of resources not
available on the local machine or else runs many times slower
than would a local procedure).  A C client program (i.e. the caller
of a remote procedure) actually calls a stub function generated by
the courier compiler, which in turn executes the code necessary to
invoke the corresponding procedure on the remote machine.  
.pp
Before calling the remote procedure it is necessary to establish a
Courier SPP connection (in the process binding a local socket to
a remote machine).  This is done by the calling the function
CourierOpen(), passing it as argument a structure identifying the remote
host.  This structure
is of type ``xn_addr''
as defined in the include file
.i "#include <netxns/xn.h>" .
CourierOpen() returns an anonymous pointer
(of type CourierConnection*) to a structure containing
the socket to be used for this courier call, which should be passed
to each remote procedure to be executed on the particular remote host.
CourierOpen() returns NULL if there is a problem opening an SPP connection
to the host specified.
.pp
As a temporary measure (until Clearinghouse lookup routines are coded),
the procedure getXNSaddr() is available, taking a string in the form
``network#a1.a2.a3.a4.a5.a6#socket'' (where all numbers are hex) or 
``networkhigh-networklow#d1-d2-d3-d4#socket'' (where all numbers are
3-digit decimal numbers)
and returning a pointer to the
corresponding xn_addr structure.  This result may then be passed as the
argument to CourierOpen().
.b ">>>add real routines soon!<<<"
.pp
The client then calls the compiler-generated stub function to invoke
the remote procedure.  This stub
function takes at least 2 arguments:
.ip  (1)
The pointer returned from CourierOpen().
If the SPP connection does not
currently exist (i.e. has been closed by the server)
it is reopened for this call.
.ip (2)
A pointer to a user-supplied function which will be used if a Bulk Data
Transfer is required (see the section on BDT below); if this remote
procedure does not involve Bulk Data Transfer, then the function pointer
should be 0 or NULL.
.ip (3...)
One additional argument corresponding to each parameter in the Courier
language description of this procedure.
.pp
The remote procedure returns in one of 2 ways:  either it returns
normally, passing back a structure corresponding to the RETURNS parameters
specified in the procedure description, or it signals an error which
may be caught using the DURING ... HANDLER mechanism described in
.i except(1) .
.pp
A Courier procedure is allowed to return multiple results, a language feature
not easily mapped into C.  To provide this functionality, all UNIX
courier remote procedures actually return a structure
containing the procedure results.  Thus, for a procedure named
Foo, declared
.sp 1
	Foo: PROCEDURE [ ] RETURNS [str: STRING];
.sp 1
the C function Foo would return a structure of type FooResults, declared
.sp 1
.nf
	typedef struct {
		String str;
	} FooResults;
.fi
.fi
.pp
Instead of returning a value of type determined by the RESULTS clause
of a PROCEDURE declaration, a Courier remote procedure call may return
a reject message indicating the remote system's inability to even
attempt a remote operation, or an abort message raising a remote error,
i.e. reporting the operation's failure.  Reject and abort messages are
handled by a signalling mechanism  or if uncaught cause termination of
the client program.  For example, a client calling the remote procedure
defined by
.sp 1
.nf
	NoSuchFile: ERROR = 0;
	OtherError: ERROR [ errorstring: STRING ] = 1;
	Example: PROCEDURE [ ] REPORTS [ NoSuchFile ];
.fi
.sp 1
might be coded as
.sp 1
.nf
	conn = CourierOpen(destaddr);
	DURING Example(conn,NULL)
	HANDLER switch(Exception.Code) {
	case REJECT_ERROR:
		fprintf(stderr,"Remote reject.\en");
		exit(1);
	case NoSuchFile:
		fprintf(stderr,"No such file\en");
		break;
	case OtherError:
		fprintf(stderr,"Remote error %s\en",
			CourierErrArgs(OtherErrorArgs,errorstring) );
		break;
	case PROTOCOL_VIOLATION:
	case INTERNAL_ERROR:
		fprintf(stderr,"Local internal error %s\en",
			Exception.Message);
		break;
	default:
		fprintf(stderr,"Unknown error, type %d\en",
			Exception.Code);
	}	
	END_HANDLER
.fi
.sp 1
The error value, Exception.code, will be one of (1) ``REJECT_ERROR'',
which indicates a REJECT message from the remote server, (2) a program-defined
ERROR value (offset by ERROR_OFFSET), (3) or ``INTERNAL_ERROR'' 
or ``PROTOCOL_VIOLATION''
indicating
a serious internal error in the Unix courier implementation; in the third
case, a string is available as Exception.Message.
Both errors and rejections may have arguments, which can be accessed
within a handler as a struct pointed to by Exception.Message.  For
programmer convenience a macro, CourierErrArgs, is defined, taking
as arguments the typedef defining the current error's arguments 
and the name of the argument to be accessed; this macro may be used
only within an error handler.
For rejection messages, the type should be ``rejectionDetails,''
as defined on page 25 of the Courier spec.
.pp
When done with a particular remote connection, the client should call
CourierClose() with argument the pointer returned from CourierOpen()
to free the socket and cleanly close the connection.
.pp
The client program should be loaded with 
.i "Example1_client.o" ,
.i "Example1_support.o" ,
and -lcourier (the XNS Courier library)
to produce an executable program.
.sh 2 "Writing a Courier Server"
.pp
The Courier protocol specifies a standard for communicating
parameters and results which has been adhered to in this
implementation.
It also specifies an initial connection protocol
and a format for messages.
.pp
There is a single Courier Daemon per machine whose function
is to listen on a well-known XNS port for Courier interface activation
requests.
A request contains the number of the Courier program
whose server is to be activated, and its version number.  These two
numbers are used as a key in the file 
.i /etc/Courierservices
to identify
the executable file associated with the particular courier program.
.i /etc/Courierservices
consists of a sequence of lines of the form
.sp 1
	program-name  program-number  version  courier-description-file  server-binary-file
.sp 1
The daemon
either spawns the executable file
or replies with a reject message.  Note that each incoming remote procedure
call may be serviced by a separate UNIX process.
This implies that connection-oriented services (services which consist of a
sequence of related procedure calls) must be implemented by maintaining
the state of connections in a file.
.pp
The executable file containing the server for a particular remote
program consists of a main program generated by the courier compiler
which interprets CALL messages from the remote client and calls
one or more user-written functions to implement the
courier procedures.
.pp
Given a courier specification for the remote program Example containing
remote procedure Example,
the server functions (including the C function Example and any support
routines needed)
should be loaded with 
.i "Example1_server.o" ,
.i "Example1_support.o" ,
and -lcourier
to produce the server process that will be invoked whenever
an activation request arrives.
.pp
A server function receives an argument list similar to that passed by
a client to a remote procedure.  The first two arguments in the
parameter list should be ignored; each succeeding argument corresponds
to one parameter in the courier declaration.
.pp
To return from a server function you must return a structure containing
the results of the procedure.  If this structure contains pointers (e.g.
Strings) you should insure that the data pointed to is allocated
staticly or on the heap rather than on the stack.
You must not call exit(3) from within the server function; doing so will
cause the client (remote caller) to hang indefinitely waiting for a
reply.
.pp
To abort a server function, returning an error code, use the
``raise(code, msg)''
library function, with arguments the error code and either NULL (if the
error takes no arguments) or a pointer to a structure containing the
arguments to the error.  Raise causes a longjump out of the function
directly to the handler, and thence to the remote procedure caller.
For example, to return the OtherError message defined above one might code
.sp 1
.nf
	OtherErrorArg randomerr;
	. . .
	randomerr.errorstring = sys_errlist[errno]);
	raise(OtherError,(char*) &randomerr);
.fi
.sp 1
.pp
When a server function returns to its main program caller the main()
sends the appropriate RETURN or ABORT message to the remote client,
and does an exit(0).  The server then holds the connection for up to 90 
seconds waiting for another Courier call to arrive.
.sh 2 "Dynamic Allocation"
.pp
One additional 
.i caveat 
is necessary when using Courier remote procedure calls from C:  C does not
have garbage collection, so you should free any dynamically allocated storage
when you are done with it.  In general, top level Courier objects are
always allocated from the stack or statically; however, strings and sequences
within an object are generally allocated from the heap using malloc().
.pp
A client program which calls the remote procedure Foo defined above,
returning a STRING,
actually has returned to it a C struct containing a pointer to an array
of malloced characters.  When done with this string, the program may free
it by calling clear_FooResults() with argument the address of the
result returned by the call to Foo.  Thus:
.sp 1
.nf
	FooResults result;
	. . .
	result = Foo(conn,NULL);
	. . .
	printf("result was %s\n",result.str);
	clear_FooResults(&result);
.fi
.sp 1
.sh 1 "Using Bulk Data Transfer"
.pp
.sh 2 "Overview"
.pp
When a Courier program needs to transfer an arbitrary amount of
information as an argument or result of a Courier procedure, the
procedure is usually defined to have an argument of type ``BulkData.Sink''
or ``BulkData.Source'' (and a ``DEPENDS UPON BulkData''
is included in the program).
The argument is a ``source'' if it is information transferred from caller
to server (as though a procedure argument), a ``sink'' if it is
information transferred from server to caller (as though a procedure
result).  In this
implementation, a Courier procedure may have at most one such argument,
which must have the value null or immediate.
In a Courier call, the bulk data is transmitted in a special way,
multiplexed into the same data path as control messages
between the arguments and the results.
.pp
The BDT user should include in the 
.i ".cr"
file a ``DEPENDS UPON BulkData (0) VERSION 1'', then reference these
arguments as BulkData.Source and BulkData.Sink in PROCEDURE
declarations.
The declaration of a Courier PROCEDURE can then indicate a bulk
data transfer by including in its formal argument list a variable
of type BulkData.Source or BulkData.Sink
as appropriate.
The corresponding parameter to the C procedure will be of type
BulkData1_Descriptor.
.sh 2 "Coding a Client"
.pp
To use Bulk Data in a client program, the Courier definition of the
procedure must include one BulkData.Source or BulkData.Sink parameter.
As the actual argument to the call, the C client passes a constant,
either BulkData1_immediateSource, BulkData1_immediateSink, BulkData1_nullSource, 
or BulkData1_nullSink as appropriate.  The client
also specifies a pointer to a user-supplied function as the second
argument to the remote procedure.
Courier sets up the transaction, then calls the supplied function with
one argument, the pointer returned by CourierOpen() describing the
SPP socket on which to write (if a source argument) or read
(if a sink) the bulk data. 
.pp
To complete the transaction normally, the
function should send packets of SPP data terminated by
an end-of-message (if writing) or read up to but not beyond an
end-of-message (if reading), and should return normally to its caller.
To do so, it should call the procedures BDTread(), BDTwrite(), or 
BDTclosewrite()
as appropriate.  BDTread() and BDTwrite() take arguments analogous to read()
and write() except that instead of a file descriptor they accept the SPP
descriptor returned by CourierOpen(); their arguments are thus
(1) the pointer to the SPP data structure,
(2) a pointer to an array of characters (the IO buffer),
and (3) a count;
they return the number of characters actually read or written.
.pp
The UNIX program can finish a write transfer by calling BDTclosewrite() or
BDTabort(), each with the SPP descriptor as argument.
BDTclosewrite() produces an SPP end-of-message, while BDTabort() instructs
Courier to discard any buffered data and
send a Bulk Data Abort to abort the transaction.  A read transfer may also be
prematurely ended by calling BDTabort().
.sh 2 "Coding a Server"
.pp
Writing a server is similar to writing the BDT function in a caller.
The server is passed a BulkData.Source or BulkData.Sink (the two are
indistinguishable at run time).
The server routine should check to make sure that the source or sink
is of type null or immediate (active and passive descriptors are not
supported in this implementation) by means of code such as:
.sp 1
.nf
Foo(bdtconnection,ignoredarg,..., s, ...)
	CourierConnection *bdtconnection;
	BulkData1_Sink s;
	. . .
	switch (s.designator) {
	case active:
	case passive:
		raise(someerrorcondition);
	case null:
		/* handle null transfer */
		break;
	case immediate:
		/* handle normal, i.e. immediate, transfer */
		break;
	}
.fi
.sp 1
.pp
Use the first argument to the server procedure as the handle on the
BDT connection.
As with a client BDTabort() may
be used to send abort messages.
.sh 2 "Sending Bulk Data"
.pp
Sending bulk data, either as client or server, is quite
straightforward.  Use the supplied procedure BDTwrite(), whose
semantics are similar to write().  BDTwrite() will return -1
if an error occurs or an abort from the listener is received; in this
case, the sender should immediately cease transmitting BDT data, and
should instead call BDTabort(). 
Typical code to send data on ``bdtconnection"",
in this case read from a buffered file open
as ``server,'' might appear as:
.sp 1
.nf
	CourierConnection *bdtconnection;
	FILE *source;
	int count;
	char buffer[SPPMAXDATA];
	...
	while ( (count = fread(buffer,1,SPPMAXDATA,source)) > 0 &&
		BDTwrite(bdtconnection,buffer,count) >= 0 )
		;
	if (count <= 0)	/* succsfull transfer */
		BDTclosewrite(bdtconnection);
	else
		BDTabort(bdtconnection);
.fi
.sp 1
.pp
Each BDTwrite() transmits one packet after checking for an abort message
from the receiver.
A sender which gets an abort from a remote receiver must immediately
stop transferring data and instead must echo an abort via BDTabort()
to acknowledge the end of the transfer.
.pp
Also, for efficiency's
sake it is desirable to write packets as near as possible to the
nominal maximum length of an SPP packet; this suggests a buffer length of
534 bytes.  For programmer convenience, the file
.i courier.h
defines the symbol SPPMAXDATA as 534.
.sh 2 "Receiving Bulk Data"
.pp
To make receiving bulk data simple, the routine BDTread() performs checks for
various conditions on receipt of each packet.  BDTread() will return the
count of the number of bytes actually read in each packet (skipping empty
packets), or 0 to indicate end of data.  If an error or BDT abort message
occurs, BDTread() will return -1; if this occurs, the receiver should 
immediately stop reading BDT data.
.pp
Typical code for receiving bulk data on the socket described by 
``bdtconnection'' and
depositing it in the file opened for buffered ouput as ``destfile''
might be:
.sp 1
.nf
	CourierConnection *bdtconnection;	/* BDT source descriptor */
	FILE *destfile;				/* output file */
	char sppdata[SPPMAXDATA];
	...
		count = BDTread(bdtconnection, sppdata, SPPMAXDATA);
		while (count > 0) {	/* actually read data */
			if (fwrite(sppdata, 1, count, destfile) == 0) {
					/* error while writing */
				BDTabort(bdtconnection);
				break;
			}
			count = BDTread(bdtconnection, sppdata, SPPMAXDATA);
		}
		if (count == 0) {
			/* successful */
			...
.fi
.sp 1
.sh 2 "StreamOf declarations"
.pp
Frequently, data to be sent on a Bulk Data connection will be described
by a StreamOfSomething declaration in the courier specification.  For
example, in Clearinghouse several routines are documented as returning
a StreamOfObjectName; unfortunately, a StreamOf declaration is recursive,
and so is not immediately translatable into automatic packing and unpacking
routines which need to know in advance how much space to allocate for the
result.  As a kludge, we translate such types as if the recursive
part of the declaration were a null record.  
Consider:
.sp 1
.nf
StreamOfObjectName: TYPE = CHOICE OF {
	nextSegment (0) => RECORD [
		segment: SEQUENCE OF ObjectName,
		restOfStream: StreamOfObjectName ],
	lastSegment (1) => SEQUENCE OF ObjectName};
.fi
.sp 1
.lp
In the above declaration, the ``restOfStream is effectively ignored.
Given this declaration, the following routine may be passed to a
remote procedure as the Bulk Data actor.
.sp 1
.nf
#include "Clearinghouse_support.c"	/* get internalize_* routines */
#define MAXPACKS 5

GetData(conn)
	CourierConnection *conn;
{
	int count, i;
	Unspecified buffer[MAXWORDS*MAXPACKS], *bp, *bufend;
	StreamOfObjectName obnames;
	
	bufend = buffer;
	bp = buffer+MAXWORDS*(MAXPACKS-1);    /* end of available space */
	while (count = BDTread(conn, (char*)bufend, 
				MAXWORDS*sizeof(Unspecified))
		) {
		bufend += count/sizeof(Unspecified);
		if (bufend > bp) {
			fprintf(stderr,"BDT read too big to fit\en");
			BDTabort(conn);
			/* should clear out stuff here if we knew how much */
		}
	}
	bp = buffer;
	while (bp < bufend) {
		bp += internalize_StreamOfObjectName(&obnames,bp);
		if (0 == (int) obnames.designator)
		   for (i = 0; i < obnames.nextSegment_case.segment.length; i++)
			ProcessObjectName(
				obnames.nextSegment_case.segment.sequence[i]);
		else {
		   for (i = 0; i < obnames.lastSegment_case.length; i++)
			ProcessObjectName(
				obnames.lastSegment_case.sequence[i]);
		   return;
		}
	}
}
.fi
.sp 1
.lp
Note that this code is very awkward, and that it requires that the whole
bulk data transfer be stored in memory before it is unpacked.  Stay
tuned; we intend to modify the compiler to make the handling of such
streams much easier, at the cost of incompatibility with the existing
scheme, of course!
.sh 1 "Example I:  Passwd.cr"
.pp
This section contains a Courier program which implements remote lookup in
.i "/etc/passwd"
(the UNIX database of user names, passwords, home directories, and so
on).
It does not utilize Bulk Data Transfer, but does illustrate most other
features of this courier implementation.
The applications programmer would first write 
.i PasswordLookup.cr ,
the description of the courier interface for the service, then might
write
.i PasswordLookup.c ,
containing the routines needed to implement a server for this courier
program, or might write
.i lookup.c ,
a typical client of this service.
.bp
.sh 2 "The specification (PasswordLookup.cr)"
.sp 1
.nf
PasswordLookup : PROGRAM 754 VERSION 1 =

BEGIN

    -- This is a translation of the passwd structure in <pwd.h>

    Passwd : TYPE = RECORD [
	pw_name, pw_passwd : STRING,
	pw_uid, pw_gid, pw_quota : LONG CARDINAL,
	pw_comment, pw_gecos, pw_dir, pw_shell : STRING
    ];

    -- Remote Errors

    NoSuchUser : ERROR = 0;
    OtherError : ERROR [ errorstring: STRING ] = 1;

    -- Remote entry points.

    LookupUid : PROCEDURE [ uid : CARDINAL ] RETURNS [ passwd : Passwd ]
		    REPORTS [ NoSuchUser ]
		    = 0;

    LookupUser : PROCEDURE [ user : STRING ]
		    RETURNS [ passwd : Passwd, forward : STRING ]
		    REPORTS [ NoSuchUser, OtherError ]
		    = 1;

END.
.fi
.sp 2
.sh 2 "PasswordLookup_defs.h"
.sp 1
.nf
/*
 * Declarations for Courier program PasswordLookup.
 */
#include <courier.h>
#include <except.h>

typedef struct {
	String pw_name;
	String pw_passwd;
	LongCardinal pw_uid;
	LongCardinal pw_gid;
	LongCardinal pw_quota;
	String pw_comment;
	String pw_gecos;
	String pw_dir;
	String pw_shell;
} Passwd;

#define NoSuchUser 0

#define OtherError 1
typedef struct {
	String errorstring;
} OtherErrorArg;

typedef struct {
	Passwd passwd;
} LookupUidResult;

typedef struct {
	Passwd passwd;
	String forward;
} LookupUserResult;

extern LookupUidResult LookupUid();

extern LookupUserResult LookupUser();
.fi
.sp 2
.sh 2 "Makefile"
.sp 1
.nf
CFLAGS = -O
USEROBJS = lookup.o PasswordLookup_client.o
SRVROBJS = PasswordLookup.o PasswordLookup_server.o
LIBS = -lcr
DESTDIR = /usr/lib/courier

all:	lookup PasswordLookup

lookup:	$(USEROBJS)
	cc -o lookup $(USEROBJS) $(LIBS)

PasswordLookup:	$(SRVROBJS)
	cc -o PasswordLookup $(SRVROBJS) $(LIBS)

$(USEROBJS) $(SRVROBJS):	PasswordLookup_defs.h

PasswordLookup_defs.h \e
PasswordLookup_server.c \e
PasswordLookup_client.c:	PasswordLookup.cr
	courier PasswordLookup.cr

install:	all
	install -s PasswordLookup $(DESTDIR)

clean:
	-rm -f *.o PasswordLookup_*.c PasswordLookup_defs.h
.fi
.bp
.sh 2 "The server procedures (PasswordLookup.c)"
.sp 1
.nf
#include <stdio.h>
#include "PasswordLookup_defs.h"

extern Passwd *getpwnam(), *getpwuid();

LookupUidResult
LookupUid(CourierConnection,CourierBDTProc,uid)
	CourierConnection *CourierConnection, CourierBDTProc;
	Cardinal uid;
{
	Passwd *pw;

	pw = getpwuid(uid);
	if (pw == NULL)
		raise(NoSuchUser,NULL);
	else {
		return (*(LookupUidResult*) &pw);
	}
}

LookupUserResult
LookupUser(CourierConnection,CourierBDTProc,user)
	CourierConnection *CourierConnection, CourierBDTProc;
	String user;
{
	LookupUserResult result;
	Passwd *pw;
	FILE *fwdfd;
	static char forward[100];

	pw = getpwnam(user);
	if (pw == NULL)
		raise (NoSuchUser,NULL);
	else {
		sprintf(forward,"%s/.forward",pw->pw_dir);
		if ((fwdfd = fopen(forward,"r")) == NULL) {
			forward[0] = '\e0';
		else {
			fgets(forward,100,fwdfd);
			fclose(fwdfd);
			if (strlen(forward) < 2) (
				static OtherErrorArg randomerr = {
					"invalid forwarding file"};
				raise(OtherError,&randomerr));
		}
		result.password = *pw;
		result.forward = forward;
		return (result);
	}
}
.fi
.bp
.sh 2 "The user program (lookup.c)"
.sp 1
.nf
/*
 * Sample program to access remote password lookup.
 * Usage: lookup machine username
 */
#include <stdio.h>
#include "PasswordLookup_defs.h"

main(argc, argv)
	int argc;  char **argv;
{
	LookupUserResult result;
	Passwd passwd;
	CourierConnection *connection;

	if (argc < 3 || (destaddr = getXNSaddr(argv[1])) == NULL)) {
		fprintf(stderr,"Usage:  %s machine file ...\en",argv[0]);
		exit(1);
	}
	if ((connection = CourierOpen(destaddr)) == NULL) {
		fprintf(stderr,"Can't open connection to %s\en",argv[1]);
		exit(1);
	}
	DURING
		result = LookupUser(connection,NULL,argv[2]);
	HANDLER
		switch(Exception.Code) {
	case Courier_reject:
		fprintf("Connection rejected, code = %d\en",
			CourierErrArgs(rejectionDetails,designator) );
		exit(1);
	case NoSuchUser:
		printf("User %s unknown on %s.\en", argv[2], argv[1]);
		exit(0);
	case OtherError:
		fprintf(stderr,"Remote error %s\en",
			CourierErrArgs(OtherErrorArg,errorstring) );
		exit(1);
	}
	END_HANDLER;

	displaypwd(& result.passwd);
	displayfwd(result.forward);
	CourierClose(connection);
}

displaypwd(p)
	Passwd *p;
{
	printf("%s:%s:%d:%d:%s:%s:%s\en",
		p->pw_name, p->pw_passwd, p->pw_uid, p->pw_gid,
		p->pw_gecos, p->pw_dir, p->pw_shell);
}

displayfwd(s)
	String s;
{
	if (*s) printf("Mail forwarding to %s\en",s);
	else	printf("Mail is not forwarded\en");
}
.fi
.bp
.sh 1 "Example II: PrintFile"
.pp
This example is a very simple application of Bulk Data Transfer.
.sh 2 "PrintFile.cr"
.sp 1
.nf
PrintFile: PROGRAM 756 VERSION 1 =
BEGIN

    DEPENDS UPON BulkData (0) VERSION 1;

    -- Remote errors.

    CantPrint: ERROR = 0;

    -- Remote entry points.

    RPrint: PROCEDURE [ s: BulkData.Source ]
		REPORTS [ CantPrint ];

END.
.fi
.sp 1
.sh 2 "A typical client (remoteprint.c)"
.pp
.sp 1
.nf
/*
 * Sample proram to print a file remotely using trivial remote print
 * protocol.
 * Usage:  remoteprint filename
 *
 */
#include <stdio.h>
#include <sys/types.h>
#include <netxns/xn.h>		/* for XNS addresses */
#include "PrintFile_defs.h"

static FILE * source;	/* communicate from main to SendSource */

main(argc, argv)
	int argc;
	char *argv[];
{
	CourierConnection *connection;
	struct xn_addr *destaddr;

	if (argc < 3 || (destaddr = getXNSaddr(argv[1])) == NULL)) {
		fprintf(stderr,"Usage:  %s machine file ...\en",argv[0]);
		exit(1);
	}
	if ((connection = CourierOpen(destaddr)) == NULL) {
		fprintf(stderr,"Can't open connection to %s\en",argv[1]);
		exit(1);
	}
	argv++;
	while (argc-- > 2) {
		argv++;
		if (strcmp(argv[0],"-") == 0) source = stdin;
		else source = fopen(argv[0],"r");
		if (source == NULL)
			fprintf(stderr,"Can't open %s\en",argv[0]);
		else DURING
			RPrint(connection,SendSource,immediateSource);
		    HANDLER
			fprintf(stderr,"Call to RPrint failed.\en");
		    END_HANDLER;
		fclose(source);
	}
	CourierClose(connection);
}

SendSource(bdtconnection)
CourierConnection *bdtconnection;
{
	int count;
	char buffer[SPPMAXDATA];

	while ( (count = fread(buffer,1,SPPMAXDATA,source)) > 0 &&
		BDTwrite(bdtconnection,buffer,count) >= 0 )
		;
	if (count <= 0)
		BDTclosewrite(bdtconnection);	/* last packet with EOM set */
	else
		BDTabort(bdtconnection);
}
.fi
.sp 2
.sh 2 "Server (PrintFile.c)"
.sp 1
.nf
#include <stdio.h>
#include <sys/types.h>
#include <netxns/xn.h>
#include "PrintFile_defs.h"

RPrintResult
Rprint(source,CourierBDTProc,s)
	CourierConnection *source, CourierBDTProc;
	BulkData1_Source s;
{
	FILE *printpipe;
	char sppdata[SPPMAXDATA];

	switch (s.designator)
	case active:
	case passive:
		raise(CantPrint);
		/*NOTREACHED*/
	case null:
		system("print /dev/null");	/* print a null file */
		return;
	case immediate:
		if ((printpipe = popen("print","w")) == NULL) {
			raise(CantPrint);
			/*NOTREACHED*/
		}
		count = BDTread(source, sppdata, SPPMAXDATA);
		while (count > 0) {	/* actually read data */
			if (fwrite(sppdata, 1, count, printpipe) == 0) {
				BDTabort(source);
				break;
			}
			count = BDTread(source, sppdata, SPPMAXDATA);
		}
		if (pclose(printpipe) == 0 && count == 0)
			return;
		else raise(CantPrint);
}
.fi
.sh 1 "One Final Example"
.pp
Finally, we present a slightly 
more useful example, a program to print an Interpress
file on a Services-8 printer.  It depends on the standard Xerox Printing
specification, program 4 version 3.
.sp 2
.nf
#include <stdio.h>
#include <sys/types.h>
#include <netxns/xn.h>
#include "Printing_defs.h"
#include <except.h>

static FILE *ipfile = NULL;

SendSource(bdtconnection)
CourierConnection *bdtconnection;
{
	int count;
	char buffer[SPPMAXDATA];

	while ( (count = fread(buffer,1,SPPMAXDATA,ipfile)) > 0 &&
		BDTwrite(bdtconnection,buffer,count) >= 0 )
		;
	if (count <= 0)
		BDTclosewrite(bdtconnection);	/* last packet with EOM set */
	else
		BDTabort(bdtconnection);
}

main(argc, argv)
	int argc;
	char *argv[];
{
	PrintResults result;
	struct xn_addr *destaddr;
	CourierConnection *conn;
	extern struct xn_addr *getXNSaddr();
	PrintAttributes attributes;
	PrintOptions options;

	/* use Cornell print server, CornellS1 (proof) */
	destaddr = getXNSaddr("2-273#2-852-159-207");
	attributes.length = 0;
	options.length = 0;
	if (argc != 2 || ((ipfile = fopen(argv[1],"r")) == NULL)) {
		fprintf(stderr,"Usage: %s file\en",argv[0]);
		exit(1);
	}
	if ((conn = CourierOpen(destaddr)) == NULL) {
		fprintf(stderr,"Can't open connection to %s\en",xnshost);
		exit(1);
	}

	DURING
		result = Print(conn, SendSource, BulkData1_immediateSource,
					attributes, options);
	HANDLER {
		switch (Exception.Code) {
		case Busy:
			fprintf(stderr,"Busy\en");
			break;
		case ConnectionError:
			fprintf(stderr,"Connection error, %d\en",
				CourierErrArgs(ConnectionErrorArgs,problem));
			break;
		case InsufficientSpoolSpace:
		case SpoolingQueueFull:
			fprintf(stderr,"Insufficient spool space\en");
			break;
		case SpoolingDisabled:
			fprintf(stderr,"Spooling disabled\en");
			break;
		case MasterTooLarge:
		case TooManyClients:
		case ServiceUnavailable:
		case SystemError:
		case InvalidPrintParameters:
		case MediumUnavailable:
		case TransferError:
			fprintf(stderr,"Some Printing error, number %d\en",
				Exception.code-ERROR_OFFSET);
			break;
		case Undefined:
			fprintf(stderr,"Undefined error, number %d\en",
				CourierErrArgs(UndefinedArgs,problem));
			break;
		case REJECT_ERROR:
			fprintf(stderr,"REJECT:  type = %d\en",
				CourierErrArgs(rejectionDetails, designator));
			break;
		default:
			fprintf(stderr,"Some random error, code %d\en",
				Exception.Code);
			break;
		}
	exit(1);
	} END_HANDLER;

	CourierClose(conn);
	printf("Done.  Request ID %x %x %x %x %x\en",
		result.printRequestID[0], result.printRequestID[1],
		result.printRequestID[2], result.printRequestID[3],
		result.printRequestID[4]);
}
.fi
.bp
.sh 1 "Final Notes"
.pp
The issues of authentication and protection are
difficult.
They are only touched upon in this implementation,
by making the Courier daemon spawn each server process with privileges
equivalent to the protection of the corresponding executable file in
/usr/lib/courier.
Currently, each Courier program must perform any further authentication
(presumably using the Authentication protocol)
if this is desired.
.pp
This implementation is fairly inefficient, especially in
the implementation of servers.  Courier calls require substantial
extra copying of courier arguments and results.  More significantly,
the requirement that each call spawn a unique process is very expensive
in UNIX, and should be reconsidered.
A partial fix, would be to have main() of a client look
ahead at the SPP stream
and if the next packet specifies the same program and version number
loop to beginning.
.pp
This implementation defines a number of reserved identifiers.
.b ">>>should list them here<<<"
.sh 1 "Appendix"
.pp
This appendix contains the grammar for the Courier language.
It is similar to the YACC specification used
by the Courier compiler.
.sp
.ps -2p
.vs -2p
.nf
.TS
l l l l l.
%token	identifier	number	string	
	ARRAY	BEGIN	BOOLEAN	CARDINAL
	CHOICE	DEPENDS	END	ERROR
	FALSE	INTEGER	LONG	OF
	PROCEDURE	PROGRAM	RECORD	REPORTS
	RETURNS	SEQUENCE	STRING	TRUE
	TYPE	UNSPECIFIED	UPON	VERSION
.TE
%%

Program :
		identifier ':' PROGRAM number VERSION number '='
		BEGIN DependencyList DeclarationList END '.'
	|
		identifier ':' PROGRAM '='
		BEGIN DependencyList DeclarationList END '.'
	;

DependencyList :
		/* empty */
	|	DEPENDS UPON ReferencedProgramList ';'
	;

ReferencedProgramList :
		ReferencedProgram
	|	ReferencedProgramList ',' ReferencedProgram
	;

ReferencedProgram :
		identifier '(' number ')' VERSION number
	;

DeclarationList :
		/* empty */
	|	DeclarationList Declaration
	;

Declaration :
		identifier ':' TYPE '=' Type ';'
	|	identifier ':' Type '=' Constant ';'
	;

Type :
		PredefinedType
	|	ConstructedType
	|	ReferencedType
	;

PredefinedType :
		BOOLEAN
	|	CARDINAL
	|	LONG CARDINAL
	|	INTEGER
	|	LONG INTEGER
	|	STRING
	|	UNSPECIFIED
	|	LONG UNSPECIFIED
	;

ConstructedType :
		'{' CorrespondenceList '}'
	|	ARRAY NumericValue OF Type
	|	SEQUENCE MaximumNumber OF Type
	|	RECORD '[' FieldList ']'
	|	RECORD '[' ']'
	|	CHOICE DesignatorType OF '{' CandidateList '}'
	|	PROCEDURE ArgumentList ResultList ErrorList
	|	ERROR ArgumentList
	;

ReferencedType :
		identifier
	|	identifier '.' identifier
	;

CorrespondenceList :
		Correspondence
	|	CorrespondenceList ',' Correspondence
	;

Correspondence :
		identifier '(' NumericValue ')'
	;

MaximumNumber :
		NumericValue
	|	/* empty */
	;

NumericValue :
		number
	|	ReferencedConstant
	;

DesignatorType :
		/* empty */
	|	ReferencedType
	;

CandidateList :
		Candidate
	|	CandidateList ',' Candidate
	;

Candidate :
		DesignatorList '=''>' Type
	;

DesignatorList :
		Designator
	|	DesignatorList ',' Designator
	;

Designator :
		identifier
	|	Correspondence
	;

ArgumentList :
		/* empty */
	|	'[' FieldList ']'
	;

ResultList :
		/* empty */
	|	RETURNS '[' FieldList ']'
	;

ErrorList :
		/* empty */
	|	REPORTS '[' NameList ']'
	;

FieldList :
		Field
	|	FieldList ',' Field
	;

Field :
		NameList ':' Type
	;

Constant :
		PredefinedConstant
	|	ConstructedConstant
	|	ReferencedConstant
	;

PredefinedConstant :
		TRUE
	|	FALSE
	|	number
	|	'-' number
	|	'"' string '"'
	;

ConstructedConstant :
		identifier
	|	'[' ElementList ']'
	|	'[' ComponentList ']'
	|	'['']'
	|	identifier Constant
	|	number
	;

ReferencedConstant :
		identifier
	|	identifier '.' identifier
	;

ElementList :
		Constant
	|	ElementList ',' Constant
	;

ComponentList :
		Component
	|	ComponentList ',' Component
	;

Component :
		NameList ':' Constant
	;

NameList :
		identifier
	|	NameList ',' identifier
	;
.fi
.vs
.ps