Subject: An anti-anti-proposal for I/O

From: Piyush Mehrotra <[email-masked]>
Date: Tue, 8 Sep 92 16:39:32 -0400

An Anti-Anti-I/O-Proposal:

There is some sentiment that nothing needs to be added to FORTRAN 90 I/O.
However, we perceive this as a problem, since FORTRAN 
enforces a sequential ordering (the column-major ordering)
on the data elements that are written out.  The point is
that relaxing this ordering may allow the compiler
and/or runtime system to optimize the input/output
operations.

For example, consider the situation where
a distributed array is written out and then read into
a target array with the same distribution.
Here, both the writing and the reading of the elements
is going to be faster if each process can just write and
read the portion of the array that it owns. However, this may not
be possible if the FORTRAN ordering is to be preserved.

Secondary storage is used for two purposes:
a) to temporarily store scratch data which is later
reused in a program, and
b) to communicate data to other programs (which includes 
display to user).

In the first case, it is possible that the compiler can analyze
the program and figure out that the ordering can be relaxed.
However, in the second case it is only the user that can 
provide any hints about the subsequent usage of the data.

We propose to add a new directive to HPF which would allow
the user to provide some hints to the compiler/runtime system
regarding the possible distributions of the target arrays.
Note that if the FORTRAN sequence is relaxed, the file management
system would have to modified so that it keeps track of the layout 
of the data in the different files.

    io-distribute-directive	is 	IO_DISTRIBUTE	dist-stuff

where  dist-stuff is as currently defined for distribution of
objects to processor arrangements.

An IO_DISTRIBUTE directive associates an io-distribution
with the arrays specified in the distributee-list.
An io-distribution for an array is a hint
to the compiler/runtime system that the data in the array will 
be read into an array of the specified distribution.  
Note that the io-distribution provides information about
the subsequent usage of data on files and does not imply
anything about the layout of the data in the I/O subsystem.
The system can  use this information to choose the layout 
so as to optimize the output and/or the input operations.

The IO_DISTRIBUTE directive may occur in the specification-part 
of a program unit or in the execution-part or in both. In the execution
part , the IO_DISTRIBUTE directives must immediately follow a WRITE
statement as attached declarations.  In this
situation they are treated more as executable statements than as
declarations.  (There is no syntactic ambiguity about whether they
are in the declaration part or execution-part of a program unit
because in this situation they follow an executable statement and
therefore are necessarily in the execution-part.)  Such attached
declarations may declare only arrays being output by the WRITE
statement.  Expressions in such directives may be any integer
expressions; they are not restricted in this situation to be
specification expressions.

If an array identifier appears in a IO_DISTRIBUTE directive in a
specification-part, then the associated io-distribution is propagated
to each WRITE statement in the program unit. If there
is a conflict between the specification in the specification-part and
a specification in the executable-part, then the latter prevails.

Omitting the dist-attribute-stuff (see syntax rules page 24 of the
draft) or using a `*' in its place signifies that the
io-distribution is the same as the actual distribution of the array.

If the user does not explicitly associate an io-distribution
with an array, the user is  signifying that there is not enough 
information available about the subsequent usage of the data 
and hence the compiler is free to do what it wants (as always).

Example:

    REAL A(100), B(100), C(100)
    ALIGN  WITH A:: B, C
    DISTRIBUTE (CYCLIC) :: A
    IO_DISTRIBUTE  * ::  A

    ....

    WRITE(10) A, B, C
    IO_DISTRIBUTE  * :: B

    ....

    WRITE(15) A, B
    IO_DISTRIBUTE (BLOCK) :: A

Here, in the case of the first WRITE statement,
arrays A, and B have their actual distributions
(CYCLIC) as their io-distributions; A because of
the io-distribute directive in the specification part 
and B because of the attached directive.
Array C does not have an associated io-distribution.
signifying that the compiler should take the default action.
The specification, however, indicates that the data elements of 
both A and  B  as written out to unit 10 will be read into 
arrays which are distributed cyclically.

The IO_DISTRIBUTE directive  attached to the second
WRITE statement overrides the initial io-distribution of A
declaring that even though A is actually distributed cyclically,
the data written out to unit 15 is subsequently going to be read 
into a block distributed array. The array B now has no associated 
io-distribution for the purposes of this WRITE statement.

Possible extension:

Allow keyword SERIAL as another option to enforce the FORTRAN 
order.  Note that using the distribution of (*,*,...,*,BLOCK) 
with n-1 `*'s for a rank n array essentially signifies a similar 
arrangement.

Barbara Chapman
Piyush Mehrotra