Subject: I/O chapter

From: Bob Knighten <[email-masked]>
Date: Mon, 19 Oct 92 00:41:31 PDT

%io.tex

%Version of October 16, 1992 - Robert L. Knighten, Intel Corporation

\chapter{Input/Output}
\label{io}

\footnote{Version of October 16, 1992 - Robert L. Knighten, Intel
  Corporation}

High Performance Fortran has exactly the same Input/Output facilities
that are available in Fortran 90.

One of the High Performance Fortran Forum working groups was dedicated
to I/O extensions for parallel machines. The objective here was to
define a set of extensions to standard Fortran 90 I/O which would
provide high I/O performance a massively parallel computer running
HPF.

Three proposals in this spirit were offered by Marc Snir of IBM and
Piyush Melrota of ICASE.  The basic idea of these proposal is outlined
below and the proposals themselves are in the Journal of Development
appendix.  

From the beginning there was also a strong feeling that no extensions
to Fortran I/O should be added to High Performance Fortran, and this
is the position ultimately taken by the High Performance Fortran
Forum.

Among the arguments for this position are:

\begin{itemize}
\item I/O systems on parallel computers from different vendors are too
  architecturally different for there to be a useful abstraction on
  which the language model can build.  For example, consider the
  difference between a machine with disks on each node, compared with
  one which has a high bandwidth disk system connected to the
  communication network, and thus globally accessible.

\item Fortran I/O is already highly expressive, some would say
  too expressive.

\item The HPF compiler must already know when it is performing I/O on
  distributed arrays, and can optimize the I/O to distributed files
  without any extensions to the source language.

\item The management of distributed files (and their implementation)
  is a matter for the operating system, not the language.

\end{itemize}

Moreover the current lack of extensions does {\bf not} limit features
that may be added by system vendors.  In particular:

\begin{itemize}
\item Vendors are allowed to implement any I/O extensions to the
  language they may wish (indeed this would be impossible to prevent
  it!)  There are simply no special I/O mechanisms mandated by HPF.

\item The HPF run-time system may use whatever facilities the
  operating system provides for accessing "high performance" files,
  though the HPF language contains no I/O extensions that specifically
  describe such access.  For example the HPF system is entirely free
  to place a status='SCRATCH' file in the highest performance file
  system it likes, and distribute it as is appropriate for the
  machine, but it is not up to the user to say all this.

\end{itemize}

The proposals made to the IO subgroup were based on the following
observations:  
\begin{itemize}
\item A massively parallel machine needs massively parallel I/O

\item Efficient programs must avoid sequential bottlenecks from
  processors to file systems
  
\item Fortran specifies that a file appears in element storage order; this
  conflicts with striped files (for example, an array distributed by
  rows may be written to a file striped by columns).

\end{itemize}

The proposals were that HPF should provide explicit control to obtain
high performance I/O.  In essence the three proposals were:
\begin{enumerate}

\item On a write, give a hint about how the data will be read.

\CODE
          DISTRIBUTE (CYCLIC) :: a
          !HPF$ IO_DISTRIBUTE * :: a
          WRITE a, b, c
          !HPF$ IO_DISTRIBUTE * :: b
\EDOC

When an array is written, it can be easily read back in the given
distribution. The annotation can be associated with either the
declaration or the write itself; in the first case it applies to all
writes of the array, while in the second it only applies to the one
statement.  The intent is that metatdata is kept in the file system to
record the "right" data layout. The advantages of this proposal
include notation and efficiency

\item Give hints about the physical layout (number of spins, record
  length, striping function, etc.) of the file when it is opened.

  This uses the HPF array mapping mechanisms. (A file is a
  1-dimensional array of records.) The syntax needs a "name" for the
  file "template"; the proposal is to use FILEMAP. The programmer can
  align/distribute FILEMAP (on I/O nodes), associate FILEMAP with a
  file on OPEN, etc. There are no changes in semantics or file system.

\item Introduce parallel read/write operations that are not
  necessarily compatible with sequential ones.

\CODE
  PWRITE a
  PREAD a
\EDOC

Data can be read back only into arrays of the same shape and mapping.
Data written by PWRITE must be read by PREAD. This solution does not
need metadata in file system or changes in the file system but is
incompatible with the standard READ and WRITE

\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\end{document}

%Revision history:

%July 7, 1992 - Original version by Marc Snir, International
%  Business Machines 
%October 16, 1992 - Revised by Robert L. Knighten,Intel Corporation,
%  to explain the HPFF position.  Used comments from James Cownie, Meiko
%  Limited.  Proposals from Marc Snir and Piyush Mehrota moved to Journal of
%  Development.