Subject: Re: time to consider options for requirements

From: [email-masked] (Chuck Koelbel)
Date: Wed, 11 May 1994 16:27:01 -0600

At  8:55 5/10/94 -0400, Alok Choudhary wrote:
>We have two basic goals.
>
>1. Writing a requirements report.

See below.

>2. Coming up qith simple additions/changes in current HPF. E.G.,
>
>    a) 64 bit pointers

As has already been pointed out, KIND type parameters allow this for
Fortran 90 already.

>    b) Out-of-core directives added to arrays to indicate large arrays
>       and guiding the compiler to handle such arrays. Also, it is
>       possible to specify amount of memory with this directive but
>       that may be very machine dependent.

This does not sound like a simple addition.  For starters, what _exactly_
does an out-of-core directive mean?  Do any vendors already have one?

>    c) Asynchronous I/O (Posix)

Saying "Use Posix" would be a simple addition.  Extending Posix in some way
would not.  I'm not familiar enough with Posix to say whether adopting it
would be a good idea.  Could somebody provide a pointer to the standard, or
summarize it for the group?

In short, I don't believe there are any quick fixes for HPF I/O, except
possibly for adopting Posix asynchronous I/O.

> We need to address 2 by next meeting for some sort of straw vote and further
> action.
>
> Things to consider wrt 1 above include distributions of files, mapping
> on disks etc.
>
> Please provide your input ASAP.
>
>thanks
>Alok

I believe the following are reasonable goals for the requirements document:

0. A definition of the "parallel I/O problem"
        (or problems, if user needs are sufficiently diverse)

1. A survey of user needs

2. A survey of models for parallel I/O

3. (Assuming we agree) Recommendations:
        I/O capabilities for HPF to standardize
        One HPF-level model for describing I/O
        A high-level strategy for HPF I/O features
        (e.g. "define a new parallel file type")

Slipping in some reasons that standard Fortran I/O statements are not
sufficient would also be a good idea.

                                                Chuck

---

From: [email-masked] (James Cownie)
Date: Thu, 12 May 1994 11:53:59 +0000

> possibly for adopting Posix asynchronous I/O.

There are some potentially serious problems with binding asynchronous
operations into F90. (I have observed these dangers in MPI, but the
same thing applies here too.)

The fundamental issue is that there is no way in F90 (or in any other
language to the best of my knowledge) of expressing the fact that the
arguments to a subroutine will continue to be accessed AFTER the
subroutine has returned.

This is exactly the behaviour of all of these asynchronous operations
(e.g. Async I/O, non-blocking message passing), where one routine
"passes a pointer to the data" and then returns. The data is actually
accessed behind the programmer's back, and finally a call is made to
check that the subversive mechanism has finished with the data.

This is not such a problem in C, where the argument passing mechanisms
are low level, and defined in terms of machine level semantics and
there are no compiler generated temporaries. It is a significant issue
in F90, where the argument passing mechanisms are not defined at this
level at all.

Consider for instance, something like this

	integer ibuffer(10000)

	call start_async_op(ibuffer[1:1000:3])

	...

If start_async_op is an external routine which appears to the F90
compiler to be written in F77 (though it's probably written in C !),
the compiler will very reasonably generate a temporary to hold the
array slice, and compress the data into it. BUT the compiler is
entirely at liberty to free this temporary immediately start_async_op
returns. Unfortunately that's all the data which is being used by the
low level asynchronous mechanism at exactly the same time as it is
being freed and reused by the compiler.

I don't have a solution to this. I am flagging a warning though. Don't
think that just saying "Adopt POSIX asynchronous I/O" is necessarily
"a simple addition".

-- Jim 
James Cownie 
Meiko Limited			Meiko Inc.
650 Aztec West			130C Baker Avenue Ext.
Bristol BS12 4SD		Concord
England				MA 01742

Phone : +44 454 616171		+1 508 371 0088
FAX   : +44 454 618188		+1 508 371 7516
E-Mail: [email-masked]   or    [email-masked]
WWW   : http://www.meiko.com:8080/welcome.html