Subject: another survey answers

From: [email-masked] (Mary E Zosel)
Date: Thu, 4 Jun 92 18:04:43 -0700

FYI

This Query  about common blocks was sent to the users of the BBN TC2000 at LLNL.

-------
Date: Fri, 29 May 92 12:24:44 -0700
From: [email-masked] (Mary E Zosel)
To: [email-masked]
Subject: hpf query

to users interested in HPF - request for feedback:
Mary Zosel would like opinions about the following question ... send any
comments to [email-masked]
    -mz-

........
The High Performance Fortran Forum (HPFF) is defining a set of directives for
use to describe how a user would like important arrays distributed across a
machine with a physically distributed memory for massively parallel machines.  
There are, for example, directives for specification of BLOCK vs CYCLIC
distributions and directives to specify how arrays should be aligned with
respect to each other.

An obvious question arise, when considering a port of existing Fortran codes -
what do you do about F77 storage association problems?  These are places where
the code uses COMMON with different sizes or shapes of data in different
subroutines - and / or uses EQUIVALENCE with arrays in COMMON - especially
EQUIVALENCE which overlaps array boundaries.

The HPFF committee looking at this issue has currently taken a very
conservative approach: 

--------
Individual arrays in  common blocks can be distributed ONLY if there are no
EQUIVALENCE statements associated with any variable in the common block and
every instance of the common block contains the same number of variables of the
same sizes.
--------

The argument FOR this conservative restriction is that the rule is simple to
understand and to implement.  In moving to parallel machines, there will be
some level of re-write happening anyway - the programmers can separate out the
important arrays into "clean" common blocks.   There is another proposal which
would allow the user to distribute an entire storage-associated COMMON block
(not array by array).  In cases where individual arrays can't be distributed,
the block can be distributed instead.

The argument AGAINST this conservative restriction says --- there are useful
cases where equivalence can happen.  Equivalence to one part of a common block
should not kill distribution of an array in another part.  Also,  compilers are
capable of sorting out what is the largest underlying equivalence array (maybe
with some interprocedure work?).  We should even allow distribution of arrays
with equivalence under certain circumstances (which might be a bit complex to
define exactly.

WHAT DO YOU THINK?  Simplicity and clean up the code   or go for the most
liberal definition possible, even if it is complex???  Mary needs feedback by
Friday June 5th.

------------------
-----------------

HERE ARE ANSWERS

Date: Fri, 29 May 92 12:35:56 PDT
From: [email-masked] (Kevin Robert Lind)
Subject: Re:  hpf query

It seems to me that the most conservative approach is best, at least for now.
The standard can be liberalized later if it should really prove to be necessary,
but our level of experience and the need for correctness of programs and 
compilers I feel dictate a conservative approach until all of these issues get
sorted out.  Since most codes will need major rewrites anyway, it would be best
to develop something definitely supportable to start with; a liberalized standard
code would execute under the more conservative standard, but if it should turn out
that the more conservative approach is required, you then have all sorts of half-done
implementations, and codes that don't match the standard as implemented.

SUMMARY: Let's do what we know we can do, and work toward the ideal model after a
bit more experience.

Regards, Kevin Lind

-------------

Date: Fri, 29 May 92 12:39:38 PDT
From: [email-masked] (Jeff Painter)
Subject: Re: hpf query

Remember that there are legitimate uses of EQUIVALENCE other than
changing the shape of an array, merging arrays, etc.: e.g. do both
fixed- and floating- point operations on the same array by
equivalencing an integer name to a real name.

The proposed rule you reported would ban even this.
Jeffrey F. Painter			[email-masked]
L-316, Lawrence Livermore Laboratory	[email-masked]
Livermore CA  94551   USA               510-422-0675, FTS 532-0675

--------------

Date: Fri, 29 May 1992 12:51 PDT
From: Ron Schmucker <[email-masked]>
Subject: Re: hpf query

I VOTE FOR SIMPLICITY AND CLEAN UP THE CODE!
							Ron Schmucker

-------------

Subject: HPFF common
Date: Fri, 29 May 92 13:35:14 -0700
From: [email-masked]

Mary,
I am responding to the "question".  I side with the conservative
method.  I think any other solution would "require" diagnostics from
the compiler to explain how it resolve the complexity and what
assumption it might have made in treating equivalence or dissimilar
size common blocks.  Those diagnostics will require as much effort to
decipher as cleaning up the common blocks.

Clean it up, make it say what you really mean.

bryan

------------

Date: Fri, 29 May 92 13:39:53 PDT
From: [email-masked] (Hanna Szoke)
Subject: Re:  hpf query

KISS - Keep It Simple and S.. don't remember what.

The anticipated problems with a straightforward implementation 
sound like symptoms of ancient Fortran that is due for a rewrite
in any case. 

Hanna Szoke.
.

-------------

Date: Fri, 29 May 92 22:08:02 PDT
From: [email-masked] (Bill Crutchfield)
Subject: Re: hpf query

I think obscuring the parallel programming model just to preserve bad
programming tricks of the past is a bad idea.  Reliable parallel programming
is difficult enough without introducing a rococco data distribution model where
arrays can be a variety of different names and shapes, while still representing
the same locations in memory.

-----------

Date: Sun, 31 May 92 14:27:54 PDT
From: [email-masked] (John H. Bolstad)
To: [email-masked]
Subject: HPFF common

Dear Mary,
   Regarding your "High Performance Fortran - Common Blocks"
memo, I favor the "conservative" position.  "Equivalence" is
just a bad habit left over from Fortran II days, and since codes
have to be rewritten anyway to go to massively parallel, this is
a good time to get rid of this.  Anyway, it just makes it more
difficult to get correct code out of compilers.

...  part of message deleted complaining about a compiler bug

   Finally, we have been rewriting several large codes for
the BBN, and we have eliminated most common arrays, except
possibly arrays whose length is independent of problem size.
There are two reasons:  One doesn't want to have to recompile
for a change in problem size, and more importantly, if one
has a fixed size problem, one doesn't want to have to recompile
when the number of processors changes.  So that means no
problem-size dependent arrays in common anyway.  For Fortran,
the only (almost) portable memory management scheme seems to
involve C subroutines that call malloc, perhaps covered up by
calls to a macro processor like M4.  So arrays in common
(even without equivalence) are not so useful.

   John Bolstad
   [email-masked]

Date: Mon, 1 Jun 92 11:51:25 CDT
From: [email-masked] (Anthony L. Kimball)

Yea! Good choice!

//alk

Date: Mon, 1 Jun 1992 11:22:58 -0600
From: Oliver McBryan <[email-masked]>

> The argument AGAINST this conservative restriction says --- there are useful
> cases where equivalence can happen.  Equivalence to one part of a common bloc>
 should not kill distribution of an array in another part.  Also,  compilers ar
> capable of sorting out what is the largest underlying equivalence array (mayb>
 with some interprocedure work?).  We should even allow distribution of arrays
> with equivalence under certain circumstances (which might be a bit complex to
> define exactly.

The problem that I see is that if HPF is to be of real use it should work
for heterogeneous systems (e.g. a pseudo-machine consisting of 2 or more
distinct computers tied together - e.g. a CRAY and a CM-5).  However distinct
systems may support different representations for integer or real variables
and so distribution of an array to possibly a different archoitecture is
possible only if one knows exactly which data types are stored in the array.

Date: Mon, 1 Jun 92 15:08:47 GMT-0700
From: [email-masked] (William J. Camp)

Art and Mary:
Personally, while I used these constructs  
for years, and while I know that their  
removal from the large existing software  
base would NOT be trivial (in some cases the  
effort incurred might not be justifiable), I  
still think that they should be eliminated  
from any parallel version of fortran. They  
are not needed and do much to reduce  
flexibility, portability and reusability in  
fortran programs. On the other hand, our  
applications programmers seem to be moving  
to C++ for these issues and for developing  
parallel code. (So, maybe my comments are  
irrelevant.) The data parallel class of  
languages don't look to us like they will  
support true MIMD programming effectively  
for  some time to come.

Bill Camp, MPCRL at Sandia 

Date: Mon, 1 Jun 92 15:45:30 -0700
From: [email-masked] (Duane F. Carbon)

Dave DiNucci, a former LLNL employee (who apparently still gets email from
your site), now works for us at NASA Ames Research Center's Numerical 
Aerodynamic Simulation (NAS) Facility.  We are NASA's main computational 
center supporting advanced research in Computational Fluid Dynamics (CFD).  
We have taken the liberty to pass around to our CFD researchers a copy of 
your survey on HPF and COMMON/EQUIVALENCE statements which Dave gave us.  
I hope that you don't mind too much.  I have appended below the first
response from one of our scientists.  Others may be forthcoming.

Good luck,

Duane Carbon

Leader, Parallel Tools Group
NAS Systems Development Branch

============================================================================

>Date: Mon, 1 Jun 92 14:20:57 -0700
>From: tedwards (Dr. Thomas A. Edwards)
>
>Duane -
>
>If you think this is the kind of feedback Mary wants, please forward it to her:
>
>Large-scale 3D CFD code development has moved away from EQUIVALENCE statements
>and common blocks that are different throughout the code.  There are a few
>reasons for this.  First, using these constructs can lead to undetected bugs.
>Once a researcher has been bitten by one (at the cost of a week or more of
>debugging), he tends to avoid using them again.  Second, we are striving toward
>machine-independent code, and the interpretation of these statements seems to
>vary using different architectures, operating systems and compilers.  Finally,
>the ease of coding using "include" statements has led most of our researchers
>to set up master common block files that are simply copied into the code at
>compile time.  This results in identical common blocks in all routines where
>they are referenced.
>
>That's my two cents.
>
>Tom
>
>P.S.  I presume you forwarded your mail to Lasinski's group as well?  I'm 
>sending it on to two other people in my branch:  Kalpana Chawla and Stuart
>Rogers, oh and Scott Thomas and Steve Ryan as well.
>

Date: Tue, 2 Jun 92 08:24:00 PDT
From: [email-masked] (Mark Christon)

Mary,

It sounds like a lot of the issues that you are discussing regarding
the distribution of arrays and common blocks came out of ParaDyn.
My thinking on this issue is that ultimately, the more complex
case of distributing common blocks would be nice, but it will be 
a while before anyone can really understand this well enough to
exploit it in a "multi-platform" sense.  

While distributing common blocks array-by-array is seems to me to
be a workable situation for now, the issue of partitioning arrays
in subroutine calls is still not being addressed.  This seems to be
a far bigger issue in porting dusty decks which make use of the 
antiquated memory management strategy of extending blank common 
(ala CRAY memory management) for the short run.  In the same 
vein, a common memory management interface would be nice, especially
in light of what TMC has done with their CM_allocate/deallocate constructs
which forces the user to perform additional work to keep track of 
array extents.

Thanks, Mark

Date: Wed, 3 Jun 92 17:53:48 PDT
From: [email-masked] (   Carol Hoover            )

This is a response to your feedback request for the
HPF meeting.  Tony De Groot, Robert Whirley, and I 
discussed the problems we are faced with installing
DYNA on the CM-5.  After some deliberation, we decided
we are strongly in favor of what you call the "liberal"
approach. 

DYNA and the mpci report I sent you is an example 
of the tremendous amount of coding changes necessary to get
Fortran standard data structures SIMPLY MOVED to a 
distributed memory machine.  There is plenty of additional
work necessary to implement efficient parallel syntax.
This work to rewrite data structures which conform to 
Fortran standards to run on parallel machines is not
productive effort for the scientist.   The reasonable
and productive work for a scientist is numerical algorithm
research and modification for parallel systems.

The technical argument we use to suggest retaining the 
Fortran 77/90 storage flexability and syntax 
in its present form for distributed memory machines is the
following:  Regardless of how you define the shapes and
sizes of arrays in the usage of a COMMON block throughout
the subprocedures as long as you know the
first word address, the address calculation for any
usage of arrays in a subprogram can be mapped into  
offsets from the fwa and this is done in the first
stage of compilation.  More specifically, the compiler
address generation is equivalent to a one-dimensional
array independen of what syntax the user has provided
to point to a particular address or sub-block.  The 
mapping of this linear array to processors and their
local memories can be prescribed in whatever way the 
seems most appropriate for the architecture.  But 
keep in mind, the user has specified a linear
address mapping that is always unambiguous, both in his
syntactical use as well as what he expects the compiler
to do. The generation of array descriptors and the scheme
for allocating the arrays to the physical processors is
completely arbitrary and can be defined by the software
and hardware specialists for any particular machine.  
 
We understand that implementation is not a trivial
task for compiler developers.  On the other and,
the mission of computer scientists is to make computers
useable for researchers (or end-users in other fields).
Software developers have made a very convincing argument 
for several years that hardware-related issues requiring
software changes be made ONCE in the system software
rather than thousands of times in user-level software. 
The idea that can be communicated to computer manufacturers
along this line is that the more useable they can 
make their product, the more researchers they will find
wanting to hurry up and use their product.

It would be extremely valuable for us if you, as a 
language specialist, could give the above ideas some
thought and  articulate it to the compiler
specialists.  I would be happy to spend some time
discussing in more detail why I think the solid mechanics
programs are not unique but rather representative of 
production codes.   Let us know the results and we
can discuss the issues more at the next ParaDyn meeting.

Date: Thu, 4 Jun 92 16:54:06 -0700
From: [email-masked] (Dale Nielsen)

Mary,
I vote for the conservative restriction.
Dale