Subject: User-defined elemental (> Cownie) - resident; local; global

From: "Rex Page" <[email-masked]>
Date: Tue, 30 Jun 92 07:52:19 -0500

> Terminology
> ===========
> "local" is very confusing in this discussion, as it refers in closely
> related contexts both to scope ("local array") and to the mapping of
> the data onto processors ("the array can be locally accessed").
> I think we should keep "local" for the scoping property (since they
> got there first), so does anyone have a good suggestion for the 
> property of being in the processor's memory ?
> (poor suggestion : "resident").

We do need a new term, and I think "resident" is a good choice.

> Content
> =======
> The local arrays for elemental functions should certainly be allocated
> resident, however the problem also arises with the global data (see
> example in previous mail).

> A solution which would meet John's objective (but is not permissible
> under his restrictions) would be to insist that the global data
> accessed by the elemental function was replicated, thus ensuring that
> it is resident on all processors.

If the demand-driven-data-access-problem is bad,
the replicated-data-coherence-problem is worse.
I hope we don't try to tackle that one this go-round.

How about requiring that global data accessed by elemental functions
be constant data (identifiers defined in PARAMETER statements,
made accessible through MODULE USE)?  That would circumvent the
coherence problem.

   - Rex Page

---

From: [email-masked] (James Cownie)
Date: Tue, 30 Jun 92 15:40:18 BST

> I hope we don't try to tackle that one this go-round.

Ahh, but we already are trying to tackle that one. The distribution
proposal shows precisely how to express this e.g. quoting the example
on page 7 (of June 9 version)
"
CHPF$ ALIGN A(:) WITH D(:,*)	
means that a copy of A is aligned with every column of D.
"

Since we're already accepting that we have to solve the "replicated
data coherence problem" allowing READ ONLY access to such replicated
global data inside ELEMENTAL functions in FORALL loops seems fine. 

In particular the ELEMENTAL assertion assures us that we don't have to
worry about any updates to this replicated global data, since
elemental functions are forbidden to update global data. (John's rule
3). Therefore this is actually a simpler case than the general one,
where replicated data can be updated. 

It's actually the interaction of loosely synchronised MIMD parallelism
with the replicated data which leads to the problems, in the SIMD (or
strongly synchronised MIMD) model which is supported by HPF at the
moment, the replicated data is not a problem, since read/write or
write/write races cannot occur because all the processors are
conceptually executing the same program with the same data in lock
step. Naively obvious implementations either do exactly this and
replicate the execution of the code and all the scalars on all nodes,
or centralise it (onto the "host node") and then broadcast the updates
before the next "parallel" operation. Serious implementations will
doubtless do a lot of data-flow analysis to determine exactly which
values are required where to reduce the amount of communication they
have to perform.

These store races only become possible when there are many threads of
control, each of which can be generating a different memory access
pattern, as inside the functions called from a FORALL, (and possibly
inside an HPF$INDEPENDENT do loop, or an HPF$BEGINDEPENDENT,
HPF$ENDEPENDENT block).

It's not clear to me (yet) whether there are any restrictions on what
one is allowed to put in an INDEPENDENT loop, and whether similar
issues  re-appear in that context.

--Jim
James Cownie 
Meiko Limited
650 Aztec West
Bristol BS12 4SD
England

Phone : +44 454 616171
FAX   : +44 454 618188
E-Mail: [email-masked] or [email-masked]

---

From: [email-masked] (James Cownie)
Date: Tue, 30 Jun 92 16:30:16 BST

--Jim
James Cownie 
Meiko Limited
650 Aztec West
Bristol BS12 4SD
England

Phone : +44 454 616171
FAX   : +44 454 618188