Subject: MIMD and subroutine call from FORALL

From: [email-masked] (Min-You Wu)
Date: Sat, 25 Jul 92 12:04:54 EDT

1. A way to obtain MIMD parallelism and loosely synchronous constructs
is to call ELEMENTAL functions (subroutines) from block Forall:

      FORALL (I = 1:N)
        CALL SUBROUTINE1 
        CALL SUBROUTINE2 
        CALL SUBROUTINE3 
      END FORALL

This way, execution of subroutines is asynchronous and after each
call there is a synchronization.  As proposed by John Merlin, MIMD
parallelism can be obtained by means of branches within an elemental
function (subroutine).  However, 

2. calling functions or subroutines from FORALL itself is not well
justified yet.  As an example, there is a problem with subroutine 
(function) calls.  It can be illustrated by the following example:

      FOO (a, b, c, d)
        REAL a, b, c, d
        a = ...
        d = c + d
        RETURN
      END

This is a perfect elemental subroutine, and if it is called by

      CALL FOO (A(I),B(I),C(I),D(I))

there is with no problem.  However, if it is called by

      CALL FOO (A(I),A(I-1),A(I+1),D(I))

the result becomes nondeterminate.  I have asked some people
at last HPFF meeting the question and didn't get a satisfactory
solution.  Using IN, OUT, or INOUT cannot solve the problem.
Maybe we have to enforce more restrictions.  What restrictions
are proper?

Min-You

---

From: Piyush Mehrotra <[email-masked]>
Date: Sat, 25 Jul 92 18:25:59 -0400

Date: Sat, 25 Jul 92 12:04:54 EDT
	From: [email-masked] (Min-You Wu)
	Status: R

	      FORALL (I = 1:N)
	        CALL SUBROUTINE1 
	        CALL SUBROUTINE2 
	        CALL SUBROUTINE3 
	      END FORALL

I am not sure how this allows MIMD parallelism. Could you please explain
again?

	      FOO (a, b, c, d)
	        REAL a, b, c, d
	        a = ...
	        d = c + d
	        RETURN
	      END

	

	
	Min-You

Calling FOO as defined above is not HPF-conforming since the execution
of one instance of the forall construct is interfering with other
instances.

	- Piyush

---

From: John Merlin <[email-masked]>
Date: Mon, 27 Jul 92 15:29:09 BST

> Maybe we have to enforce more restrictions.  What restrictions

I think the use of IN, OUT and INOUT *do* solve the problem.

If FOO is elemental and the above call appears in a FORALL, e.g.:

	FORALL (I=1:N)  CALL FOO (A(I), A(I-1), A(I+1), D(I))

this is equivalent to the following (array-valued) subroutine call:

	CALL FOO (A(1:N), A(0:N-1), A(2:N+1), D(1:N))

Fortran 90 has a rule that if any part of an actual argument is defined 
through the dummy argument, the actual argument may be referenced only 
through that dummy argument.  Since the first two arguments of FOO are 
both defined  (assuming for the sake of argument that the last assignment 
in FOO was meant to be 'b = c + d'),  the fact that they share the same 
section A(1:N) conflicts with this rule, as does the fact that the third 
argument overlaps with the first two.

The specification of argument intent allows this rule to be checked 
(at least partially).  If a dummy argument has INTENT (OUT) or (INOUT), 
the corresponding actual argument shouldn't overlap with any other 
actual argument (whatever its intent).   If there is, or may be, 
an overlap, I would guess the compiler is justified in generating 
at least a warning message, and perhaps even an error message if the 
overlap can be proved statically (as in your example).
The first two arguments of FOO have INTENT (OUT) and the last two have 
INTENT (IN), so the compiler would generate a warning or error for the 
above call.

In short, the relevant restrictions follow from F90's present
restrictions on actual arguments, generalised to FORALL  (i.e. using
the correspondence between FORALL and an array-valued assignment or
subroutine call).

Piyush writes (<[email-masked] Sat Jul 25 23:37:45 1992>):
> ...

In the above example, MIMD paralleism would normally require the
elemental procedures to have arguments!:-)  Then, the functional
parallelism comes from argument-dependent branches in the procedure, e.g:

	!HPF$ ELEMENTAL f
	FUNCTION f (x,i)
	  IF (x > 0) THEN      ! content-based conditional
	    ...
	  ELSE IF (i == 1 .OR. i == n) THEN   ! index-based conditional
	    ...
	  ENDIF
	END FUNCTION f

	...
	FORALL (i=1:n) x(i) = f (x(i), i)
	...

(Another way of obtaining MIMD parallelism is via different elemental
function calls in the WHERE and ELSEWHERE parts of a WHERE construct 
-- assuming the compiler could avoid a synchronisation barrier between 
the two parts).

                    John Merlin