Subject: Fortran Preprocessor

From: [email-masked] (Jeanne Martin)
Date: Tue, 15 Dec 92 15:08:28 PST

Here is the proposal that Len Moss mentioned (minus italic and bold font, but
still readable).
____________________________________________________________________________
                                     ISO/IEC JTC1/SC22/WG5 - N539 

                A Standard Fortran Preprocessor

                         Jeanne Martin

                         July 13, 1990

                            CONTENTS

  Is a Fortran preprocessor needed?                    1
  A Preliminary Proposal                               2
    Preprocessor Source                                2
    Preprocessor Variables                             3
    Conditional Inclusion of Code                      4
    Macro Definition                                   4
    Macro Expansion                                    5
    Macro Files                                        6
    Preprocessor Intrinsic Functions                   6
    Relocating Text                                    8
  Summary of Proposed Fortran Preprocessor Features   10
  Summary of ANSI C Preprocessor Features             11

Is a Fortran preprocessor needed?

What can a preprocessor do for the Fortran programmer?

(1)  Frequently a programmer needs to maintain more
than one version of a code.  There may be a need to run
the code in different environments; there may be
special requirements for some users of the code; or it
may be necessary to test an expermimental version
without disrupting the original.  It is frequently
difficult enough to maintain a large code, much less
several versions of the code.  For the programmer, the
best solution is a single source file that can be
preprocessed to extract the exact version needed for
each special purpose.  This strategy has many
advantages for the programmer.  There is only one
source to maintain.  If an error is discovered in one
version that applies to all versions, it can be
rectified only once.  There is no chance that it will
be overlooked and crop up again.

(2)  Frequently in a program, there will be a sequence
of statements that appear in many places, perhaps with
slight modifications.  Duplicating the statements and
making minute changes is error prone.  If an error is
discovered in the sequence, every place where the
sequence is replicated must be located and rectified.
It would be much better to specify the sequence once,
designating the places where variations occur.  Then
the sequence can be inserted, properly adjusted,
wherever it is needed.  This has the advantage of
shortening the source representation of the code and
making trivial the correction of an error or the
installation of an enhancement in the sequence.

What preprocessor features make these advantages
possible?

There are two basic features: conditional code
inclusion and macro definition and expansion.  In
addition there are two minor features: a mechanism to
specify preprocessor variables to phrase the conditions
that determine code inclusion and a mechanism to
include macro definitions kept in external files.

The new Fortran 90 standard does not contain
specifications for conditional inclusion of source
lines or for macros with arguments.  It does specify an
include facility.  At LLNL, where Fortran preprocessors
have been provided as part of the Fortran environment,
it has been estimated that  84% of the lines of an
important class of applications require some form of
preprocessor support for macro expansion, conditional
code, or inclusion of code from another source file.
Since there is no standard for most of the needed
facilities, it will be difficult to move these
applications to platforms supported by vendor-supplied
software.

The C standard specifies conditional inclusion of code
and macros with arguments.  Generally preprocessing is
built into C compilers.  There are stand alone C
preprocessors but these are heavily tied to C syntax
and source form.  Although C programmers have come to
rely on preprocessor capabilities, there does not
appear to be any good Fortran-based equivalent in the
Unix community.  What appears to be needed is a
preprocessor uniquely specified for Fortran, but easy
to use and understand.  It could provide facilities
that C programmers have grown to expect in UNIX
environments.

Thus there are at least two communities that feel
preprocessing is required for programming language
support: LLNL Fortran programmers and C programmers.
Should WG5 attempt to develop a standard for Fortran
preprocessing support?

A new product that conformed to such a standard could
be built on reusable modules.  Some of the same
functionality (analyzing Fortran source, evaluating
Fortran expressions, creating Fortran output, etc.) is
needed for other tools that could further enrich the
Fortran environment.

A Preliminary Proposal

The following proposal is suggested as a starting point
for discussion leading to a more formal design.  All
keywords, syntax, semantics, and rules proposed are
subject to change for better ways to meet user
requirements.  Unresolved issues are flagged by ???.

The meta language used to describe the statements and
constructs in this section makes use of square brackets
([  ]) to indicate optional elements and ellipses (...)
to indicate multiple occurrences of preceding optional
elements.

   Preprocessor Source
   
   Preprocessor source should be ordinary Fortran
   source with interspersed preprocessor statements.
   A preprocessor statement would begin with a special
   character.  C uses a number sign (#), but this is
   one of the characters which the SC22 Ad Hoc Group
   on Character Handling in Programming Languages
   recommends be avoided (along with $, [, ], and
   others).  In the existing Fortran character set,
   this leaves only the question mark (?) as an
   available, as-yet-unused character.  So even though
   it is not ideal, it will be used for illustrative
   purposes in this preliminary proposal.  It is
   proposed that this chosen character may appear in
   any character position and must be followed
   immediately by the text of the preprocessor
   command; that is, no space is allowed between the ?
   and the command.  Note that Fortran 90 include
   lines are not preprocessor statements.
   
   For scoping purposes, the preprocessor needs to
   recognize subprogram boundaries; that is, it must
   parse and record the following Fortran statements
   and all of their variations:
   
                        PROGRAM
                        END PROGRAM
                        SUBROUTINE
                        END SUBROUTINE
                        FUNCTION
                        END FUNCTION
                        MODULE
                        END MODULE
                        BLOCK DATA
                        END BLOCK DATA
                        END
   
   Design Decision:  ???Should a Fortran preprocessor
   assume source will always be in free form and that
   the preprocessor output is to be in free form???

   Preprocessor Variables
   
   Preprocessor variables are needed to specify the
   conditions in conditional inclusion constructs.  In
   C, a preprocessor variable is treated like a simple
   macro and uses the same syntax as is used for
   within-line macros with no arguments.  For Fortran,
   the following is proposed:
   
                      ?define  name  string
   
   This results in every occurrence of the token name
   outside of comments, (character contexts and format
   statements???) being replaced by string (without
   evaluation).  The definitions themselves are not
   transferred to the output text.  The string
   associated with a given name may change at any time
   by redefinition.  The most recently specified
   string is the one used for replacement.
   
   Inline replacements with arguments are permitted
   when the general form is implemented:
   
   ?define  [function] name  [( arg [,arg]...)] string-with-args-inserted
   
   The keyword function must be present if there are
   arguments.  The argument names (arg) must be
   recognizable tokens in the string.
   
   For example:
   
          Preprocessor Source
   
   ?DEFINE NEW_VERSION .TRUE.
   ?DEFINE FUNCTION CONVERT(TO_CELSIUS) (5*(TO_CELSIUS- 32))/9
               ...
   IF (NEW_VERSION) CALL EXTEND
   TEMP = CONVERT(72)
   
          Preprocessor Output
   
   IF (.TRUE.) CALL EXTEND
   TEMP = (5*(72-32))/9
   
   With these preprocessor variables defined, the
   source is easier to read, and an optimizing
   compiler produces efficient code.
   
   The scope of preprocessor variables is the
   subprogram in which they are defined (main program,
   subroutine, function, module, or block data
   subprogram).  A preprocessor variable defined
   outside any of these scoping units has global
   scope, and its definition is available inside any
   of the scoping units in the precompilation unit
   being processed.  This is analogous to Fortran 90
   entities that are specified to be global to a
   module by being defined outside of any module
   procedures in the module.

   Conditional Inclusion of Code
   
   The proposed conditional inclusion facility ( the
   ?if construct)  is modeled on the Fortran IF
   construct.
   
                 ?if (condition1) then
                    block1
                 [?elseif (condition2) then
                    block2 ]
                     ...
                 [?else
                    blockn ]
                 ?endif
   
   No more than one block will appear in the
   preprocessed text.  The preprocessor variables used
   to determine whether code inclusion takes place are
   specified by ?define statements, which causes
   variable names in the conditions to be replaced by
   strings.  The preprocessor must convert these
   condition strings to appropriate operands and
   operators and evaluate them to get a true or false
   result.  The preprocessor standard must specify
   which data types are permitted for operands and
   which operators are allowed.  These might be
   different from what is currently allowed for
   Fortran 90 scalar values (???reals excluded?, long
   character strings allowed?, the concatenation
   operator allowed?, some of the Fortran 90 intrinsic
   functions allowed???, etc.).  The types of operands
   and the operators allowed in ?if constructs must be
   specified in the final design.
   
   For example:
   
        Preprocessor Source
   
   ?DEFINE NEW_VERSION .TRUE.
      ...
   ?IF (NEW_VERSION) THEN
    CALL SUPER_CALC (X, Y, Z)
   ?ELSE
    CALL CALC (X, Y, Z)
   ?ENDIF
   
       Preprocessor Output
   
   CALL SUPER_CALC (X, Y, Z)

   Conditional inclusion allows two versions of a
   program to be maintained within one copy of the
   source for the program.

   Macro Definition
   
   Macros are similar in purpose to preprocessor
   variables.  They are needed when the replacement
   text (the macro-body) involves several statements.
   Macro definitions have the form:
   
         ?define macro [function] name {(arg {,arg }... )}
           macro-body
         ?end[ ]macro [name]
   
   Arguments are treated much as subprogram arguments
   are treated in Fortran.  They are indicated by name
   on the ?define line.  The places where they are
   replaced in the macro body are signaled by the
   appearance of the argument name.  As with
   preprocessor variable definitions, macro
   definitions do not appear in the preprocessor
   output.
   
   For example:
   
               ?DEFINE MACRO ADD_ITEM (ITEM)
                 TABLE_SIZE = TABLE_SIZE + 1
                 IF (TABLE_SIZE <= MAX) THEN
                   TABLE(TABLE_SIZE) = ITEM
                 ELSE
                   CALL ERROR ("TABLE OVERFLOW")
                   STOP
                 END IF
               ?END MACRO ADD_ITEM
   
   This allows frequently repeated code to be
   specified once.  If, at some future time, the error
   handling should change, it can be adjusted in just
   this one place.
   
   The scope of macros is the subprogram in which they
   are defined (main program, subroutine, function,
   module, or block data subprogram).  A macro defined
   outside any of these scoping units has global
   scope, and its definition is available inside any
   of the scoping units in the precompilation unit
   being processed.  The macro-body  associated with a
   given name may change at any time.  The most
   recently specified macro-body  is the one used for
   replacement.  (See the ?include directive below for
   the means of incorporating macro definitions from
   external files.)
   
   A formal macro argument name may appear as the
   argument of the intrinsic preprocessor function
   ?present, which has the same purpose as the present
   intrinsic function in Fortran 90.  It permits
   inquiries about the presence of that argument in a
   particular expansion.  If an argument is not
   present, it may be associated with a default
   string.

   Macro Expansion
   
   Macro expansions take the form
   
                 name [(arg-spec [,arg-spec]...)]
   
   where arg-spec is one of the following
   
                 arg-string
                 arg-name = arg-string
   
   The second form for arg-spec is identical to the
   keyword form for subprogram arguments in Fortran
   90, and, as in Fortran 90 subprogram invocations
   when keywords are used, the arguments may appear in
   any order.
   
   Macro bodies may contain macro expansions to any
   depth.  Recursion is allowed.  [Note: Recursive
   expansion is not permitted in C, but this seems
   unnecessarily restrictive.]
   
   A macro expansion may appear on a line by itself.
   In this case it is analogous to a CALL statement in
   Fortran except that the expansion line is replaced
   by the macro body with arguments properly inserted.
   For example using the macro ADD_ITEM defined above:
   
   Preprocessor Source    Preprocessor Output
   
   ADD_ITEM (X(I,J))    TABLE_SIZE = TABLE_SIZE + 1
                        IF (TABLE_SIZE <= MAX) THEN
                         TABLE(TABLE_SIZE) = X(I,J)
                        ELSE
                         CALL ERROR ("TABLE OVERFLOW")
                         STOP
                        END IF
   
   A macro expansion may appear embedded in
   preprocessor text.  In this case, the define macro
   line must contain the keyword function and the
   macro-body  must contain at least one reference to
   the preprocessor intrinsic function ?return.  This
   kind of macro expansion is analogous to a Fortran
   function invocation.  The argument to the ?return
   intrinsic serves as the replacement text for the
   macro expansion.  For example:
   
          Preprocessor Source           Preprocessor Output
   
   ?DEFINE COUNTRY "UK"
   
   ?DEFINE MACRO SIDE
    ?IF (COUNTRY.EQ."UK") THEN
     ?RETURN ("LEFT")
    ?ELSE
     ?RETURN ("RIGHT")
    ?ENDIF
   ?END MACRO SIDE
   
   RULE = "KEEP TO "//SIDE           RULE = "KEEP TO "//"LEFT"

   Macro Files
   
   It is convenient to collect related macro
   definitions into files.  A mechanism is required to
   make externally defined macros and possibly other
   information known to a particular execution of the
   preprocessor.  The following statement is used
   
                 ?include  filename
   
   It is similar in function to the Fortran 90 include
   line.  Nested ?includes are permitted.

   Preprocessor Intrinsic Functions
   
   There are occasions when special preprocessor
   actions are desired.  It is convenient to have
   preprocessor intrinsic functions that perform
   operations on an argument string and replace the
   function invocation with the result of the
   operations.  Two of these intrinsic functions have
   already been mentioned: ?present that permits
   inquiries about the presence of an argument in a
   macro expansion  and ?return that is used to
   specify the replacement text for a macro invoked
   like a Fortran function.

   A third such function is ?len.  When
   
                 ?len (arg-name)
   
   appears in a macro body where arg-name is an
   argument to the macro, the expansion text will
   contain an integer that is the number of characters
   in the actual arg-string.  For example, if LIST is
   a growing character string of terms and NOCHARS is
   the number of characters put into LIST, then a
   macro can be defined to add terms to the list and
   keep track of the ending character position:
   
       Preprocessor Source                Preprocessor Output
   
   ?DEFINE MACRO ADDTERM (TERM)
    LIST = LIST(1:NOCHARS)//"TERM"
    NOCHARS = NOCHARS + ?LEN(TERM)
   ?END MACRO ADDTERM
      ...
   CHARACTER (20000) LIST              CHARACTER (20000) LIST
   INTEGER :: NOCHARS=0                INTEGER :: NOCHARS=0
      ...                                 ...
   ADDTERM (BEGIN )                    LIST = LIST(1:NOCHARS)//"BEGIN "
                                       NOCHARS = NOCHARS + 6

   A fourth preprocessor intrinsic function is ?eval.
   It evaluates expressions with the same allowed
   operands and operators as are permitted in the ?if
   construct.  For example, the string "?eval(2*10)"
   would be replaced by the string "20".  There are
   two places where evaluation is necessary in the
   preprocessor syntax (?if constructs and substring
   expressions (see below)), and in these places it is
   handled automatically.  There may be other places
   where programmers might want a string replaced by a
   different string that represents the evaluated
   result and in these places they would need ?eval,
   for example, if it is necessary to generate unique
   statement labels.  The programmer could set aside a
   range for numeric labels that can be used
   exclusively by the preprocessor, for example every
   label above 90000.  The programmer could then
   define a global preprocessor variable
     
                 ?DEFINE LABEL 90000
     
   Then a label occurrence could appear in any macro
   definition as follows
     
           LABEL CONTINUE
                 ?DEFINE LABEL ?EVAL(LABEL+1)
   
   The preprocessor variable LABEL now has the value
   90001 and can be used again.

   A fifth preprocessor intrinsic is ?defined.  It
   could be used to test the definition status of a
   preprocessor variable or macro.  It might be used
   in the conditional inclusion construct and would
   return a true or false result.  For example:
   
            Preprocessor Source
   
   ?DEFINE NEW_VERSION .TRUE.
      ...
   ?IF (?DEFINED(NEW_VERSION) THEN
    CALL SUPER_CALC (X, Y, Z)
   ?ELSE
    CALL CALC (X, Y, Z)
   ?ENDIF
   
           Preprocessor Output
   
   CALL SUPER_CALC (X, Y, Z)

   An arg-string can be considered as a character
   string.  Fortran 77 provides a substring syntax for
   character strings of the form
   
                 string (initial:terminal)
   
   such that if string is "abcdefg", then
   
                 string(2:5) is "bcde"
   
   This same functionality could be allowed in a macro
   body and would permit the selection of a portion of
   an argument string.  It could be specified by a
   sixth preprocessor intrinsic function ?substring
   (string-name, initial, terminal).  The initial  and
   terminal specifications may be expressions, and ,
   if so, they are evaluated automatically, just as
   the expressions in the ?if construct are evaluated,
   only these produce integer results.  For example if
   ARG is the string ABCDE and M has the value 2:
     
      Preprocessor Source                Preprocessor Output
   
   ?SUBSTRING(ARG, ,1)                A    (first character of arg)
   
   ?SUBSTRING(ARG,?LEN(ARG), )        E    (last character of arg)
   
   ?SUBSTRING(ARG, ,M)                AB   (first m charac ters of arg)
   
   ?SUBSTRING(ARG,?LEN(ARG)-M+1, )    DE   (last m characters of arg)

   Some times it is necessary to generate a unique
   name for an entity in a macro; for example, a loop
   index in a recursive expansion.  For this purpose a
   seventh intrinsic
   
                 ?unique (alias)
   
   could be provided.  It has one argument that serves
   as an alias within this macro expansion for the
   unique name the preprocessor has created.

   Relocating Text
   
   Fortran 77 and Fortran 90 have strict statement
   ordering rules.  They require all declaratives
   before all executable statements.  Sometimes it is
   convenient to specify declaratives in macros that
   are expanded after the executable statements begin.
   For example, take the loop index in a recursively
   expanded macro.  If the program contains an
   IMPLICIT NONE statement, each new loop index must
   be explicitly declared and the declarations placed
   ahead of the executable code.  To allow for this
   the preprocessor could provide a ?divert construct
   and an ?insert statement.
   
                 ?divert name
                    statement-stream
                 ?end[ ]divert [name]
   
                 ?insert name
   
   Sequences of statements are given names and any
   number of named sequences of statements are
   permitted.  The ?divert construct may appear in a
   macro body to name a sequence of declaratives that
   need to be relocated.    The sequence is
   cumulative, that is any subsequent statements given
   the same name will be treated as continuations of
   the earlier sequence.  This is analogous to COMMON
   and NAMELIST lists in Fortran 90.  The ?insert
   statement may then appear before any executable
   statements in a Fortran subprogram to indicate the
   position to which the named stream has been
   "diverted".  For example:
   
               File MYFILE
   
         ?DEFINE MACRO MYMACRO (Z)
           ?DIVERT DECLARATIVES
             REAL ?UNIQUE(I)
           ?END DIVERT DECLARATIVES
           CALL CALCULATE (I,Z)
           Z=I+Q
         ?END MACRO MYMACRO

      Preprocessor Source     Preprocessor Output
   
   SUBROUTINE EXAMPLE      SUBROUTINE EXAMPLE
    ?INCLUDE MYFILE          REAL P,Q,R
    REAL P,Q,R               REAL I?0000
    ?INSERT DECLARATIVES     REAL I?0001
    CALL CALCULATE (P,Q)     CALL CALCULATE (P,Q)
    MYMACRO (P)              CALL CALCULATE (I?0000,P)
    MYMACRO (R)              P=I?0000+Q
   END SUBROUTINE EXAMPLE    CALL CALCULATE (I?0001,R)
                             R=I?0001+Q
                           END SUBROUTINE EXAMPLE
   
   In this case I?0000 and I?0001 are the unique names
   generated by a hypothetical preprocessor (using a
   character not permitted in names and a sequence
   number).  This diversion mechanism can be used for
   statements other than declaratives, but its primary
   purpose is to get around the rigid statement
   ordering rules in Fortran.
   
   ???An alternative to this mechanism could be an
   automatic sort of statement types by the
   preprocessor such that all USE statements were
   first, followed by IMPLICIT statements, followed by
   type definition and declaration statements, etc.,
   according to the Fortran rules.???

Summary of Proposed Fortran Preprocessor Features
   
   For preprocessor variable definition:
   
      ?define [function] name  [( arg [,arg]...)] string-with-args-inserted
   
   For macro definition:
   
         ?define macro [function] name [(arg [,arg ]... )]
           macro-body-with-args-inserted
         ?end[ ]macro [name]
   
   For conditional code inclusion:
   
                 ?if (condition1) then
                    block1
                 [?elseif (condition2) then
                    block2 ]
                     ...
                 [?else
                    blockn ]
                 ?endif

   For preprocessor variable replacement and macro
   expansion
   
                 [text] name [(arg-spec [,arg- spec]...)] [text]
   
      where arg-spec is
   
                 [arg-name  =]arg-string
   
   For external file inclusion
   
                 ?include filename
   
   For statement relocation
   
                 ?divert name
                    statement-stream
                 ?end[ ]divert [name]
   
                 ?insert name
   
   Preprocessor intrinsic functions are ?present,
   ?return, ?len, ?eval, ?substring,?defined, and
   ?unique.

Summary of ANSI C Preprocessor Features

   For preprocessor variable definition:
   
         #define  name  [( arg [,arg]...)] string-with- args-inserted
         #undef name
   
   For macro definition:
   
         #define name [(arg [,arg ]... )] macro-body- with-args-inserted
   
   For conditional code inclusion:
   
                 #if (condition1)
                    block1
                 [#elif (condition2)
                    block2 ]
                     ...
                 [#else
                    blockn ]
                 #endif

                 #ifdef name
                    block
                 #endif

                 #ifndef name
                    block
                 #endif
   
   For preprocessor variable replacement and macro
   expansion
   
                 [text] name [(arg-spec [,arg- spec]...)] [text]
   
                 where arg-spec is arg-string
   
   For external file inclusion
   
                 #include filename
   
   For communication to the debugger and compiler
   
                 #line linenumber [filename]
                 #pragma [token]...
                 #error error-number
   
   Preprocessor operators
   
                 #name          paste name
                 ##             concatenate
                 defined(name)  inquiry intrinsic for definition status of name