The ModulaTor

MaX V5

Migration of 32 bit Modula-2 programs to 64 bit on OpenVMS Alpha

© (1999) by Günter Dotzel, ModulaWare

2nd edition, 16-Feb-1999

Introduction

This article is a follow-up of the implementation notes MaX V5: 64 bit Modula-2 for OpenVMS Alpha.

Remember, a 64 bit Modula-2 compiler offers you SIZE(INTEGER) = SIZE(CARDINAL) = SIZE(ADDRESS) = 8.

You might not want to migrate your applications to 64 bit, because you don't need the very large 64 bit address space. There might however be some reasons to consider a 64 bit migration:

• You want to use 64 bit whole number arithmetic in parts of your application program to take advantage of the higher integer precision or larger number range. In this case it might be easier to recompile all other parts of your application in 64 bit mode, to avoid type incompatibilities with different integer sizes.
• Apart from double precision floating point numbers, you already have 64 bit sized data objects and you are dealing with these objects as an array of two 32 bit integer components, a record with two 32 bit integer fields, a SYSTEM.SIGNED_64, which is the most advanced solution, but you have to import SYSTEM, or even a SYSTEM.QUADWORD, which has a type-size of 8 bytes. These data type could be replaced by an INTEGER.
• To speed up the loading of cardinal variables from memory. On Alpha there is no dedicated load instruction for unsigned 32 bit whole numbers. The LDL (load longword integer) instruction loads a 32 bit integer into an integer register, sign-extending bit 31 into bit 63. The compiler has to emit code to clear the upper 4 bytes before arithmetic or array indexing can be performed. This clear instruction is not required with 64 bit cardinals.
• You want to import a 64 bit third party library, using 64 bit sized whole numbers, addresses, or pointers in its interface.
• You think it is more natural to have 64 bit sized whole numbers, addresses, and pointers, when developing on and for a 64 bit machine.
• Or simply, because you never know, 64 bit is more than double fun than 32 bit.

What is a 64 bit Modula-2 compiler?

A 64 bit Modula-2 compiler is able to generate code for application programs, which want to exploit the 64 bit address space (very large memory, VLM).

One precondition to use VLM is that addresses, pointers, storage allocation, and all address arithmetic must be 64 bit wide.

From natural, pragmatic, and practical reasons, a 64 bit compiler requires that the pervasive whole number data types also have 64 bit size, otherwise existing source code, even if previously considered portable, would no longer compile.

Why? Assume for a moment that SIZE(ADDRESS)=8 and that SIZE(INTEGER)=SIZE(CARDINAL)=4 and lets examine two examples:

1. Look at the test_VLM example below and you'll notice, that the variable "i" can not span the array index range, once "amax" gets greater than MAX(INTEGER), which it became already. Not only do you need to modify the source code, but you'd also need to import the type SIGNED_64 from module SYSTEM in a module like test_VLM, which otherwise would not need module SYSTEM.

2. Look at the definition of the procedure WriteRecord below. The formal parameter "size" is of type CARDINAL and a 32 bit size would restrict the number of storage locations (LOC) to be transfered to MAX(CARDINAL)=4294967295, although such a restriction is not required.

What can be stored in the 64 bit space?

The underlying operating system might restrict the use of VLM. For example, the OpenVMS Linker restricts VLM to dynamic data only, which in Modula-2 terms means heap. The heap can be used to allocate large arrays. Example:

  MODULE test_VLM;
  FROM Storage IMPORT ALLOCATE;
  CONST
    amax=12345678901;
  TYPE
    Arr = ARRAY [0..amax] OF LONGREAL;
  VAR
    p: POINTER TO Arr;
    i: INTEGER;
  BEGIN
    NEW(p);
    FOR i:=0 TO amax DO
      p^[i]:=VAL(LONGREAL, i);
    END;
  END test_VLM.

The code generated by MaX V5, independent from the compilation command option /pointersize=32 or =64, can be linked for example under OpenVMS V6.1. The resulting program can be executed under OpenVMS V6.x or V7.x, under the provision that sufficient virtual memory and process quota are available.
The above example test_VLM can't be run without VLM, because the size of the array to be allocated is larger than the 32 bit address space. In fact, the array requires about 80 times the size of the 32 bit address space.

Assuming that NEW substitutes the standard Storage.ALLOCATE, any storage allocated with NEW, is allocated in the
- 32 bit address space when run on OpenVMS 6.x,
- 64 bit address space when run on OpenVMS Alpha 7.0 or later with /pointersize=64.

The OpenVMS version is detected in the body of the module Storage. If both, /pointersize=64 and VLM is present, ALLOCATE uses LIB$GET_VM_64, otherwise it uses LIB$GET_VM.

Is direct 64 bit Input/Output supported?

Yes! Being able to do direct input and output from and to VLM is important, not only for performance reasons. If direct 64 bit I/O were not possible, data in a buffer located outsize the 32 bit virtual address space must be transfered via a buffer located within the 32 but virtual address space. Also, since the OpenVMS V6.x RMS (record management services) restrict the data size per transfer to 65535 (64KB) bytes, larger transfers must split-up into several I/O-operations. OpenVMS V7.0 or later allows 2147483647 bytes (2GB) per transfer from anywhere in the 64 bit virtual address space. (See the OpenVMS Alpha Guide to 64 bit Addressing for restrictions with the SYS$PUT service routine when writing to unit record devices which do not support 64 bit addresses.)

As read- and write-operations are symetrical, we only consider writing below, using procedure WriteRecord of the Modula-2 library module FileSystem. Its definition and implementation module were compiled with /pointersize=64.

The formal parameters of the procedure FileSystem.WriteRecord are 64 bit compatible, i.e., there are no 32 bit restrictions.

  (* write an entire record to f *)
  PROCEDURE WriteRecord(
    VAR f   :File; 
    VAR rec :ARRAY OF LOC; 
      size  :CARDINAL
  );

  FileSystem.WriteRecord (f, p^, SIZE(Arr));

How does 64 bit Input/Output work?

To prove that 64 bit direct I/O works, let's go down the call hierarchy and follow the data until the control is passed to RMS. Among other statements, which do not touch the data to be transfered, procedure FileSystem.WriteRecord executes the statement

  RMSFiles.Put(f^.rab, SYSTEM.ADR(rec), size, status );

  PROCEDURE Put (
    VAR s      :STREAM;
      record   :SYSTEM.ADDRESS;
      size     :CARDINAL;
    VAR status :CARDINAL
  );
  BEGIN
    WITH s DO
      IF os64 THEN
        RBF64 := record;
        RSZ64 := size;
        status := VMS$.SYS$PUT(Adr32(s), 0, 0 );
      ELSE
        RBF := record;
        RSZ := SYSTEM.CAST(SYSTEM.SHORTWORD,size);
        status := VMS$.SYS$PUT(SYSTEM.ADR(s), 0, 0 );
      END;
    END
  END Put;

SYS$PUT transfers control directly to the device driver of the storage device (or network, if that matters).

RAB must be allocated in the 32 bit address space. This is a restriction of OpenVMS V7.x. The caller of procedure RMSFiles.Put is responsible for the allocation of RAB. In our case this is module FileSystem, where the allocation of the RAB takes place when the file is created or opened and connected to a stream. Module FileSystem uses a 32 bit storage allocation module. To check that any client of module RMSFiles also allocated RAB in the 32 bit address space, the function procedure Adr32 is used instead of SYSTEM.ADR.
Adr32 returns the address of its formal variable parameter "s", if it is a legal 32 bit address, i.e., must be sign-extended to 64 bit (MBSE) and otherwise raises the run-time error "subrange value out-of-range", when executing the VAL function in the RETURN statement of the procedure's body.
The Adr32 check is not required, since SYS$PUT would return a status value specifying the error SS$_ARG_GTR_32_BIT (Argument greater 32 bit) as function result, if the RAB passed as the first parameter in the call of SYS$PUT were outside the 32 bit address space. But this assumes that the caller of RMSFiles.Put checks the formal parameter "status", which you never know.
OpenVMS Alpha V6.x does not allocate any virtual memory outside the 32 bit addresses space. Hence a run-time check of the address of variable "s" in case of os64=FALSE is not required.

Here is the implementation of Adr32:

  PROCEDURE Adr32(
    VAR s: ARRAY OF SYSTEM.LOC
  ): SYSTEM.ADDRESS_64;
  VAR i: SYSTEM.SIGNED_64;
  BEGIN 
    i := ADR(s);
    RETURN SYSTEM.CAST(SYSTEM.ADDRESS_64, VAL(SYSTEM.SIGNED_32, i));
  END Adr32;       

In both cases, 32 bit and 64 bit mode, the procedure RMSFiles.Put then calls the OpenVMS RMS-routine SYS$PUT, which has the following definition in the foreign definition module VMS$:

  PROCEDURE SYS$PUT (
    %IMMED  rab : SYSTEM.ADDRESS_64;
    %IMMED  err : SYSTEM.ADDRESS_64;
    %IMMED  suc : SYSTEM.ADDRESS_64
   ): CARDINAL;

  ADDRESS     = CARDINAL   [0..MAX(CARDINAL)]            
  ADDRESS     = ADDRESS_64
  CARDINAL    = UNSIGNED_64
  ADDRESS_64  = UNSIGNED_64[0..MAX(UNSIGNED_64)]
  ADDRESS_32  = UNSIGNED_64[0..MAX(UNSIGNED_32)]
  UNSIGNED_32 = UNSIGNED_64[0..MAX(UNSIGNED_32)]

The Modula-2 type compatibility rules suppose an identical subrange type for ADDRESS and CARDINAL. This implies that the types ADDRESS_32 and UNSIGNED_32 have the same subrange type. A valid 32 bit address when loaded in a 64 bit register must be in canonical form, i.e., must be sign-extended from bit 31 to bit 63 (MBSE). Whole number procedure function results are returned in register 0. This precludes, that the check for a valid 32 bit address is peformed via a conversion (VAL) or an assignment of an expression with type ADDRESS_64 to a variable of type ADDRESS_32, e.g., RETURN ADR(s), with the function result type of Adr32 being ADDRESS_32.
This is the reason why procedure Adr32 above converts the value of type SIGNED_64 to type SIGNED_32, to generate the MBSE check. The shortest possible MBSE check consists of only two Alpha instructions; it is generated for the expression expression "ADR(s)" when compiling procedure Adr32 with /pointersize=32:

  SEXTL    R24,R28         ; Sign-extend longword (synonym for LDL)
  CMPEQ    R24,R28,R28     ; R28 := ORD(R24=R28)
  BEQ      R28,MBSE_error  ; If R28=0 then branch to MBSE_error,

 [MIN(SIGNED_32)     .. MAX(SIGNED_32)]
 or
 [0FFFFFFFF80000000H ..      7FFFFFFFH]

Here is the memory layout of the virtual address space on OpenVMS Alpha 7.0 and later:

   0000 0000 0000 0000H  start 32 bit user P0/P1-space, also called
                           process private space
   0000 0000 7FFF FFFFH  end   32 bit user space; total size 2 GB 

   0000 0000 8000 0000H  start 64 bit user P2-space, which includes
                           - the private page-table space,
                           - the shared page-table space, and
                           - the system S2-space
   FFFF FFFF 7FFF FFFFH  end   64 bit system S2-space; total size about
                           17,179,869 TB, but current Alpha architecture
                           implementations restrict the physical address
                           space to 4 TB

   FFFF FFFF 8000 0000H  start 32 bit S0/S1 system space
   FFFF FFFF FFFF FFFFH  end   32 bit system space; total size 2 GB

  PROCEDURE Write(
    cid: ChanId;
    from: ARRAY OF SYSTEM.LOC
  );

  PROCEDURE RawWrite(
    cid         :ChanId;
    from        :SYSTEM.ADDRESS;
    locsToWrite :CARDINAL
  );

DEFINITION MODULE RawOut;
IMPORT IOChan, SYSTEM;

PROCEDURE Write (cid: IOChan.ChanId; VAR from: ARRAY OF SYSTEM.LOC);
  (* Writes storage units to cid from successive components of from. *)
  (* same as RawIO.Write, except that "from" is VAR parameter *)

END RawOut.

IMPLEMENTATION MODULE RawOut;

FROM IOChan IMPORT ChanId, RawWrite;
FROM SYSTEM IMPORT LOC, ADR;

PROCEDURE Write(cid: ChanId; VAR from: ARRAY OF LOC);
BEGIN 
  RawWrite(cid, ADR(from), HIGH(from)+1);
END Write;

END RawOut.

What effort is to be expected with the migration to 64 bit? To get a first impression, study the modifications of the source code in the 64 bit enabled version of example module CountBits.mod, which is already listed in the recently revised edition of The ModulaTor, nr. 27 How to count bits?

Here is a non-exhaustive list of constructs to look at, when making your programs 64 bit upward compatible:

1. You have to examine every occurence of SYSTEM.CAST and variant record declaration, as well as the CAST-like use of variant record fields, whether you made any assumption about the data type size.

   ---

2. Calling a procedure having formal parameters of system storage type are a potential source of a wide range of data type size dependency errors. For value-parameters, the Modula-2 compiler accepts different sizes of formal and actual parameter. For Example:

   SIZE(SYSTEM.WORD)=4, independent from /pointersize,

   FileSystem.WriteWord(IntegerOrCardinalorAddress_Expression)

   still writes only 4 bytes, although SIZE(INTEGER)=8 with /pointersize=8, the upper 4 bytes being ignored. Using

   FileSystem.ReadWord(IntegerOrCardinalOrAddress_Variable)

   will result in a compilation error, because the variable's size must match SIZE(WORD). If you want to read only 4 bytes, independent from /pointersize, an auxiliary variable must be used. Example:

   
     VAR i: INTEGER;
     BEGIN
       FileSystem.ReadWord(f, i); 
   

   needs to be changed to

   
     VAR i: INTEGER;
        ia: SIGNED_32;
     BEGIN
       FileSystem.ReadWord(f, ia); i:=ia;
   

   The latter works independent from /pointersize.

   ---

3. As in (2) above, any call of a procedures which has a formal value or variable parameter of type ARRAY OF SYSTEM.(BYTE, LOC, SHORTWORD, WORD, LONGWORD, QUADWORD) must be inspected, whether there are implicit data type size assumptions.

Note, that most of the above also applies to migrating 32 bit Oberon-2 programs to 64 bit, except that the 64 bit OpenVMS Alpha Oberon-2 Compiler offers SIZE(LONGINT) = SIZE(SYSTEM.PTR) = 8.

The ModulaTor Forum

Education and Amusement

In January 1999, this book was the number two bestseller on Amazon:
The Motley Fool's Rule Breakers, Rule Makers by David and Tom Gardner is a business and investing book about companies who got successful, because they did break rules and how to locate winning long-term businesses. This book is the latest from their investment book trilogy. The famous Gardner brothers also wrote
The Motley Fool Investment Guide and
The Motley Fool Investment Workbook.
The Unemotional Investor: Simple Systems for Beating the Market by Robert Sheard. Until 1998, he was affiliated with the Motley Fools and is now a professional money manager.

IMPRESSUM: The ModulaTor is an unrefereed journal. Technical papers are to be taken as working papers and personal rather than organizational statements. Items are printed at the discretion of the Editor based upon his judgement on the interest and relevancy to the readership. Letters, announcements, and other items of professional interest are selected on the same basis. Office of publication. The Editor of The ModulaTor is Günter Dotzel; he can be reached at mailto:[email deleted due to spam]

Books Music Video	Enter keywords...