gRPC Interface
The google Remote Procedure Call (gRPC) interface acts as a gateway for communication between the AMT Job Control Language (JCL) and remote AMT applications. gRPC is an open-source universal RPC framework that provides efficient communication. RPC is a protocol that allows a program to execute a procedure on a remote system as if it were a local call.
The AMT gRPC interface provides a unified communication layer that enables:
- Remote file operations and File Cycle management:
- Opening or creating a File Cycle
- Updating a File Cycle
- Closing a File Cycle
- Cross-platform communication between AMT components.
- Secure, authenticated access to AMT services through OAuth 2.0 using OpenID Connect (OIDC).
gRPC Service-to-Service Authentication
The gRPC interface requires authenticated requests. Authentication uses OAuth 2.0 with OIDC. A client (JCL) obtains an access token (JWT) from the OIDC provider and sends it to the server (application) as a Bearer token.
The authentication flow involves the following steps:
- Client requests an access token from the OIDC provider's /token endpoint.
- Provider validates the request and returns an access token.
- Client includes access token in gRPC call headers.
- Server validates token signature, issuer, audience, scopes, and expiration before processing the request.
| Service Account Access |
|---|
| The OIDC client for JCL/gRPC must have client authentication enabled and be set up as a service account to obtain access tokens. |
Authenticating Application REST and gRPC Calls
The application requires two OIDC providers to be configured in the amt-config.yaml file. One for REST calls for communication between the application's backend and frontend, and another for gRPC calls. See the configuration example below:
authentication:
type: OIDC
clients:
rest:
settings:
issuerURL: <URL OIDC provider>
clientID: <Client ID>
grpc:
settings:
issuerURL: <URL OIDC provider>
clientID: <Client ID>
AMT JCL OIDC Configuration
In order to use AMT JCL through gRPC, OIDC for AMT JCL must be configured in the jcl-config.json file. See the configuration examples below.
Azure Entra Example: Below is the configuration for Microsoft Azure Active Directory (Entra ID) authentication.
"oidc_url": "<URL OIDC provider>", "oidc_client_id": "<Client ID>", "oidc_client_secret": "<Client Secret>", "oidc_scope": "api://id/.default"
Keycloak Example: Below is the configuration for Keycloak authentication with SSL certificate verification:
Set the environment variable SSL_CERT_FILE to the cert file used by keycloak. For example:
C:\AMT\Java\certs\server1.pem. Python uses the cert file to verify the certificate of Keycloak.
"oidc_url": "<localhost:port/realms/AMTRealm/protocol/openid-connect/token>", "oidc_client_id": "<Client ID>", "oidc_client_secret": "<Client Secret>", "oidc_scope": "openid offline_access"
gRPC Configuration
The gRPC interface is configured through the cc-config.yaml and amt-config.yaml files. Key configuration settings are:
| Setting | Description |
|---|---|
| grpcIdleTimeout | Timeout in seconds for both how long a client waits for a server response and when the server closes idle channels. This is a global setting, not configurable per connection. |
| gRPCPort | The network endpoint port on the host server where the gRPC service listens for incoming connections. |
| fileMessageSize | Maximum message size in MB for file operations through gRPC. |
| expiredFileTaskTimeout | Timeout in seconds for file tasks before they expire. |
Supported JCL Functions
The gRPC interface exposes the core methods for file and folder operations. The table below maps the supported AMT JCL functions to their corresponding gRPC methods:
| JCL Function | gRPC Methods | Description |
|---|---|---|
| start_procedure | FolderExists | Verifies that the procedure library folder exists before executing a procedure. |
| find_file | FileExists | Checks the presence of a target file (dataset) before use. |
| remove_file | DeleteFile, FileExists | Confirms a file exists, then removes it. This is used by step/job cleanup as well. |
| remove_empty_folders | FolderExists, GetDirectories, GetFiles, DeleteFolder | Walks a path, verifies directories are empty, and deletes them recursively. |
| remove_folder | DeleteFolder | Deletes a specified directory (optionally including contents). |
| new_record_file | OpenCreateFile, CloseFile | Creates an indexed/record-oriented file (DEFINE-like) and closes the handle. |
| export_file | CreateFolder, ExportSQLiteFile | Ensures destination directory exists, then exports a file handle to SQLite on disk. |
| copy_file | CopyFile | Duplicates a file to a new location (with retry logic at the JCL layer). |
| replace_file | CopyFile | Overwrites a target by copying a source file (explicit close/open handled in JCL layer). |
| copy_reproduce_records | CopyFile, MergeFiles | Copies data and, when needed, merges multiple sources to reproduce record sets. |
| invoke_append_file | OpenCreateFile, CloseFile, MergeFiles | Opens/creates the target and appends (merges) the source file content into it. |
| merge_files | MergeFiles | Combines multiple input files into a single output file, including long-running task polling. |
| merge_with_virtual_files | MergeFiles | Resolves a virtual input (list of files) and merges it into a concrete file before processing. |
| invoke_sort | SortFile, MergeFiles, GetFileTaskStatus | Parses SORT control statements, runs server-side sort, and optionally merges virtual inputs. |
| amt_iceman | SortFile, MergeFiles, GetFileTaskStatus | ICEMAN-compatible entry that delegates to invoke_sort. |
| amt_icetool | SortFile, MergeFiles, GetFileTaskStatus | ICETOOL-compatible orchestration of multiple SORT FROM/TO/USING operations. |
| amt_listcat | GetFiles, AddRecord | Lists files in a directory and writes catalog-style lines into an output file. |
| amt_print | ReadFileContent | Reads file content in chunks for display/validation/reporting. |
| cleanup_job / close_all_files | DeleteFolder, FileExists | Removes temporary paths and conditionally deletes files/folders at end of step/job. |
| get_file_info | GetFileInfo | Retrieves server-side metadata/status for a specific file (not invoked elsewhere by default). |
| fileaid | FileAid / FileAidObject, GetFileTaskStatus | Executes File-AID style operations (e.g., COPY, COPYALL, DROP, UPDATE) with parameter objects and polls for completion. |
| fileaid_user | FileAidUser, GetFileTaskStatus | Batch submission of multiple File-AID object operations in one request and polls for completion. |
| resolve_file_cycle / file_cycle_exists | ResolveFileCycle, FileCycleExists | GDG-like helpers to resolve an actual generation path and validate cycle existence. |
gRPC Debug Logging
The gRPC interface provides logging capabilities to help diagnose connection and performance issues. Only errors are logged by default. To enable more comprehensive logging, configure trace options and verbosity levels using environment variables.
| Environment Variables |
|---|
| For new and changed environment variables to be recognized and take effect in terminals, terminals must be reopened, including those found in IDEs. |
Trace Logging
Enable detailed tracing by setting the GRPC_TRACE environment variable with one or multiple comma-separated values:
| Trace Option | Description |
|---|---|
| api | Logs API calls and responses. |
| compression | Traces message compression/decompression. |
| op_failure | Logs operation failures and errors. |
| timer | Tracks timing information for operations. |
| timer_check | Detailed timer validation and checks. |
| http | HTTP/2 transport layer logging. |
Verbosity Levels
Control logging detail with the GRPC_VERBOSITY environment variable:
| Level | Description |
|---|---|
| DEBUG | Detailed debugging information. |
| INFO | General informational messages. |
| ERROR | Error messages only (default). |
| NONE | No logging output. |