Specification: Ballerina gRPC Library

Owners: @shafreenAnfar @daneshk @BuddhiWathsala @MadhukaHarith92 @dilanSachi
Reviewers: @shafreenAnfar @daneshk @dilanSachi
Created: 2021/12/05
Updated: 2022/02/17
Edition: Swan Lake

Introduction

This is the specification for the gRPC standard library of Ballerina language, which provides APIs for gRPC client and server implementation.

The gRPC library specification has evolved and may continue to evolve in the future. The released versions of the specification can be found under the relevant GitHub tag.

If you have any feedback or suggestions about the library, start a discussion via a GitHub issue or in the Slack channel. Based on the outcome of the discussion, the specification and implementation can be updated. Community feedback is always welcome. Any accepted proposal which affects the specification is stored under /docs/proposals. Proposals under discussion can be found with the label type/proposal in GitHub.

The conforming implementation of the specification is released and included in the distribution. Any deviation from the specification is considered a bug.

Contents

  1. Overview
  2. gRPC command line interface (CLI)
  3. Protocol buffers to Ballerina data mapping
  4. gRPC communication
  5. gRPC security
  6. gRPC utility functions

1. Overview

Ballerina gRPC standard library has five primary aspects in handling values.

  1. gRPC CLI (command line interface)
  2. Protocol buffers to Ballerina data mapping
  3. gRPC communication
  4. gRPC Security
  5. gRPC utility functions

2. gRPC command line interface (CLI)

Ballerina language has a command-line interface that manages the lifecycle of a Ballerina program (such as build, test, and run). In addition, Ballerina CLI contains all the gRPC related stub and service skeleton generation capabilities. The gRPC command in Ballerina CLI is as follows.

The --input parameter is the only mandatory parameter for the Ballerina gRPC command, and it specifies the path of the protobuf file of a gRPC service. The optional --output parameter indicates the path that output will be written to. If the output path is not specified, the output will be written to a directory corresponding to the package in the Protocol Buffers definition. If the package is not specified, the output will be written to a temp directory in the current location. The optional --mode indicate what type of output files are needed. For example, if mode specifies as service, the gRPC command will generate the relevant stub file along with a service skeleton. If the mode is client, the gRPC command will generate a sample client code along with the stub. If nothing is specified, only the stub file is generated. The optional --proto-pathparameter states the path to a directory, in which to look for .proto files when resolving import directives.

3. Protocol buffers to Ballerina data mapping

The following table illustrates the data mapping of protocol buffers data types to relevant Ballerina types.

Protobuf TypeBallerina Type
google.protobuf.DoubleValuefloat
google.protobuf.FloatValuefloat
google.protobuf.Int64Valueint
google.protobuf.UInt64Valueint
google.protobuf.Int32Valueint
google.protobuf.UInt32Valueint
google.protobuf.BoolValueboolean
google.protobuf.StringValuestring
google.protobuf.BytesValuebyte[]
google.protobuf.Empty()
google.protobuf.Timestamptime:Utc
google.protobuf.Durationtime:Seconds
google.protobuf.Structmap<anydata>
google.protobuf.Any'any:Any

Note that here the 'any is the namespace of the ballerina/protobuf.types.'any submodule. Additionally, the google.protobuf.Any need serialization and deserialization mechanisms. To do that, ballerina/protobuf.types.'any module contains two APIs called pack and unpack to serialize and deserialize Any type records.

4. gRPC communication

gRPC has 4 types of RPCs (Remote Procedure Calls), and Ballerina supports all of them.

  1. Simple
  2. Server streaming
  3. Client streaming
  4. Bidirectional streaming

Note that, to explain the behaviour of these 4 RPC types, this document uses the standard Route Guide example.

4.1. Simple RPC

The RPC service definition of a simple RPCs is as follows.

The Ballerina service implementation of a gRPC can be done in two ways.

  1. Using direct returning
  2. Using a caller

Directly returning the response is the most convenient implementation. However, for asynchronous RPC calls, directly returning is not suitable, and for such use cases, using a caller is the ideal approach. In addition, each RPC call (simple, server streaming, client streaming, and bidirectional streaming) can be implemented in both ways.

RPC using direct return

Ballerina CLI generates the relevant service skeleton, and the implementation of the simple RPC call using direct return is as follows.

Here, the RPC implementation creates a featured record and directly return it from the remote method.

RPC using a caller

The Ballerina implementation of the same simple RPC using a caller is as follows.

RPC invocation

For each RPC in the protobuf definition, the generated Ballerina stub contains a client. That generated client interacts with the actual RPC service during an RPC call.

4.2. Server streaming RPC

The RPC service definition of a server streaming call is as follows.

RPC using direct return

The Ballerina implementation of the server streaming RPC using a direct return is as follows.

RPC using a caller

The Ballerina implementation of the server streaming RPC using a caller return is as follows.

RPC invocation

For each RPC in the protobuf definition, the generated Ballerina stub contains a client which interacts with the actual RPC service. In Ballerina gRPC, invoking a server streaming returns a Ballerina streaming object that can iterate through using streaming operations provided by the language.

4.3. Client streaming RPC

The RPC service definition of a client streaming call is as follows.

RPC using direct return

The Ballerina implementation of the client streaming RPC using a direct return is as follows.

RPC using a caller

The Ballerina implementation of the client streaming RPC using a caller return is as follows.

RPC invocation

For each RPC in the protobuf definition, the generated Ballerina stub contains a client. That generated client interacts with the actual RPC service during an RPC call. Unlike the server streaming scenario, the Ballerina client streaming does not use a streaming object to pass data to the client-side because it should allow users to send and receive data asynchronously. Instead, it uses a streaming object to send and receive data from the server.

4.4. Bidirectional streaming RPC

The RPC service definition of a bidirectional streaming call is as follows.

RPC using direct return

The Ballerina implementation of the bidirectional streaming RPC using a direct return is as follows.

Note that, here using direct return will not address the exact use case. This example was added, only for completeness.

RPC using a caller

The Ballerina implementation of the bidirectional streaming RPC using a caller return is as follows.

RPC invocation

For each RPC in the protobuf definition, the generated Ballerina stub contains a client. That generated client interacts with the actual RPC service during an RPC call. As the client streaming scenario, the bidirectional streaming case also uses a streaming object to send and receive data from servers.

5. gRPC security

5.1 Authentication and authorization

There are two ways to enable authentication and authorization in gRPC.

  1. Declarative approach
  2. Imperative approach

5.1.1 Declarative approach

This is also known as the configuration-driven approach, which is used for simple use cases, where users have to provide a set of configurations and do not need to be worried more about how authentication and authorization works. The user does not have full control over the configuration-driven approach.

The service configurations are used to define the authentication and authorization configurations. Users can configure the configurations needed for different authentication schemes and configurations needed for authorizations of each authentication scheme. Also, the configurations can be provided at the service level. The priority will be given from bottom to top. Then, the auth handler creation and request authentication/authorization is handled internally without user intervention. The requests that succeeded both authentication and/or authorization phases according to the configurations will be passed to the business logic layer.

5.1.1.1 Service - basic auth - file user store

Ballerina gRPC services enable authentication and authorization using a file user store by setting the grpc:FileUserStoreConfigWithScopes configurations in the listener.

5.1.1.2 Service - basic auth - LDAP user store

Ballerina gRPC services enable authentication and authorization using an LDAP user store by setting the grpc:LdapUserStoreConfigWithScopes configurations in the listener.

5.1.1.3 Service - JWT auth

Ballerina gRPC services enable authentication and authorization using JWTs by setting the grpc:JwtValidatorConfigWithScopes configurations in the listener.

5.1.1.4 Service - OAuth2

Ballerina gRPC services enable authentication and authorization using OAuth2 by setting the grpc:OAuth2IntrospectionConfigWithScopes configurations in the listener.

5.1.1.5 Client - basic auth

Ballerina gRPC clients enable basic auth with credentials by setting the grpc:CredentialsConfig configurations in the client.

5.1.1.6 Client - bearer token auth

Ballerina gRPC clients enable authentication using bearer tokens by setting the grpc:BearerTokenConfig configurations in the client.

5.1.1.7 Client - self-signed JWT auth

Ballerina gRPC clients enable authentication using JWTs by setting the grpc:JwtIssuerConfig configurations in the client.

5.1.1.8 Client - OAuth2

Ballerina gRPC clients enable authentication using OAuth2 by setting the grpc:OAuth2GrantConfig configurations in the client. OAuth2 can configure in 4 ways:

i. Client credentials grant type

ii. Password grant type

iii. Refresh token grant type

iv. JWT bearer grant type

5.1.2 Imperative approach

This is also known as the code-driven approach, which is used for advanced use cases, where users need to be worried more about how authentication and authorization work and need to have further customizations. The user has full control of the code-driven approach. The handler creation and authentication/authorization calls are made by the user at the business logic layer.

5.1.2.1 Service - basic auth - file user store

Ballerina gRPC services enable authentication and authorization using a file user store by employing the class grpc:ListenerFileUserStoreBasicAuthHandler.

5.1.2.2 Service - basic auth - LDAP user store

Ballerina gRPC services enable authentication and authorization using an LDAP user store by employing the class grpc:ListenerLdapUserStoreBasicAuthHandler.

5.1.2.3 Service - JWT auth

Ballerina gRPC services enable authentication and authorization using JWTs by employing the class grpc:ListenerJwtAuthHandler.

5.1.2.4 Service - OAuth2

Ballerina gRPC services enable authentication and authorization using OAuth2 by employing the class grpc:OAuth2IntrospectionConfig.

5.1.2.5 Client - basic auth

Ballerina gRPC clients enable authentication and authorization using basic auth by employing class grpc:ClientBasicAuthHandler. To enable authentication and authorization, the generated headers of the enrich API needs to pass to the RPC call.

5.1.2.6 Client - bearer token auth

Ballerina gRPC clients enable authentication and authorization using bearer tokens by employing class grpc:ClientBearerTokenAuthHandler. To enable authentication and authorization, the generated headers of the enrich API needs to pass to the RPC call.

5.1.2.7 Client - self-signed JWT auth

Ballerina gRPC clients enable authentication and authorization using JWTs by employing class grpc:ClientSelfSignedJwtAuthHandler. To enable authentication and authorization, the generated headers of the enrich API needs to pass to the RPC call.

5.1.2.8 Client - OAuth2

Ballerina gRPC clients enable authentication and authorization using OAuth2 by employing class grpc:ClientOAuth2Handler. To enable authentication and authorization, the generated headers of the enrich API needs to pass to the RPC call.

5.2 SSL/TLS and mutual SSL

A gRPC listener with configuration grpc:ListenerSecureSocket exposes gRPC services with SSL/TLS.

A gRPC client with configuration grpc:ClientSecureSocket can invoke gRPC services with SSL/TLS.

By configuring the mutualSsl entry in the grpc:ListenerSecureSocket, gRPC services can expose with mutual SSL.

6. gRPC utility functions

6.1. gRPC deadline

The following API sets a deadline for each request.

If a particular RPC exceeds the specified deadline, the response will be a grpc:DeadlineExceededError.

6.2. gRPC compression

The following API enables compression for gRPC calls. Currently, Gzip compression is supported by the Ballerina gRPC library.

6.3. gRPC access and trace Logs

Access and trace logs can be enabled by adding the following configurations to the Config.toml file in a Ballerina project.

6.4. gRPC retry

Client-level retrying can be enabled by passing the following configurations to the client initialization.