Coding Illness: Building an OpenSSL Engine: Part1

Let's start the engine

In Part 0, we learn how to developp a basic OpenSSL engine : a dynamic library which offloads cryptography computation from the crypto library.The second step is to use our engine.

OpenSSL is a complete solution, it provides:

libraries for cryptography (libcrypto), for secure connections management (libssl)
numerous command line utilities (e.g. certificate generations, encryption ..)

OpenSSL CLI (command line interface) is accessible through the "openssl" command. Three options to this command are of interest to us:

openssl engine which lists available engines or load a new engine
openssl dgst which computes a message digest from a given data source (stdin, input file ..)
openssl enc which encrypts or decrypts messages (which we will use in a later article)

We already saw in part 0 how to use openssl engine to check that our newly built engine could be loaded successfully. Let us now use it through openssl dgst .

$ echo abc | openssl dgst -sha1
(stdin)= 03cfd743661f07975fa2f1220c5194cbaff48451

This first example computes the SHA1 digest of the string "abc" using the standard implementation of the SHA1 algorithm provided by openssl. The string is forwarded through a pipe but it could have been read from a file (using -in <input filename> option ).
Let us now apply the same command with an extra -engine option to force openssl to rely on our own SHA1 implementation:

$ echo abc | openssl dgst -engine `pwd`/build/engine_ex.so -sha1

engine "engineX" set.

(stdin)= 03cfd743661f07975fa2f1220c5194cbaff48451

Happilly the digests match. You can also notice that our engine was set which means the openssl registered it as requested. (Note: To make sure OpenSSL in indeed using your engine you can introduce a small error in the digest computed by the engine (e.g. in the function sha1_final in "e_ex.c" ) to make sure it is visible in the result).

Accessing our engine in a C program

OpenSSL CLI is a powerful tools but if you want to developp your own efficient cryptography application you may be more inclined to access OpenSSL capabilities through its C API.
In the following sections we are going to look in details at the example tests/basic_digest.c, available on the article series github.
In those sections we will see several things: how to write an OpenSSL configuration file so OpenSSL can learn the existence of our engine and where to load it from, how to load this configuration file in a program using OpenSSL libraries (loading our engine with it), how to use explicit our engine as a digest implementation to compute a message digest and finally how to define our engine as the by default implementation of OpenSSL so each time a SHA1 digest is computed through the EVP API the computation is offloaded to our engine.

Writing an OpenSSL Configuration File

A clean way to load an engine from within a C program is to integrate the engine description to an OpenSSL configuration file and to load the configuration file in your application.
The following example is a configuration file which described how to load the engine built in the previous lesson:

openssl_conf            = openssl_def
[openssl_def]
engines = engine_section
[engine_section]
engine_x = engine_x_section
[engine_x_section]
engine_id = engineX
dynamic_path = ${ENV::PWD}/build/engine_ex.so 
init = 1

An interesting feature of OpenSSL configuration file is the possibility to include envrionemment (shell) variables with the ${ENV::<var name>} syntax.

Initializing OpenSSL and loading our engine

There are several calls necessary to initialize OpenSSL properly, a detailed explaination can be found in OpenSSL wiki. In this example we will simply make the basic calls to enable engine load and use:

// initializing OpenSSL library
OPENSSL_load_builtin_modules();
ERR_load_crypto_strings();
OpenSSL_add_all_algorithms();
ENGINE_load_dynamic();

Once the library is initialized we may load the configuration file:

// building OpenSSL's configuration file path
char openssl_cnf_path[] = "./openssl.cnf"; 

// loading configuration
if (CONF_modules_load_file(openssl_cnf_path, "openssl_conf", 0) != 1) {
  fprintf(stderr, "OpenSSL failed to load required configuration\n");
  ERR_print_errors_fp(stderr);
  return 1;
}

The configuration loading will make the library load the engine described in "openssl.cnf". Thereafter, it may be accessed as follows:

ENGINE* eng = ENGINE_by_id("engineX");
if(NULL == eng) {
  printf("failed to retrieve engine by id (mppa)\n");
  return 1;
}

Forcing the use of our engine in an explicit digest call

The handle of type Engine* may be used in a EVP_DigestInit_ex call to force the use of the engine.

unsigned char digest[20];
// message digest context declaration
EVP_MD_CTX sctx;
EVP_MD_CTX* ctx = &sctx;
EVP_MD_CTX_init(ctx);
// declaring the use of SHA-1 and specifying the
// engine as the implementation to use
EVP_DigestInit_ex(ctx, EVP_sha1(), eng);
// updating digest with input data
EVP_DigestUpdate(ctx, data, len);
unsigned int dlen = -1;
// finishing and retrieving digest
EVP_DigestFinal(ctx, digest, &dlen);

The previous examples (basic_digest.c) is available on the article serie github. It must be linked with libcrypto and libssl (e.g. using -lcrypto and -lopenssl link options).

Configuring our engine to be used as default

OpenSSL provides through the ENGINE API some functions to bind an engine as the default method for any algorithm including digest:

// defining our engine as the default implementations
// for digest algorithms
ENGINE_set_default_digests(eng);

// Initializing a SHA-1 digest context
// without specifying an implementation,
// thus falling back on the default method
EVP_DigestInit(ctx, EVP_sha1());

An other way to define our engine as the default method for algorithms it supports, is to add the line "default_algorithms = ALL" to our engine section in openssl configuration file :

openssl_conf            = openssl_def
[openssl_def]
engines = engine_section
[engine_section]
engine_x = engine_x_section
[engine_x_section]
engine_id = engineX
dynamic_path = ${ENV::PWD}/build/engine_ex.so 
default_algorithms = ALL
init = 1

The "ALL" option value associates our engine as the default implementation for all algorithms (restrain to all the algorithm the engine supports). The other possible values are: RSA, DSA, DH, EC, RAND, CIPHERS, DIGESTS, PKEY, PKEY_CRYPTO and PKEY_ASN1. As our engine only support a digest (SHA1), the values ALL and DIGESTS have the same outcome in our configuration : declaring our engine as the default implementation for SHA1.

Warning: some OpenSSL non standard methods (such as SHA256(...) ) bypass the EVP API and do not depend on the default implementation. Those methods only use the software implementation built-in OpenSSL and never the engines even if defined as default methods.

References

Wikipedia's page on SHA1 https://en.wikipedia.org/wiki/SHA-1
OpenSSL manual page on Engine API https://wiki.openssl.org/index.php/Manual:Engine%283%29
OpenSSL manual page on library initialization https://wiki.openssl.org/index.php/Library_Initialization
External documentation / example of OpenSSL configuration files : file https://www.nlnetlabs.nl/downloads/publications/hsm/hsm_node18.html
An other tutorial which talks about OpenSSL configuration files : https://www.sinodun.com/2009/02/developing-an-engine-for-openssl/

Coding Illness

Building an OpenSSL Engine: Part1