ExceptionFactory

Producing content that a reasonable developer might want to read

Back to The Future with PEM Certificates in Apache NiFi

NiFi Security Cryptography

2025-05-02 • 7 minute read • David Handermann

Background

Structural standards for Privacy-Enhanced Mail files include RFC 989, published in 1987, which defined message formatting in ASCII characters using headers, footers, and body encoding in Base64. RFC 1421 obsoleted several earlier versions of the PEM specification, retaining the basic structure as shown in Section 4.6. Although its name refers to mail messages, the core structure of a header line with the word BEGIN and a footer line with the word END, along with a descriptive label, has proven to provide a common pattern for other types of files, including certificates, keys, and binary messages.

Public and private encryption keys have their own binary structure, making PEM wrapping a useful strategy for serializing information in a textual format. Common public and private key algorithms use Distinguished Encoding Rules to serialize binary structures defined according to Abstract Syntax Notation One. DER encoding for ASN.1 supports interoperation across programming languages, but its historical status and complex rules present implementation challenges. Most modern programming languages abstract away these encoding details, but understanding the basic relationship of ASN.1 definition, to DER binary encoding, to PEM file formatting is important when building capabilities around key and certificate handling. Java libraries such as SSHJ highlight the variety of key encoding strategies, and the challenges of interoperability related to public and private key material. Building on years of de facto standardization, RFC 7468 codified standard PEM representations for multiple types of cryptographic structures, including private keys and certificates.

Introduction

Building on significant restructuring and technical debt reduction in Apache NiFi 2.0.0, version 2.1.0 introduced support for PEM encoded certificate and private key files in both framework configuration and extension components. NiFi supported Java KeyStore and PKCS #12 files since project inception, adding support for Bouncy Castle FIPS KeyStores in version 1.13.0. Although these keystore formats align with widespread usage, the implementation details are also specific to the Java platform. Even PKCS #12, which is supported in the popular OpenSSL library, has formatting attributes that are particular to Java implementations. Moreover, standard keystore implementations require one or more passwords to load private key and certificate information. This approach has the appearance of security, but with passwords often stored adjacent to keystore files, the actual amount of protection is minimal. With the introduction of PEM support in version 2.1.0, NiFi provides additional options for streamlined management of private keys and certificates.

Implementation Overview

The core implementation for handling PEM private key and certificate files is located in the nifi-security-ssl module, containing no additional dependencies. Although other libraries such as Bouncy Castle provide robust support for reading and writing PEM files, building support based on standard Java language features minimizes the dependency maintenance required. The direct implementation strategy also enables selective control over supported algorithms and formats. Introducing new code has its own maintenance considerations, but building the core capabilities and adding framework configuration support required a relatively small set of changes as compared to other framework features.

Supported Formats and Algorithms

PEM certificate support enables reading one or more X.509 certificates, each delimited with a BEGIN CERTIFICATE header and an END CERTIFICATE footer. RFC 5280 defines the extensive structure of X.509 Version 3 certificates, with standard ASN.1 DER encoding. The NiFi class StandardPemCertificateReader handles reading X.509 certificates, building on the java.security.cert.CertificateFactory class for converting the encoded bytes to Java Certificate objects.

PEM private key processing includes reading a single key from a file, with support for several key algorithms encoded according to PKCS #8. The PKCS #8 format defines BEGIN PRIVATE KEY as the standard header and END PRIVATE KEY as the standard footer.

PEM private key processing also supports the historical PKCS #1 encoding for RSA keys, with BEGIN RSA PRIVATE KEY as the standard header and END RSA PRIVATE KEY as the standard footer.

The NiFi class StandardPemPrivateKeyReader supports the following key algorithms:

Java 17 added native support for reading RSA private keys encoded according to PKCS #1 as described in JDK-8023980. The NiFi implementation builds on this feature and uses the java.security.KeyFactory.translateKey() method to convert the encoded binary to an RSAPrivateKey

The Java PKCS8EncodedKeySpec class provides the basis for reading PKCS #8 binary keys. The Java java.security.KeyFactory class supports generating a private key from an input key specification, but it requires obtaining an instance based on the key algorithm. The PKCS #8 structure includes the key algorithm encoded as an Object Identifier and reading the OID involves processing DER binary fields. With DER handling unavailable in public Java classes, the NiFi PEM key reader implements minimal DER field processing to read the algorithm OID. This strategy provides initial format determination, avoiding unnecessary attempts to parse binary keys with unsupported algorithms.

Controller Service Changes

The SSLContextService interface and the StandardSSLContextService implementation have supported flow configuration of keystore and truststore information since project inception. The service interface evolved to support direct access to configuration properties, which introduced tighter coupling to the standard service implementation in several Processors, requiring a new approach to support PEM certificates and private keys.

NIFI-14027 introduced a new abstraction in the SSLContextProvider Controller Service interface, supporting creation of javax.net.ssl.SSLContextService instances without access to source properties. By introducing this new interface and adjusting existing Processors, SSLContextProvider set the foundation required for a new Controller Service implementation supporting PEM certificates and private keys.

NIFI-14049 added the PEMEncodedSSLContextProvider with support for configuring certificate chains and private keys from files or property values. Building on both new capabilities and refactored service interface design, the PEM provider can be configured in most components with an SSL Context Service property.

Configuration Options

With support for PEM certificates and private keys implemented in a common module, NiFi 2.1.0 added new application properties, and flow configuration support through the PEMEncodedSSLContextProvider.

The initial implementation does not support password protection for private keys. Historical encryption strategies for PEM private keys depend on poor password-based key derivation, leading to a false sense of security and requiring an additional layer of configuration. The cert-manager project provides solid answers to questions around password protection for keystore files, and these answers also apply to PEM private keys.

Application Properties

The nifi.security.keystoreType and nifi.security.truststoreType properties control the expected format of keystore and truststore files, defaulting to PKCS12 since NiFi 1.14.0. Setting the type property value to PEM instructs the application to read separate certificate and private key properties.

Setting the nifi.security.keystoreType property to PEM requires configuring the following properties:

nifi.security.keystore.certificate
nifi.security.keystore.privateKey

Setting the nifi.security.truststoreType property to PEM requires configuring the following property:

nifi.security.truststore.certificate

All properties must reference a file path containing the required PEM encoded information.

The file referenced in nifi.security.truststore.certificate can contain multiple X.509 certificates to meet deployment requirements for organizational certificate authorities.

Controller Service Configuration

The PEMEncodedSSLContextProvider supports multiple configuration strategies based on several properties.

The Certificate Authorities Source property supports either Properties or System. The System option loads certificate authorities from the Java runtime environment without requiring additional configuration. The Properties option enables setting the Certificate Authorities property with either a file location or PEM encoded certificates.

The Private Key Source property supports Properties, Files, or Undefined. The Undefined option supports configuring the service without a private key and certificate chain for use cases that do not require mutual TLS authentication. The Properties option enables the Certificate Chain and Private Key properties, which support providing certificate and private key content as the property values. The Files option enables the Certificate Chain Location and Private Key Location properties, supporting configuration from external file locations. Although the NiFi framework supports property descriptors that can be configured with either direct values or file paths, the separate properties are necessary to enable sensitive value status for private key material.

These configuration strategies support integration with various sources for certificate and private key information, including secrets managers or containerized deployment platforms.

Conclusion

The long history of the PEM format reflects its persistence as a platform-agnostic encoding for X.509 certificates and private keys. In spite of the internal complexities around PEM, ASN.1, and DER, having support for these encoding formats in the Java platform unlocks additional integration options. Support for PEM files in NiFi 2.1.0 reflects continued focus on the joint concerns of security and compatibility, both of which remain vital to project success.