ExceptionFactory

Producing content that a reasonable developer might want to read

Improving JWT Authentication in Apache NiFi

NiFi JWT Security

2021-10-23 • 14 minute read • David Handermann

Background

JSON Web Tokens provide a flexible standard for authentication and authorization in numerous web applications and frameworks. RFC 7519 outlines the foundational elements of a JWT, enumerating required encoding, formatting, and registered names for common claim attributes. Additional publications describe supporting standards, such as JSON Web Signatures in RFC 7515 and JSON Web Algorithms in RFC 7518. Other security standards such as the OAuth 2.0 framework build on these protocols to enable authorization across a variety of services.

Libraries for generating and verifying JSON Web Tokens are available for all major programming languages, making it a popular approach across a number of platforms. Due to its flexibility, and implementation problems in several libraries, some have criticized the JWT approach to application security. Despite some level of complexity compared to traditional server session management, the functional properties of JSON formatting, standard field naming, and cryptographic signing have contributed to the widespread deployment of JSON Web Tokens.

Elements of a JSON Web Token

The JWT standard defines three elements of a token: a header, a payload, and a signature. Each element consists of a string encoded using Bas64 to provide compatibility with the ASCII character set required for HTTP headers. The serialized token structure separates each of the three elements using a period character. The header and payload elements contain JSON objects with one or more properties, and the signature element contains the binary signature of the header and payload elements. RFC 7519 Section 3.1 provides an example JWT that includes both encoded and decoded representations of each element.

JWT Header

The majority JWT deployments include a header with a signing algorithm that describes both the type of cryptographic key and the hashing algorithm. The JSON Web Signature standard defines symmetric-key algorithms that leverage Hash-based Message Authentication Codes, as well as several types of asymmetric-key algorithms. Both types of cryptographic key strategies depend on the SHA-2 hash algorithm with a selectable output sizes of 256, 384, or 512 bits.

A JWT header specifying the use of symmetric-key HMAC verification with SHA-256 can be represented in JSON as follows:

{"typ":"JWT","alg":"HS256"}

The JWT header appears as follows when encoded using Base64:

eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9

JWT Payload

The payload of a JSON Web Token contains an extensible number of properties known as claims. RFC 7519 Section 4.1 defines a set of registered claims intended to provide basic identity and validity details. Implementing services may also include custom claims to provide additional authorization status information.

A JWT payload specifying a subject claim with username value and an expiration timestamp can be represented using the following JSON:

{"sub":"username","exp":1640995200}

The JWT payload appears as follows when encoded using Base64:

eyJzdWIiOiJ1c2VybmFtZSIsImV4cCI6MTY0MDk5NTIwMH0

JWT Signature

For most implementations, the signature provides a verifiable digest of the header and payload. Changing any portion of the header or payload results in a different signature. When created with a symmetric key or the private key portion of an asymmetric key pair, the signature guarantees that the header and payload contain what the issuing service initially provided. RFC 7519 Section 6 describes unsecured JWTs where the signature element is an empty string and the signing algorithm is none, but this implementation is uncommon, and requires additional security measures that make it unsuitable for most use cases.

Introduction

Apache NiFi has leveraged JSON Web Tokens to provide persistent user interface access since version 0.4.0. Outside of mutual TLS authentication using X.509 certificates, JWTs support most NiFi authentication strategies, including LDAP, Kerberos, OpenID Connect, and SAML. Following a successful exchange of credentials, NiFi issues a JWT, which the web browser uses for all subsequent requests. This approach minimizes the impact on identity providers and also streamlines application access after completing the login process.

Although JWT generation, signing, and verification are not directly visible to NiFi users or administrators, these functions are essential to application security. Recent changes in NiFi improve various aspects of JWT processing, enhancing application security in both server and client processing. These updates cover key generation, secret storage, signature verification, and token revocation for all JSON Web Tokens that NiFi produces in the course of login processing. Understanding NiFi JWT handling in light of the updated implementation is useful when evaluating authentication strategies and considering overall system security.

Implementation Summary

Updates to JWT processing involve almost every aspect of the implementation, from supporting libraries to client request formatting. Both initial and updated approaches depend on Spring Security to provide the foundational structure for web application security.

Initial Implementation

The JSON Web Token implementation in NiFi 1.14.0 and earlier includes the following features:

Updated Implementation

Updates to JWT processing include the following features:

Implementation Comparison

Refactoring NiFi JWT processing involved substantial code changes to the nifi-web-security module, including both configuration and request processing components. Changing JWT generation and handling also provided the opportunity to introduce new unit tests to validate component behavior. Recent developments in the Spring Security framework allowed replacing several custom classes with standard implementations. Although NiFi does not implement the OAuth 2.0 specification, the updated JWT implementation makes use of several Spring Security OAuth 2.0 components that provide configurable token validation. A new configuration class wires together supporting components, and individual elements use private variables to specify various aspects such as key size and processing algorithms. Although some attributes could be exposed as NiFi application properties, the internal defaults provide a high level of security for all deployments.

Supporting Libraries

The JJWT library supported token generation, signing, and verification since initial JWT processing debuted in NiFi 0.4.0. Although JJWT incorporates a wide array of features and includes extensive tests, Spring Security OAuth 2.0 relies on Nimbus JOSE JWT, which provides some additional capabilities, such as simplified support for token verification using JSON Web Keys. Both libraries provide a solid foundation for JWT handling, but for applications depending on Spring Security OAuth 2.0, Nimbus JOSE JWT is the most straightforward option for building custom capabilities. With the introduction of Spring Security dependencies including spring-security-oauth2-resource-server and spring-security-oauth2-jose , migrating to Nimbus JOSE JWT provided the most direct path to feature completion.

The Spring Security OAuth 2.0 libraries provide a number of useful components for implementing token authentication. The JwtAuthenticationProvider implements the standard Spring Security AuthenticationProvider interface and allows custom authentication conversion strategies that align with NiFi authorization components. Leveraging the Spring Security implementation removed the need for a custom class. Spring Security also provides generic JwtDecoder and OAuth2TokenValidator interfaces that abstract token parsing and validation. With extensible and composable implementations, the Spring Security OAuth 2.0 modules streamline NiFi JWT processing and provide a natural fit with the rest of the web security configuration.

The Nimbus library incorporates several standard interfaces including JWTProcessor and JWSKeySelector that provide extension points for claim validation and signature verification. Implementing these interfaces supported direct integration with Spring Security OAuth 2.0 components and also provided the opportunity for unit tests focusing on discrete features. The Nimbus library also includes a complete set of classes modeling JWT objects, making it easier to implement features without concern for direct JSON parsing and serialization.

Key Generation

The cryptographic key used for JSON Web Signature generation and verification is a foundational element of implementation security. Having a key of sufficient length and randomness is essential. A weak or compromised key allows an adversary to produce rogue JWTs that can impersonate other users or provide escalated privileges.

NiFi 1.14.0 and earlier used java.util.UUID.randomUUID() to generate a unique symmetric key for each authenticated user. Having a unique key for each user ensured that a compromised key could not be used to generate a JWT for a different user. Although the random UUID method produces a string of 36 characters, the effective randomness is much smaller.

The random UUID method uses java.security.SecureRandom to generate 16 random bytes, but UUID Version 4 requires using one byte to indicate the UUID version, and one byte to indicate the variant, reducing the effective random number of bytes to 14, or 122 bits. Although the number of potential random values is still extremely large, 122 bits is less than half of the required minimum key length for HMAC with SHA-256 as described in RFC 7518 Section 3.2.

The updated JWT implementation replaces the HMAC SHA-256 algorithm with digital signatures based on an RSA key pair. Instead of creating a key for each user, NiFi generates a shared key pair with a key size of 4096 bits. RFC 7518 Section 3.5 requires a minimum key size of 2048 bits when using RSASSA-PSS, and the NiFi value of 4096 meets current recommendations for strong RSA key pairs. NiFi uses the standard Java KeyPairGenerator interface, which delegates to the configured Java Security Provider and leverages the SecureRandom class for random generation.

Key Rotation Period

To mitigate potential key compromise, NiFi generates a new key pair at a configurable interval that defaults to one hour. The following property in nifi.properties can be configured to adjust the key rotation interval:

nifi.security.user.jws.key.rotation.period

The property supports a configurable interval using an ISO 8601 duration with a default value of PT1H. Generating new key pairs more frequently uses additional computing resources, while rotating less frequently impacts the length of time that a compromised key could remain valid.

Secret Storage

The initial NiFi JWT implementation stored generated symmetric keys in a persistent H2 database located on the file system. The database table contained one record for each user, associating the generated UUID with the user identifier. Prior to NiFi 1.10.0, the H2 database retained the same UUID symmetric key for each user after initial login. This approach did not support any type of JWT revocation, relying on the expiration claim to invalidate the token. Following updates released in NiFi 1.10.0, logging out of the user interface deleted the user’s current symmetric key, effectively invalidating current tokens and forcing generation of a new UUID on subsequent login. Despite this improvement, the H2 database stored the symmetric key without any additional protection.

Leveraging the attributes of asymmetric cryptography, the updated implementation stores generated private keys separately from public keys. Instead of persisting sensitive information to the file system, NiFi retains the current private key in memory and stores the associated public key in the local State Provider. This approach allows NiFi to verify current tokens across application restarts while avoiding insecure storage of private key material. The default local State Provider persists entries in a directory named local under the state directory within the NiFi installation. NiFi tracks public key usage in order to support verification for the lifespan of signed tokens.

Signature Algorithm

Building on the changes to key generation and secret storage, the new NiFi JWT implementation uses the PS512 JSON Web Signature algorithm in place of HS256. The HMAC SHA-256 algorithm relies on a symmetric key for both signature generation and verification, whereas other algorithms use a private key for signing and a public key for verification. Since NiFi serves as both the token issuer and resource server, the HMAC SHA-256 algorithm provided an acceptable implementation. However, the necessity of using the same key for both token creation and verification requires persistent storage of sensitive information. Migrating to an algorithm based on asymmetric key pairs removed this requirement.

In technical terms, the signature portion of a JWT generated using HMAC SHA-256 is not a cryptographic signature, but a Message Authentication Code that provides a measure of data integrity. The PS512 algorithm is one a several options that leverages asymmetric key pairs. Both RS512 and PS512 use RSA key pairs, but PS512 uses the newer RSA Signature Scheme with Appendix-Probabilistic Signature Scheme strategy described in RFC 8017 Section 8.1. The RSASSA-PSS standard provides better security in comparison to RSASSA-PKCS1-v1_5, which embeds a hash function specification that is subject to potential substitution with weaker alternatives. Although RFC 8017 Section 8 notes that no known attacks exist against the signing strategy that backs RS512, using PS512 follows current recommendations. Newer asymmetric key pair algorithms are available, such the Edwards-curve Ed25519 variant defined in RFC 8037 Section 3.1. These algorithms require additional supporting libraries, which could be considered for inclusion in future versions.

Token Revocation

With the transition from symmetric keys for each user to shared asymmetric key pairs, it was necessary to introduce a new approach to token revocation. The expiration claim enforces a limited lifespan of up to 12 hours, but revocation ensures that a token is no longer valid after completing the logout process. NiFi versions 1.10.0 to 1.14.0 implemented effective token revocation through deletion of the user’s symmetric key, but the updated approach involves tracking revoked token identifiers.

The JWT ID registered claim provides a standard method identifying unique tokens. During token generation, NiFi assigns a random UUID as the JWT ID. When the user initiates the logout process, NiFi caches the JWT ID with the associated token expiration. NiFi rejects future requests based on the cached JWT ID. Using the token expiration as the JWT ID cache expiration enables NiFi to handle the period of time between token issuance and token expiration. The JWT ID cache relies on the NiFi local State Provider, enforcing revocation across restarts. This revocation strategy stores a minimal amount of information and provides a fine-grained approach using standard JWT properties. NiFi uses the configurable key rotation period to find and remove expired revocation records.

Browser Integration

In the initial implementation of JWT processing, NiFi used the HTTP Authorization header to pass the token using the Bearer scheme defined in RFC 6750 Section 2.1. After a successful exchange of credentials, the NiFi user interface stored the JWT using Local Storage for persistent access. Based on token lifespan and persistent storage across browser instances, the user interface maintained an authenticated session without additional requests for access credentials. The interface also leveraged presence of the token to indicate whether to display the logout link.

Local Storage Issues

Using the standard HTTP Authorization header provides a straightforward means of passing the JWT in subsequent requests, but leveraging Local Storage raises potential concerns around the security of the token itself. Browser Local Storage persists across application restarts, creating the potential for valid or stale tokens to remain available on the file system for an unknown length of time. If a user closed the browser without completing the NiFi logout process, the token would remain persistent and accessible for future browser sessions. Local Storage is available to all JavaScript code executing within the NiFi user interface, creating the limited potential for Cross Site Scripting attacks. For these reasons, the Open Web Application Security Project recommends against persisting any sensitive information to Local Storage.

In addition to potential security issues, using Local Storage also created challenges for accessing application resources in separate browser instances. Features such as the NiFi content viewer required implementing a custom one-time password authentication strategy, and also caused access issues when the browser attempted to load resources for advanced user interface extensions.

To address both security and usability issues, recent updates to NiFi replaced JWT Local Storage with an HTTP session cookie. The session cookie implementation uses the HttpOnly attribute to restrict access, making it unavailable to JavaScript, which mitigates a number of potential vulnerabilities. The browser does not maintain session cookies across restarts, which avoids issues related to extended persistence of valid or stale tokens.

The new implementation uses the Strict setting for the SameSite attribute, which instructs supporting browsers to avoid sending the cookie in requests initiated from third party sites. The session cookie also uses Cookie Name Prefixes to inform supporting browsers that the cookie must include the Secure attribute, mandating HTTPS for transmission in subsequent requests.

With JavaScript access restrictions on the HTTP session cookie, the updated implementation also takes a different approach to logout support status. Rather than storing the entire token in Local Storage, the NiFi user interface stores the expiration timestamp in Session Storage. Similar to session cookies, the browser removes items from Session Storage on shutdown. This strategy relies on storing a minimal amount of information with a shorter lifespan, avoiding both the security concerns related to the token itself and potential persistence issues.

Conclusion

JSON Web Tokens in NiFi are not the most visible aspect of web application security, but they serve a vital purpose for many deployment configurations. As a flexible standard with multiple interrelated publications, developing an optimal JWT implementation involves a number of considerations. The initial deployment of JWT support in NiFi 0.4.0 addressed a variety of use cases, but technical advances and recent library developments provided several opportunities for improving the implementation. The updated JWT integration enhances security in both server and browser code, providing additional protection against potential and theoretical attacks. Most aspects of web application security require continual evaluation, and NiFi JWT support is no exception.

Acknowledgments

Special thanks to Apache NiFi maintainer Kevin Doran for highlighting opportunities for improvement in the JWT implementation, as well as reviewing significant code changes. Thanks to Apache NiFi maintainers Nathan Gough and Joe Gresock for contributing features, feedback, and testing throughout the update process.