ExceptionFactory

Producing content that a reasonable developer might want to read

Supporting HTTP/2 in Apache NiFi

NiFi HTTP TLS

2022-07-07 • 8 minute read

Background

The Hypertext Transfer Protocol is fundamental to Internet communication, providing a shared communication standard for websites and services across the world. After RFC 1945 defined HTTP/1.0 in 1996, and RFC 2068 described HTTP/1.1 in 1997, the protocol remained unchanged for almost 20 years.

RFC 7540 introduced a new major version of the protocol, retaining the request and response paradigm along with other basic concepts. HTTP/2 runs over the Transmission Control Protocol and supports Transport Layer Security just like HTTP/1.1, but it uses a different strategy for packet structure and socket communication. Although maintaining the same basic distinction between headers and body, HTTP/2 uses binary encoding for header information, enabling header compression known as HPACK. HTTP/1.1 supported reusing the same socket for multiple requests, but HTTP/2 extended the concept to support multiplexing, enabling parallel request and response handling over a single socket connection.

HTTP/2 communication can occur without TLS, but all major web browsers mandate encryption for protocol negotiation. HTTP/2 requires a minimum version of TLS 1.2, and also disallows a number of cipher suites that do not provide adequate security, as enumerated in RFC 7540 Appendix A. TLS 1.2 supports an extension strategy named Application-Layer Protocol Negotiation, defined in RFC 7301, which allows HTTP version selection when establishing a TLS connection. Incorporating these features enables HTTP servers to support both HTTP/1.1 and HTTP/2 on the same port, allowing HTTP clients to use HTTP/2 when available or proceed with HTTP/1.1 when necessary.

Introduction

The Internet Engineering Task Force published the HTTP/2 standard in May 2015, but supporting the protocol in Java involved several complicating factors. Enabling HTTP/2 support for server components in Apache NiFi required several adjustments to Jetty configuration settings, including customizations to work with all Java versions that NiFi supports.

NiFi leverages Jetty as the framework HTTP server, and also provides several Processors built on Jetty libraries. NiFi 1.10.0 introduced Java 11 support in November 2019, and included Java 17 support with version 1.16.0 released in November 2021, but continues to support Java 8 as of version 1.16.3. Jetty released HTTP/2 support not long after the IETF published the official standard, but more recent alignment of Java and Jetty support for Application-Layer Protocol Negotiation simplified NiFi integration. The main branch of NiFi now includes optional HTTP/2 support with ALPN for both framework and extension components across all supported Java versions.

Implementation Details

The implementation of HTTP/2 support in NiFi illustrates the importance of minor version improvements and minimum version requirements. The efforts of Jetty Project developers coordinating with OpenJDK maintainers to backport ALPN capabilities to Java 8 streamlined the process of implementing HTTP/2 support in NiFi.

Java Version Requirements

NiFi 1.15.0 introduced Java 8 Update 251 as the minimum supported version based on enhanced security requirements for JSON Web Token signatures. Released as OpenJDK 8 Update 252 in April 2020, this version of Java also included backported ALPN methods described in JEP 244, initially released in Java 9. Methods supporting application protocol selection on classes such as javax.net.ssl.SSLEngine enabled ALPN without complex workarounds. Along with various security and performance improvements, ALPN is one of multiple reasons to track current versions of the Java platform.

Jetty and ALPN

Jetty released version 9.3.0 with HTTP/2 support in June 2015, also making Java 8 the minimum runtime version required. The lack of native ALPN support in available versions of Java 8 required Jetty to introduce a specialized library with boot classpath customization. Building on backported ALPN support in OpenJDK 8 Update 252, Jetty released version 9.4.28, streamlining ALPN integration.

Available Jetty ALPN Implementations

Jetty ALPN integration includes a common ALPNProcessor interface with multiple available implementations based on the target Java version. Jetty 9.4 provides the following ALPN server implementation libraries:

The OpenJDK8ServerALPNProcessor class in jetty-alpn-openjdk8-server leverages backported ALPN support in OpenJDK 8 Update 252 and following, and also allows falling back to custom ALPN handling using a custom Jetty library. This approach enables ALPN on Java 8 with minimal dependencies. The initialization process, however, includes a runtime requirement on Java 8, which results in an IllegalStateException when attempting to run on Java 9 or higher.

The JDK9ServerALPNProcessor class in jetty-alpn-java-server uses the standard JEP 244 methods on Java 9 later, which provides the optimal solution for Java 9 and following. Although backported ALPN methods in OpenJDK 8 Update 252 use the same signature, the Jetty processor initialization includes a runtime requirement on Java 9 or later, throwing an IllegalStateException when running on Java 8, regardless of the update version.

The ConscryptServerALPNProcessor class in jetty-alpn-conscrypt-server avoids specific Java version requirements and instead relies on the Conscrypt Java cryptography provider. Conscrypt provides a solution that works across Java versions, but it introduces additional dependencies that include specific architecture and operating system requirements. For applications that do not already depend on Conscrypt, the additional libraries do not present an ideal approach.

NiFi Jetty ALPN Implementation

Changes for NIFI-3869 introduced the StandardALPNProcessor in a new nifi-jetty-configuration module to enable direct ALPN processing for Java 8, 11, and 17. The class follows the pattern of the Jetty JDK9ServerALPNProcessor, but does not include runtime evaluation of the Java version. This approach supports Java 8 Update 251 and higher, aligning with current NiFi requirements. This strategy has a minimal dependency on the jetty-io library to provide the ALPNProcessor interface and avoids depending on Jetty ALPN Server implementation libraries.

Common Jetty Configuration

Initial support for HTTP/2 under NIFI-3869 covered the ListenHTTP and HandleHttpRequest Processors. Part of the effort included the introduction of a module named nifi-jetty-configuration, which abstracts Jetty Server Connector configuration using the ServerConnectorFactory interface.

Building on the common configuration approach, changes for NIFI-9804 added configurable HTTP/2 support to the NiFi framework Jetty server. Prior to these changes, NiFi framework and extension components contained duplicative code for configuring TLS communication and other protocol options. Abstracting and refactoring similar code not only reduced duplication, but also enabled more focused unit testing. Instead of requiring extensive test setup with unrelated settings, new unit tests exercise the majority of HTTP configuration options with less complexity than previous approaches.

New Configuration Properties

Although HTTP/2 supports transparent fallback to HTTP/1.1 in absence of client-requested application protocols, some NiFi deployments may not be prepared to enable HTTP/2. Despite the benefits of HTTP/2, such as header compression and request multiplexing, making it a configurable property enables greater control and additional troubleshooting options. The initial implementation of HTTP/2 support disables the protocol in the default configuration, maintaining HTTP/1.1 as the standard setting.

Processor Properties

HTTP/2 support in ListenHTTP and HandleHttpRequest Processors added the following property with enumerated allowed values:

Configuring the HTTP Protocols property depends on specifying the SSL Context Service property to support TLS communication and protocol negotiation during the handshake process.

The allowed values contain the Application-Layer Protocol Names used during negotiation, where h2 indicates HTTP/2 and http/1.1 indicates HTTP/1.1. The default value is http/1.1.

Selecting h2 http/1.1 provides the greatest degree of compatibility with HTTP clients. Enabling both protocols allows clients connecting with TLS to indicate supported options.

Selecting h2 requires clients to support HTTP/2 without the option of falling back to HTTP/1.1. This setting is suitable for deployments where both NiFi and clients can be configured to guarantee HTTP/2 support across the board.

Application Properties

HTTP/2 support in the NiFI framework server introduced the following application property in the nifi.properties configuration:

As indicated in the property name, configuring supported application protocols requires configuring other standard HTTPS properties.

Following the same pattern as the new Processor property, the default value is http/1.1.

The property supports a space-separated list of values. The order of property values does not change runtime behavior. Configuring the following property value enables ALPN with support for both HTTP/2 and HTTP/1.1 over TLS:

nifi.web.https.application.protocols=h2 http/1.1

Runtime Differences

Although HTTP/2 maintains the same request-response semantics as HTTP/1.1, protocol changes introduce several noticeable differences in runtime behavior. Understanding these differences can be useful for evaluating potential impacts on operational systems.

Multiplexing on Single Connections

With the ability of HTTP/2 to send and receive multiple requests and responses in parallel over a single TCP connection, the overall number of connections should be fewer with HTTP/2. Lowering the number of open connections on busy systems reduces the impact on operating system connection tracking. This feature can benefit environments with reliable network communication, but it can also have an adverse impact when network packet loss is common. For these reasons, transitioning from HTTP/1.1 to HTTP/2 does not guarantee improved performance under all circumstances.

HTTP Header Capitalization

The HTTP/1.1 standard defines header field names as case-insensitive in RFC 7230 Section 3.2, and HTTP/2 maintains the same approach. As a text-based protocol, HTTP/1.1 request and response headers preserved casing, but with the switch to binary, HTTP/2 converts header names to lowercase. As a result of this approach, header names received over HTTP/2 will appear different from header names received over HTTP/1.1. This difference can impact NiFi flows that depend on evaluating HTTP headers as FlowFile attributes. Using the UpdateAttribute Processor to normalize HTTP headers enables subsequent Processors to work with expected FlowFile attribute names.

Conclusion

In June 2022, the IETF standardized HTTP/3 in RFC 9114. HTTP/3 shares some characteristics of HTTP/2, but departs from the use of TCP in favor of QUIC, which leverages UDP for the transport layer. In February 2022, Jetty introduced experimental support for HTTP/3 in versions 10.0.8 and 11.0.8. These versions of Jetty require Java 11 as the minimum runtime platform. With support for HTTP/3 still evolving, and with current NiFi support for Java 8, enabling HTTP/3 for server components is not targeted for imminent releases.

HTTP/2 was a significant step forward for the protocol, and general adoption continues to grow year over year. With HTTP/2 server support now merged into the main branch, upcoming NiFi releases will provide a robust implementation built on standard Jetty and Java features.