Building a Component Inventory Before a NiFi 2 Upgrade

Every NiFi 2 upgrade estimate we have seen that went wrong went wrong for the same reason: nobody knew what the flows were made of. The canvas shows you one process group at a time. The upgrade cares about every component type in every group, including the ones in a process group someone stopped in 2021 and never deleted.

So before anyone talks about Java 21 or a staging cluster, build the inventory. It is mechanical work, and it is the work that turns "a few weeks, probably" into a number you can defend.

Step 1: Get the flow definition, not the canvas

On a 1.x node the whole flow lives in the flow configuration file under the conf directory: flow.xml.gz on older releases, with flow.json.gz alongside it on later 1.x releases. Copy it off a node (any node in a cluster holds the same flow) and parse it. Both formats are verbose but regular: every processor, controller service and reporting task appears with its fully qualified type, its bundle (group, artifact, version) and its properties.

Working from the file rather than the UI or REST API has two advantages. It includes everything, stopped and disabled components too, and it gives you property values, which matter for scripted components.

Step 2: Count by type and bundle

Group the components by type and by the NAR bundle that provides them. You want a table with four columns: component type, bundle, number of instances, and the process groups they sit in. On a typical estate of a few hundred flows this comes to somewhere between forty and a hundred distinct types, heavily skewed: a dozen standard processors account for most instances, and a long tail appears once or twice each.

The long tail is where the upgrade work hides.

Step 3: Mark each type's fate in NiFi 2

For each type, record one of four outcomes:

  • Unchanged. Present in NiFi 2 with the same behaviour. Most core processors fall here.
  • Changed. Present, but with properties renamed, defaults changed or behaviour adjusted. These need configuration review and a test.
  • Replaced. Removed, with a successor that does the same job differently. Examples include the versioned Kafka processors, which give way to processors that use a Kafka connection service, and older HTTP-based Elasticsearch processors, which give way to the client-service-based ones.
  • Removed. Gone with no direct successor in the standard distribution. These need a design decision: a different component, a custom processor, or moving that part of the flow out of NiFi.

The NiFi 2 release notes and migration guidance on the Apache NiFi wiki are the primary sources for this column. Running the estate on the final 1.x release for a while also helps: its deprecation logging names components your flows use that are going away, which is a useful cross-check against your own table.

Step 4: List what is not in the standard distribution

Custom NARs deserve their own section. For each one, record what it does, where its source lives, which NiFi API version it builds against, and whether anyone can still build it. A custom processor compiled against 1.x APIs has to be rebuilt against NiFi 2 and Java 21, and a few internal APIs changed along the way. A NAR whose source is lost is a rewrite.

Step 5: Pull out every script

Search the flow for ExecuteScript, InvokeScriptedProcessor and the scripted record readers, writers and lookup services. For each, record the Script Engine property and whether the script is inline or a file path. NiFi 2 removed support for several of the languages 1.x offered, including ECMAScript, Lua, Ruby and the Jython-based Python engine. Groovy remains. Every script in one of the removed languages is a rewrite, and the script body tells you how big.

Step 6: Find templates and variables

Two more NiFi 2 removals change how flows are managed rather than what they do. XML templates are gone, so any flow that exists only as a template, or is routinely redeployed from one, needs to become a versioned flow in a registry. The variable registry is gone too, so every ${variable} reference resolved from process-group variables has to move to a parameter context. Count both.

What the inventory gives you

At the end you have a short document: total component instances, the type table with fates, the custom NAR list, the script list by language, and the template and variable counts. From that you can estimate effort per category, sequence the work, and spot the two or three flows that will consume half the schedule.

It also tells you what to retire. An inventory usually turns up process groups that have been stopped for years and flows whose output nobody reads. The cheapest component to migrate is the one you delete.

If you would rather have this done for you, it is the first half of our flow estate assessment.