Skip to main content

CompressContent

Description​

Compresses or decompresses the contents of FlowFiles using a user-specified compression algorithm and updates the mime.type attribute as appropriate. A common idiom is to precede CompressContent with IdentifyMimeType and configure Mode='decompress' AND Compression Format='use mime.type attribute'. When used in this manner, the MIME type is automatically detected and the data is decompressed, if necessary. If decompression is unnecessary, the data is passed through to the 'success' relationship. This processor operates in a very memory efficient way so very large objects well beyond the heap size are generally fine to process.

Tags​

brotli, bzip2, compress, content, decompress, deflate, gzip, lz4-framed, lzma, snappy, snappy framed, snappy-hadoop, xz-lzma2, zstd

Properties​

In the list below required Properties are shown with an asterisk (*). Other properties are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Mode *Modecompress
  • compress
  • decompress
Indicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress'
Compression Format *Compression Formatuse mime.type attribute
  • use mime.type attribute
  • gzip
  • deflate
  • bzip2
  • xz-lzma2
  • lzma
  • snappy
  • snappy-hadoop
  • snappy framed
  • lz4-framed
  • zstd
  • brotli
The compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Framed
Compression Level *Compression Level1
  • 0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
The compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size.

This property is only considered if:
  • the property Compression Format has a value of zstd or use mime.type attribute or deflate or brotli or gzip or xz-lzma2
  • the property Mode has a value of compress
Update Filename *Update Filenamefalse
  • true
  • false
If true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing data

Dynamic Properties​

This component does not support dynamic properties.

Relationships​

NameDescription
failureFlowFiles will be transferred to the failure relationship if they fail to compress/decompress
successFlowFiles will be transferred to the success relationship after successfully being compressed or decompressed

Reads Attributes​

NameDescription
mime.typeIf the Compression Format is set to use mime.type attribute, this attribute is used to determine the compression type. Otherwise, this attribute is ignored.

Writes Attributes​

NameDescription
mime.typeIf the Mode property is set to compress, the appropriate MIME Type is set. If the Mode property is set to decompress and the file is successfully decompressed, this attribute is removed, as the MIME Type is no longer known.

State Management​

This component does not store state.

Restricted​

This component is not restricted.

Input Requirement​

This component requires an incoming relationship.

Example Use Cases​

Use Case 1​

Compress the contents of a FlowFile

Configuration​

"Mode" = "compress"
"Compression Format" should be set to whichever compression algorithm should be used.

Use Case 2​

Decompress the contents of a FlowFile

Configuration​

"Mode" = "decompress"
"Compression Format" should be set to whichever compression algorithm was used to compress the data previously.

Example Use Cases Involving Other Components​

Multiprocessor Use Case 1​

Check whether or not a FlowFile is compressed and if so, decompress it.

If IdentifyMimeType determines that the content is not compressed, CompressContent will pass the FlowFile along to the 'success' relationship without attempting to decompress it.

Components Involved​

  • IdentifyMimeType
    1. Default property values are sufficient.
    2. Connect the 'success' relationship to CompressContent.
  • CompressContent
    1. "Mode" = "decompress"
    2. "Compression Format" = "use mime.type attribute"

System Resource Considerations​

ScopeDescription
CPUAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.

See Also​