PublishGCPubSub
Description
Publishes the content of the incoming flowfile to the configured Google Cloud PubSub topic. The processor supports dynamic properties. If any dynamic properties are present, they will be sent along with the message in the form of 'attributes'.
Tags
gcp, google, google-cloud, message, publish, pubsub
Properties
In the list below required Properties are shown with an asterisk (*). Other properties are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| GCP Credentials Provider Service * | GCP Credentials Provider Service | Controller Service: GCPCredentialsService Implementations: GCPCredentialsControllerService | The Controller Service used to obtain Google Cloud Platform credentials. | |
| Project ID | gcp-project-id | Google Cloud Project ID Supports Expression Language, using Environment variables. | ||
| Topic Name * | gcp-pubsub-topic | Name of the Google Cloud PubSub Topic Supports Expression Language, using Environment variables. | ||
| Message Derivation Strategy * | Message Derivation Strategy | FlowFile Oriented |
| The strategy used to publish the incoming FlowFile to the Google Cloud PubSub endpoint. |
| Record Reader * | Record Reader | Controller Service: RecordReaderFactory Implementations: AvroReader CEFReader CSVReader ExcelReader GrokReader JsonPathReader JsonTreeReader ReaderLookup ScriptedReader Syslog5424Reader SyslogReader WindowsEventLogReader XMLReader YamlTreeReader | The Record Reader to use for incoming FlowFiles This property is only considered if:
| |
| Record Writer * | Record Writer | Controller Service: RecordSetWriterFactory Implementations: AvroRecordSetWriter CSVRecordSetWriter FreeFormTextRecordSetWriter JsonRecordSetWriter RecordSetWriterLookup ScriptedRecordSetWriter XMLRecordSetWriter | The Record Writer to use in order to serialize the data before sending to GCPubSub endpoint This property is only considered if:
| |
| Input Batch Size * | Input Batch Size | 100 | Maximum number of FlowFiles processed for each Processor invocation | |
| Maximum Message Size * | Maximum Message Size | 1 MB | The maximum size of a Google PubSub message in bytes. Defaults to 1 MB (1048576 bytes) This property is only considered if:
| |
| Batch Size Threshold * | gcp-pubsub-publish-batch-size | 15 | Indicates the number of messages the cloud service should bundle together in a batch. If not set and left empty, only one message will be used in a batch | |
| Batch Bytes Threshold * | gcp-batch-bytes | 3 MB | Publish request gets triggered based on this Batch Bytes Threshold property and the Batch Size Threshold property, whichever condition is met first. Supports Expression Language, using Environment variables. | |
| Batch Delay Threshold * | gcp-pubsub-publish-batch-delay | 100 millis | Indicates the delay threshold to use for batching. After this amount of time has elapsed (counting from the first element added), the elements will be wrapped up in a batch and sent. This value should not be set too high, usually on the order of milliseconds. Otherwise, calls might appear to never complete. | |
| API Endpoint * | api-endpoint | pubsub.googleapis.com:443 | Override the gRPC endpoint in the form of [host:port] Supports Expression Language, using Environment variables. | |
| Proxy Configuration Service | proxy-configuration-service | Controller Service: ProxyConfigurationService Implementations: StandardProxyConfigurationService | Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN |
Dynamic Properties
| Name | Value | Description |
|---|---|---|
| Attribute name | Value to be set to the attribute | Attributes to be set for the outgoing Google Cloud PubSub message Supports Expression Language: Yes, evaluated using FlowFile Attributes and Environment variables. |
Relationships
| Name | Description |
|---|---|
| failure | FlowFiles are routed to this relationship if the Google Cloud Pub/Sub operation fails. |
| retry | FlowFiles are routed to this relationship if the Google Cloud Pub/Sub operation fails but attempting the operation again may succeed. |
| success | FlowFiles are routed to this relationship after a successful Google Cloud Pub/Sub operation. |
Reads Attributes
This processor does not read attributes.
Writes Attributes
| Name | Description |
|---|---|
| gcp.pubsub.count.records | Count of pubsub messages published to the configured Google Cloud PubSub topic |
| gcp.pubsub.messageId | ID of the pubsub message published to the configured Google Cloud PubSub topic |
| gcp.pubsub.topic | Name of the Google Cloud PubSub topic the message was published to |
State Management
This component does not store state.
Restricted
This component is not restricted.
Input Requirement
This component requires an incoming relationship.
System Resource Considerations
| Scope | Description |
|---|---|
| MEMORY | The entirety of the FlowFile's content will be read into memory to be sent as a PubSub message. |