Developer Guide

Learn about the concepts and the functionalities of the Connect File Pulse Plugin.

The Developer Guide section helps you learn about the functionalities of the File Pulse Connector and the concepts File Pulse uses to process and transform your data, and helps you obtain a deeper understanding of how File Pulse Connector works.


Installation

How to install Connect File Pulse

Basic Configuration

The commons configuration for deploying a File Pulse connector.

Scanning Files

The commons configuration for Connect File Pulse.

File Readers

The commons configuration for Connect File Pulse.

Filter Chain Definition

Learn how to define complex pipelines to transform and structure your data before integration into Kafka.

Accessing Data and Metadata

The commons configuration for Connect File Pulse.

Conditional Execution

Learn how to conditionally execute a transformation filter.

Handling Failures

Learn how to handle failures thrown while processing of records by the filter chain.

Processing Filters

The list of available transformation filters.

Tracking File Status

Learn how Connect FilePulse tracks the processing of each input file.

File Cleanup Policies

The commons configuration for Connect File Pulse.