What is Connect FilePulse ?

An introduction to Connect File Pulse

What is it?

Connect FilePulse is a polyvalent, scalable and reliable, Apache Kafka Connect plugin that makes it easy to parse, transform and stream any file, in any format, into Apache Kafkaâ„¢.

Key Features

Connect FilePulse provides a set of built-in features for streaming local files into Kafka. This includes, among other things:

  • Support for recursive scanning of local directories.
  • Reading and writing files into Kafka line by line.
  • Support multiple input file formats (e.g: CSV, Avro, XML).
  • Parsing and transforming data using built-in or custom processing filters.
  • Error handler definition
  • Monitoring files while they are being written into Kafka
  • Support plugeable strategies to cleanup up completed files
  • Etc.

Why do I want it?

Connect FilePulse helps you streams local files into Apache Kafka.

  • What is it good for?: Connect FilePulse lets you define complex pipelines to transform and structure your data before integration into Kafka.

  • What is it not good for?: Connect FilePulse is not attented to be used for streaming files from a remote storage (AWS S3, HDFS, etc).

Where should I go next?

Give your users next steps from the Overview. For example:


Last modified November 11, 2020: docs(site): add site archive for v1.5.x (00d3752)