The commons configuration for deploying a File Pulse connector.
Whatever the kind of files you are processing a connector should always be configured with the below properties. Those configuration are described in detail in subsequent chapters.
|The fully qualified name of the class which is used to scan file system||class||io.streamthoughts.kafka.connect.filepulse.scanner.local.LocalFSDirectoryWalker||medium|
|The fully qualified name of the class which is used to cleanup files||class||-||high|
|The input directory to scan||string||-||high|
|Time interval (in milliseconds) at wish to scan input directory||long||10000||high|
|Filters use to list eligible input files||list||-||medium|
|List of filters aliases to apply on each data (order is important)||list||-||medium|
|Name of the internal topic used by tasks and connector to report and monitor file progression.||class||connect-file-pulse-status||high|
|A list of host/port pairs uses by the reporter for establishing the initial connection to the Kafka cluster.||string||-||high|
|The fully qualified name of the class which is used by tasks to read input files||class||io.streamthoughts.kafka.connect.filepulse.reader.RowFileReader||high|
|A separated list of attributes, using ||string||path+name||high|
|The default output topic to write||string||-||high|
Prior to Connect FilePulse 1.3.x (deprecated)
|The reporter identifier to be used by tasks and connector to report and monitor file progression (default null). This property must only be set for users that have run a connector in version prior to 1.3.x to ensure backward-compatibility (when set, must be unique for each connect instance).||string||-||high|
Last modified December 9, 2021: build(github): migrate github-pages build to github actions (413f8f6d)