org.apache.flink.connector.kafka.source.reader.fetcher

类 KafkaSourceFetcherManager

java.lang.Object

org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager<E,SplitT>

org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>

org.apache.flink.connector.kafka.source.reader.fetcher.KafkaSourceFetcherManager

@Internal
public class KafkaSourceFetcherManager
extends org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>

The SplitFetcherManager for Kafka source. This class is needed to help commit the offsets to Kafka using the KafkaConsumer inside the KafkaPartitionSplitReader.

字段概要

从类继承的字段 org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

fetchers

构造器概要

构造器

构造器和说明

KafkaSourceFetcherManager(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>>> elementsQueue, java.util.function.Supplier<org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>> splitReaderSupplier, java.util.function.Consumer<Collection<String>> splitFinishedHook) Creates a new SplitFetcherManager with a single I/O threads.

方法概要

限定符和类型	方法和说明
void	commitOffsets(Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> offsetsToCommit, org.apache.kafka.clients.consumer.OffsetCommitCallback callback)

从类继承的方法 org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager

addSplits, getRunningFetcher

从类继承的方法 org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

checkErrors, close, createSplitFetcher, getNumAliveFetchers, maybeShutdownFinishedFetchers, pauseOrResumeSplits, startFetcher

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

KafkaSourceFetcherManager

public KafkaSourceFetcherManager(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>>> elementsQueue,
                                 java.util.function.Supplier<org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>,KafkaPartitionSplit>> splitReaderSupplier,
                                 java.util.function.Consumer<Collection<String>> splitFinishedHook)

Creates a new SplitFetcherManager with a single I/O threads.

参数:

elementsQueue - The queue that is used to hand over data from the I/O thread (the fetchers) to the reader (which emits the records and book-keeps the state. This must be the same queue instance that is also passed to the SourceReaderBase.

splitReaderSupplier - The factory for the split reader that connects to the source system.

splitFinishedHook - Hook for handling finished splits in split fetchers.

方法详细资料

commitOffsets

public void commitOffsets(Map<org.apache.kafka.common.TopicPartition,org.apache.kafka.clients.consumer.OffsetAndMetadata> offsetsToCommit,
                          org.apache.kafka.clients.consumer.OffsetCommitCallback callback)