org.apache.flink.streaming.connectors.kafka.internals

类 KafkaFetcher<T>

java.lang.Object

org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

org.apache.flink.streaming.connectors.kafka.internals.KafkaFetcher<T>

类型参数:

T - The type of elements produced by the fetcher.

直接已知子类:

KafkaShuffleFetcher

@Internal
public class KafkaFetcher<T>
extends AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

A fetcher that fetches data from Kafka brokers via the Kafka consumer API.

字段概要

从类继承的字段 org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher

checkpointLock, sourceContext, unassignedPartitionsQueue, watermarkOutput

构造器概要

构造器

构造器和说明

KafkaFetcher(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T> sourceContext, Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets, org.apache.flink.util.SerializedValue<org.apache.flink.api.common.eventtime.WatermarkStrategy<T>> watermarkStrategy, org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeProvider, long autoWatermarkInterval, ClassLoader userCodeClassLoader, String taskNameWithSubtasks, KafkaDeserializationSchema<T> deserializer, Properties kafkaProperties, long pollTimeout, org.apache.flink.metrics.MetricGroup subtaskMetricGroup, org.apache.flink.metrics.MetricGroup consumerMetricGroup, boolean useMetrics)

方法概要

限定符和类型	方法和说明
void	cancel()
org.apache.kafka.common.TopicPartition	createKafkaPartitionHandle(KafkaTopicPartition partition) Creates the Kafka version specific representation of the given topic partition.
protected void	doCommitInternalOffsetsToKafka(Map<KafkaTopicPartition,Long> offsets, KafkaCommitCallback commitCallback)
protected String	getFetcherName() Gets the name of this fetcher, for thread naming and logging purposes.
protected void	partitionConsumerRecordsHandler(List<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>> partitionRecords, KafkaTopicPartitionState<T,org.apache.kafka.common.TopicPartition> partition)
void	runFetchLoop()

从类继承的方法 org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher

addDiscoveredPartitions, commitInternalOffsetsToKafka, emitRecordsWithTimestamps, snapshotCurrentState, subscribedPartitionStates

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

KafkaFetcher

public KafkaFetcher(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T> sourceContext,
                    Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
                    org.apache.flink.util.SerializedValue<org.apache.flink.api.common.eventtime.WatermarkStrategy<T>> watermarkStrategy,
                    org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeProvider,
                    long autoWatermarkInterval,
                    ClassLoader userCodeClassLoader,
                    String taskNameWithSubtasks,
                    KafkaDeserializationSchema<T> deserializer,
                    Properties kafkaProperties,
                    long pollTimeout,
                    org.apache.flink.metrics.MetricGroup subtaskMetricGroup,
                    org.apache.flink.metrics.MetricGroup consumerMetricGroup,
                    boolean useMetrics)
             throws Exception

抛出:

Exception

方法详细资料

runFetchLoop

public void runFetchLoop()
                  throws Exception

指定者:

runFetchLoop 在类中 AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

抛出:

Exception

cancel

public void cancel()

指定者:

cancel 在类中 AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

getFetcherName

protected String getFetcherName()

Gets the name of this fetcher, for thread naming and logging purposes.

partitionConsumerRecordsHandler

protected void partitionConsumerRecordsHandler(List<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>> partitionRecords,
                                               KafkaTopicPartitionState<T,org.apache.kafka.common.TopicPartition> partition)
                                        throws Exception

抛出:

Exception

createKafkaPartitionHandle

public org.apache.kafka.common.TopicPartition createKafkaPartitionHandle(KafkaTopicPartition partition)

从类复制的说明: AbstractFetcher

Creates the Kafka version specific representation of the given topic partition.

指定者:

createKafkaPartitionHandle 在类中 AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

参数:

partition - The Flink representation of the Kafka topic partition.

返回:

The version-specific Kafka representation of the Kafka topic partition.

doCommitInternalOffsetsToKafka

protected void doCommitInternalOffsetsToKafka(Map<KafkaTopicPartition,Long> offsets,
                                              @Nonnull
                                              KafkaCommitCallback commitCallback)
                                       throws Exception

指定者:

doCommitInternalOffsetsToKafka 在类中 AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

抛出:

Exception