org.apache.flink.streaming.connectors.kafka.internals

类 KafkaShuffleFetcher<T>

java.lang.Object

org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher<T,org.apache.kafka.common.TopicPartition>

org.apache.flink.streaming.connectors.kafka.internals.KafkaFetcher<T>

org.apache.flink.streaming.connectors.kafka.internals.KafkaShuffleFetcher<T>

@Internal
public class KafkaShuffleFetcher<T>
extends KafkaFetcher<T>

Fetch data from Kafka for Kafka Shuffle.

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	KafkaShuffleFetcher.KafkaShuffleElement An element in a KafkaShuffle.
static class	KafkaShuffleFetcher.KafkaShuffleElementDeserializer<T> Deserializer for KafkaShuffleElement.
static class	KafkaShuffleFetcher.KafkaShuffleRecord<T> One value with Type T in a KafkaShuffle.
static class	KafkaShuffleFetcher.KafkaShuffleWatermark A watermark element in a KafkaShuffle.

字段概要

从类继承的字段 org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher

checkpointLock, sourceContext, unassignedPartitionsQueue, watermarkOutput

构造器概要

构造器

构造器和说明

KafkaShuffleFetcher(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T> sourceContext, Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets, org.apache.flink.util.SerializedValue<org.apache.flink.api.common.eventtime.WatermarkStrategy<T>> watermarkStrategy, org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeProvider, long autoWatermarkInterval, ClassLoader userCodeClassLoader, String taskNameWithSubtasks, KafkaDeserializationSchema<T> deserializer, Properties kafkaProperties, long pollTimeout, org.apache.flink.metrics.MetricGroup subtaskMetricGroup, org.apache.flink.metrics.MetricGroup consumerMetricGroup, boolean useMetrics, org.apache.flink.api.common.typeutils.TypeSerializer<T> typeSerializer, int producerParallelism)

方法概要

限定符和类型	方法和说明
protected String	getFetcherName() Gets the name of this fetcher, for thread naming and logging purposes.
protected void	partitionConsumerRecordsHandler(List<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>> partitionRecords, KafkaTopicPartitionState<T,org.apache.kafka.common.TopicPartition> partition)

从类继承的方法 org.apache.flink.streaming.connectors.kafka.internals.KafkaFetcher

cancel, createKafkaPartitionHandle, doCommitInternalOffsetsToKafka, runFetchLoop

从类继承的方法 org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher

addDiscoveredPartitions, commitInternalOffsetsToKafka, emitRecordsWithTimestamps, snapshotCurrentState, subscribedPartitionStates

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

KafkaShuffleFetcher

public KafkaShuffleFetcher(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T> sourceContext,
                           Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
                           org.apache.flink.util.SerializedValue<org.apache.flink.api.common.eventtime.WatermarkStrategy<T>> watermarkStrategy,
                           org.apache.flink.streaming.runtime.tasks.ProcessingTimeService processingTimeProvider,
                           long autoWatermarkInterval,
                           ClassLoader userCodeClassLoader,
                           String taskNameWithSubtasks,
                           KafkaDeserializationSchema<T> deserializer,
                           Properties kafkaProperties,
                           long pollTimeout,
                           org.apache.flink.metrics.MetricGroup subtaskMetricGroup,
                           org.apache.flink.metrics.MetricGroup consumerMetricGroup,
                           boolean useMetrics,
                           org.apache.flink.api.common.typeutils.TypeSerializer<T> typeSerializer,
                           int producerParallelism)
                    throws Exception

抛出:

Exception

方法详细资料

getFetcherName

protected String getFetcherName()

从类复制的说明: KafkaFetcher

Gets the name of this fetcher, for thread naming and logging purposes.

覆盖:

getFetcherName 在类中 KafkaFetcher<T>

partitionConsumerRecordsHandler

protected void partitionConsumerRecordsHandler(List<org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]>> partitionRecords,
                                               KafkaTopicPartitionState<T,org.apache.kafka.common.TopicPartition> partition)
                                        throws Exception

覆盖:

partitionConsumerRecordsHandler 在类中 KafkaFetcher<T>

抛出:

Exception