org.apache.flink.connector.kafka.source.enumerator.initializer

接口 OffsetsInitializer

所有超级接口:

Serializable

所有已知实现类:

NoStoppingOffsetsInitializer

@PublicEvolving
public interface OffsetsInitializer
extends Serializable

An interface for users to specify the starting / stopping offset of a KafkaPartitionSplit.

另请参阅:

ReaderHandledOffsetsInitializer, SpecifiedOffsetsInitializer, TimestampOffsetsInitializer

嵌套类概要

嵌套类

限定符和类型	接口和说明
static interface	OffsetsInitializer.PartitionOffsetsRetriever An interface that provides necessary information to the OffsetsInitializer to get the initial offsets of the Kafka partitions.

方法概要

限定符和类型	方法和说明
static OffsetsInitializer	committedOffsets() Get an OffsetsInitializer which initializes the offsets to the committed offsets.
static OffsetsInitializer	committedOffsets(org.apache.kafka.clients.consumer.OffsetResetStrategy offsetResetStrategy) Get an OffsetsInitializer which initializes the offsets to the committed offsets.
static OffsetsInitializer	earliest() Get an OffsetsInitializer which initializes the offsets to the earliest available offsets of each partition.
org.apache.kafka.clients.consumer.OffsetResetStrategy	getAutoOffsetResetStrategy() Get the auto offset reset strategy in case the initialized offsets falls out of the range.
Map<org.apache.kafka.common.TopicPartition,Long>	getPartitionOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions, OffsetsInitializer.PartitionOffsetsRetriever partitionOffsetsRetriever) Get the initial offsets for the given Kafka partitions.
static OffsetsInitializer	latest() Get an OffsetsInitializer which initializes the offsets to the latest offsets of each partition.
static OffsetsInitializer	offsets(Map<org.apache.kafka.common.TopicPartition,Long> offsets) Get an OffsetsInitializer which initializes the offsets to the specified offsets.
static OffsetsInitializer	offsets(Map<org.apache.kafka.common.TopicPartition,Long> offsets, org.apache.kafka.clients.consumer.OffsetResetStrategy offsetResetStrategy) Get an OffsetsInitializer which initializes the offsets to the specified offsets.
static OffsetsInitializer	timestamp(long timestamp) Get an OffsetsInitializer which initializes the offsets in each partition so that the initialized offset is the offset of the first record whose record timestamp is greater than or equals the given timestamp (milliseconds).

方法详细资料

getPartitionOffsets

Map<org.apache.kafka.common.TopicPartition,Long> getPartitionOffsets(Collection<org.apache.kafka.common.TopicPartition> partitions,
                                                                     OffsetsInitializer.PartitionOffsetsRetriever partitionOffsetsRetriever)

Get the initial offsets for the given Kafka partitions. These offsets will be used as either starting offsets or stopping offsets of the Kafka partitions.

If the implementation returns a starting offset which causes OffsetsOutOfRangeException from Kafka. The OffsetResetStrategy provided by the getAutoOffsetResetStrategy() will be used to reset the offset.

参数:

partitions - the Kafka partitions to get the starting offsets.

partitionOffsetsRetriever - a helper to retrieve information of the Kafka partitions.

返回:

A mapping from Kafka partition to their offsets to start consuming from.

getAutoOffsetResetStrategy

org.apache.kafka.clients.consumer.OffsetResetStrategy getAutoOffsetResetStrategy()

Get the auto offset reset strategy in case the initialized offsets falls out of the range.

The OffsetStrategy is only used when the offset initializer is used to initialize the starting offsets and the starting offsets is out of range.

返回:

An OffsetResetStrategy to use if the initialized offsets are out of the range.

committedOffsets

static OffsetsInitializer committedOffsets()

Get an OffsetsInitializer which initializes the offsets to the committed offsets. An exception will be thrown at runtime if there is no committed offsets.

返回:

an offset initializer which initialize the offsets to the committed offsets.

committedOffsets

static OffsetsInitializer committedOffsets(org.apache.kafka.clients.consumer.OffsetResetStrategy offsetResetStrategy)

Get an OffsetsInitializer which initializes the offsets to the committed offsets. Use the given OffsetResetStrategy to initialize the offsets if the committed offsets does not exist.

参数:

offsetResetStrategy - the offset reset strategy to use when the committed offsets do not exist.

返回:

an OffsetsInitializer which initializes the offsets to the committed offsets.

timestamp

static OffsetsInitializer timestamp(long timestamp)

Get an OffsetsInitializer which initializes the offsets in each partition so that the initialized offset is the offset of the first record whose record timestamp is greater than or equals the given timestamp (milliseconds).

参数:

timestamp - the timestamp (milliseconds) to start the consumption.

返回:

an OffsetsInitializer which initializes the offsets based on the given timestamp.

另请参阅:

Admin.listOffsets(Map)

earliest

static OffsetsInitializer earliest()

Get an OffsetsInitializer which initializes the offsets to the earliest available offsets of each partition.

返回:

an OffsetsInitializer which initializes the offsets to the earliest available offsets.

latest

static OffsetsInitializer latest()

Get an OffsetsInitializer which initializes the offsets to the latest offsets of each partition.

返回:

an OffsetsInitializer which initializes the offsets to the latest offsets.

offsets

static OffsetsInitializer offsets(Map<org.apache.kafka.common.TopicPartition,Long> offsets)

Get an OffsetsInitializer which initializes the offsets to the specified offsets.

参数:

offsets - the specified offsets for each partition.

返回:

an OffsetsInitializer which initializes the offsets to the specified offsets.

offsets

static OffsetsInitializer offsets(Map<org.apache.kafka.common.TopicPartition,Long> offsets,
                                  org.apache.kafka.clients.consumer.OffsetResetStrategy offsetResetStrategy)

Get an OffsetsInitializer which initializes the offsets to the specified offsets. Use the given OffsetResetStrategy to initialize the offsets in case the specified offset is out of range.

参数:

offsets - the specified offsets for each partition.

offsetResetStrategy - the OffsetResetStrategy to use when the specified offset is out of range.

返回:

an OffsetsInitializer which initializes the offsets to the specified offsets.