org.apache.flink.connector.base.source.reader

类 SourceReaderBase<E,T,SplitT extends org.apache.flink.api.connector.source.SourceSplit,SplitStateT>

java.lang.Object

org.apache.flink.connector.base.source.reader.SourceReaderBase<E,T,SplitT,SplitStateT>

类型参数:

E - The rich element type that contains information for split state update or timestamp extraction.

T - The final element type to emit.

SplitT - the immutable split type.

SplitStateT - the mutable type of split state.

所有已实现的接口:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SourceReader<T,SplitT>

直接已知子类:

SingleThreadMultiplexSourceReaderBase

@PublicEvolving
public abstract class SourceReaderBase<E,T,SplitT extends org.apache.flink.api.connector.source.SourceSplit,SplitStateT>
extends Object
implements org.apache.flink.api.connector.source.SourceReader<T,SplitT>

An abstract implementation of SourceReader which provides some synchronization between the mail box main thread and the SourceReader internal threads. This class allows user to just provide a SplitReader and snapshot the split state.

This implementation provides the following metrics out of the box:

• OperatorIOMetricGroup.getNumRecordsInCounter()

字段概要

字段

限定符和类型	字段和说明
protected org.apache.flink.configuration.Configuration	config The raw configurations that may be used by subclasses.
protected org.apache.flink.api.connector.source.SourceReaderContext	context The context of this source reader.
protected SourceReaderOptions	options The configuration for the reader.
protected RecordEmitter<E,T,SplitStateT>	recordEmitter The record emitter to handle the records read by the SplitReaders.
protected SplitFetcherManager<E,SplitT>	splitFetcherManager The split fetcher manager to run split fetchers.

构造器概要

构造器

构造器和说明
SourceReaderBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>> elementsQueue, SplitFetcherManager<E,SplitT> splitFetcherManager, RecordEmitter<E,T,SplitStateT> recordEmitter, org.apache.flink.configuration.Configuration config, org.apache.flink.api.connector.source.SourceReaderContext context)

方法概要

限定符和类型	方法和说明
void	addSplits(List<SplitT> splits)
void	close()
int	getNumberOfCurrentlyAssignedSplits() Gets the number of splits the reads has currently assigned.
void	handleSourceEvents(org.apache.flink.api.connector.source.SourceEvent sourceEvent)
protected abstract SplitStateT	initializedState(SplitT split) When new splits are added to the reader.
CompletableFuture<Void>	isAvailable()
void	notifyNoMoreSplits()
protected abstract void	onSplitFinished(Map<String,SplitStateT> finishedSplitIds) Handles the finished splits to clean the state if needed.
void	pauseOrResumeSplits(Collection<String> splitsToPause, Collection<String> splitsToResume)
org.apache.flink.core.io.InputStatus	pollNext(org.apache.flink.api.connector.source.ReaderOutput<T> output)
List<SplitT>	snapshotState(long checkpointId)
void	start()
protected abstract SplitT	toSplitType(String splitId, SplitStateT splitState) Convert a mutable SplitStateT to immutable SplitT.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.api.connector.source.SourceReader

notifyCheckpointComplete

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

字段详细资料

recordEmitter

protected final RecordEmitter<E,T,SplitStateT> recordEmitter

The record emitter to handle the records read by the SplitReaders.

splitFetcherManager

protected final SplitFetcherManager<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit> splitFetcherManager

The split fetcher manager to run split fetchers.

options

protected final SourceReaderOptions options

The configuration for the reader.

config

protected final org.apache.flink.configuration.Configuration config

The raw configurations that may be used by subclasses.

context

protected org.apache.flink.api.connector.source.SourceReaderContext context

The context of this source reader.

构造器详细资料

SourceReaderBase

public SourceReaderBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>> elementsQueue,
                        SplitFetcherManager<E,SplitT> splitFetcherManager,
                        RecordEmitter<E,T,SplitStateT> recordEmitter,
                        org.apache.flink.configuration.Configuration config,
                        org.apache.flink.api.connector.source.SourceReaderContext context)

方法详细资料

start

public void start()

指定者:

start 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

pollNext

public org.apache.flink.core.io.InputStatus pollNext(org.apache.flink.api.connector.source.ReaderOutput<T> output)
                                              throws Exception

指定者:

pollNext 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

抛出:

Exception

isAvailable

public CompletableFuture<Void> isAvailable()

指定者:

isAvailable 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

snapshotState

public List<SplitT> snapshotState(long checkpointId)

指定者:

snapshotState 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

addSplits

public void addSplits(List<SplitT> splits)

指定者:

addSplits 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

notifyNoMoreSplits

public void notifyNoMoreSplits()

指定者:

notifyNoMoreSplits 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

handleSourceEvents

public void handleSourceEvents(org.apache.flink.api.connector.source.SourceEvent sourceEvent)

指定者:

handleSourceEvents 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

pauseOrResumeSplits

public void pauseOrResumeSplits(Collection<String> splitsToPause,
                                Collection<String> splitsToResume)

指定者:

pauseOrResumeSplits 在接口中 org.apache.flink.api.connector.source.SourceReader<T,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

close

public void close()
           throws Exception

指定者:

close 在接口中 AutoCloseable

抛出:

Exception

getNumberOfCurrentlyAssignedSplits

public int getNumberOfCurrentlyAssignedSplits()

Gets the number of splits the reads has currently assigned.

These are the splits that have been added via addSplits(List) and have not yet been finished by returning them from the SplitReader.fetch() as part of RecordsWithSplitIds.finishedSplits().

onSplitFinished

protected abstract void onSplitFinished(Map<String,SplitStateT> finishedSplitIds)

Handles the finished splits to clean the state if needed.

initializedState

protected abstract SplitStateT initializedState(SplitT split)

When new splits are added to the reader. The initialize the state of the new splits.

参数:

split - a newly added split.

toSplitType

protected abstract SplitT toSplitType(String splitId,
                                      SplitStateT splitState)

Convert a mutable SplitStateT to immutable SplitT.

参数:

splitState - splitState.

返回:

an immutable Split state.