org.apache.flink.streaming.connectors.kafka

类 FlinkKafkaProducer<IN>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer<IN>

所有已实现的接口:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<IN>

直接已知子类:

FlinkKafkaShuffleProducer

已过时。

Please use KafkaSink.

@Deprecated
 @PublicEvolving
public class FlinkKafkaProducer<IN>
extends org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

Flink Sink to produce data into a Kafka topic. By default producer will use FlinkKafkaProducer.Semantic.AT_LEAST_ONCE semantic. Before using FlinkKafkaProducer.Semantic.EXACTLY_ONCE please refer to Flink's Kafka connector documentation.

另请参阅:

序列化表格

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	FlinkKafkaProducer.ContextStateSerializer 已过时。 TypeSerializer for FlinkKafkaProducer.KafkaTransactionContext.
static class	FlinkKafkaProducer.KafkaTransactionContext 已过时。 Context associated to this instance of the FlinkKafkaProducer.
static class	FlinkKafkaProducer.KafkaTransactionState 已过时。 State for handling transactions.
static class	FlinkKafkaProducer.NextTransactionalIdHint 已过时。 Keep information required to deduce next safe to use transactional id.
static class	FlinkKafkaProducer.NextTransactionalIdHintSerializer 已过时。 TypeSerializer for FlinkKafkaProducer.NextTransactionalIdHint.
static class	FlinkKafkaProducer.Semantic 已过时。 Semantics that can be chosen.
static class	FlinkKafkaProducer.TransactionStateSerializer 已过时。 TypeSerializer for FlinkKafkaProducer.KafkaTransactionState.

从类继承的嵌套类/接口 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction

org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.State<TXN,CONTEXT>, org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.StateSerializer<TXN,CONTEXT>, org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.StateSerializerSnapshot<TXN,CONTEXT>, org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction.TransactionHolder<TXN>

从接口继承的嵌套类/接口 org.apache.flink.streaming.api.functions.sink.SinkFunction

org.apache.flink.streaming.api.functions.sink.SinkFunction.Context

字段概要

字段

限定符和类型	字段和说明
protected Exception	asyncException 已过时。 Errors encountered in the async producer are stored here.
protected org.apache.kafka.clients.producer.Callback	callback 已过时。 The callback than handles error propagation or logging callbacks.
static int	DEFAULT_KAFKA_PRODUCERS_POOL_SIZE 已过时。 Default number of KafkaProducers in the pool.
static org.apache.flink.api.common.time.Time	DEFAULT_KAFKA_TRANSACTION_TIMEOUT 已过时。 Default value for kafka transaction timeout.
protected String	defaultTopicId 已过时。 The name of the default topic this producer is writing data to.
static String	KEY_DISABLE_METRICS 已过时。 Configuration key for disabling the metrics reporting.
protected AtomicLong	pendingRecords 已过时。 Number of unacknowledged records.
protected Properties	producerConfig 已过时。 User defined properties for the Producer.
static int	SAFE_SCALE_DOWN_FACTOR 已过时。 This coefficient determines what is the safe scale down factor.
protected FlinkKafkaProducer.Semantic	semantic 已过时。 Semantic chosen for this instance.
protected Map<String,int[]>	topicPartitionsMap 已过时。 Partitions of each topic.
protected boolean	writeTimestampToKafka 已过时。 Flag controlling whether we are writing the Flink record's timestamp into Kafka.

从类继承的字段 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction

pendingCommitTransactions, state, userContext

构造器概要

构造器

构造器和说明
FlinkKafkaProducer(String defaultTopic, KafkaSerializationSchema<IN> serializationSchema, Properties producerConfig, FlinkKafkaProducer.Semantic semantic) 已过时。 Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String defaultTopic, KafkaSerializationSchema<IN> serializationSchema, Properties producerConfig, FlinkKafkaProducer.Semantic semantic, int kafkaProducersPoolSize) 已过时。 Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String topicId, KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig) 已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)
FlinkKafkaProducer(String topicId, KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig, FlinkKafkaProducer.Semantic semantic) 已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)
FlinkKafkaProducer(String defaultTopicId, KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig, Optional<FlinkKafkaPartitioner<IN>> customPartitioner) 已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)
FlinkKafkaProducer(String defaultTopicId, KeyedSerializationSchema<IN> serializationSchema, Properties producerConfig, Optional<FlinkKafkaPartitioner<IN>> customPartitioner, FlinkKafkaProducer.Semantic semantic, int kafkaProducersPoolSize) 已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)
FlinkKafkaProducer(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema, Properties producerConfig) 已过时。 Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema, Properties producerConfig, FlinkKafkaPartitioner<IN> customPartitioner, FlinkKafkaProducer.Semantic semantic, int kafkaProducersPoolSize) 已过时。 Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema, Properties producerConfig, Optional<FlinkKafkaPartitioner<IN>> customPartitioner) 已过时。 Creates a FlinkKafkaProducer for a given topic.
FlinkKafkaProducer(String brokerList, String topicId, KeyedSerializationSchema<IN> serializationSchema) 已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)
FlinkKafkaProducer(String brokerList, String topicId, org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema) 已过时。 Creates a FlinkKafkaProducer for a given topic.

方法概要

限定符和类型	方法和说明
protected void	abort(FlinkKafkaProducer.KafkaTransactionState transaction) 已过时。
protected void	acknowledgeMessage() 已过时。 ATTENTION to subclass implementors: When overriding this method, please always call super.acknowledgeMessage() to keep the invariants of the internal bookkeeping of the producer.
protected FlinkKafkaProducer.KafkaTransactionState	beginTransaction() 已过时。
protected void	checkErroneous() 已过时。
void	close() 已过时。
protected void	commit(FlinkKafkaProducer.KafkaTransactionState transaction) 已过时。
protected FlinkKafkaInternalProducer<byte[],byte[]>	createProducer() 已过时。
protected void	finishRecoveringContext(Collection<FlinkKafkaProducer.KafkaTransactionState> handledTransactions) 已过时。
protected static int[]	getPartitionsByTopic(String topic, org.apache.kafka.clients.producer.Producer<byte[],byte[]> producer) 已过时。
static long	getTransactionTimeout(Properties producerConfig) 已过时。
FlinkKafkaProducer<IN>	ignoreFailuresAfterTransactionTimeout() 已过时。 Disables the propagation of exceptions thrown when committing presumably timed out Kafka transactions during recovery of the job.
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context) 已过时。
protected Optional<FlinkKafkaProducer.KafkaTransactionContext>	initializeUserContext() 已过时。
void	invoke(FlinkKafkaProducer.KafkaTransactionState transaction, IN next, org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context) 已过时。
void	open(org.apache.flink.configuration.Configuration configuration) 已过时。 Initializes the connection to Kafka.
protected void	preCommit(FlinkKafkaProducer.KafkaTransactionState transaction) 已过时。
protected void	recoverAndAbort(FlinkKafkaProducer.KafkaTransactionState transaction) 已过时。
protected void	recoverAndCommit(FlinkKafkaProducer.KafkaTransactionState transaction) 已过时。
void	setLogFailuresOnly(boolean logFailuresOnly) 已过时。 Defines whether the producer should fail on errors, or only log them.
void	setTransactionalIdPrefix(String transactionalIdPrefix) 已过时。 Specifies the prefix of the transactional.id property to be used by the producers when communicating with Kafka.
void	setWriteTimestampToKafka(boolean writeTimestampToKafka) 已过时。 If set to true, Flink will write the (event time) timestamp attached to each record into Kafka.
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context) 已过时。

从类继承的方法 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction

currentTransaction, enableTransactionTimeoutWarnings, finish, finishProcessing, getUserContext, invoke, invoke, notifyCheckpointAborted, notifyCheckpointComplete, pendingTransactions, setTransactionTimeout

从类继承的方法 org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.functions.sink.SinkFunction

writeWatermark

字段详细资料

SAFE_SCALE_DOWN_FACTOR

public static final int SAFE_SCALE_DOWN_FACTOR

已过时。

This coefficient determines what is the safe scale down factor.

If the Flink application previously failed before first checkpoint completed or we are starting new batch of FlinkKafkaProducer from scratch without clean shutdown of the previous one, FlinkKafkaProducer doesn't know what was the set of previously used Kafka's transactionalId's. In that case, it will try to play safe and abort all of the possible transactionalIds from the range of: [0, getNumberOfParallelSubtasks() * kafkaProducersPoolSize * SAFE_SCALE_DOWN_FACTOR)

The range of available to use transactional ids is: [0, getNumberOfParallelSubtasks() * kafkaProducersPoolSize)

This means that if we decrease getNumberOfParallelSubtasks() by a factor larger than SAFE_SCALE_DOWN_FACTOR we can have a left some lingering transaction.

另请参阅:

常量字段值

DEFAULT_KAFKA_PRODUCERS_POOL_SIZE

public static final int DEFAULT_KAFKA_PRODUCERS_POOL_SIZE

已过时。

Default number of KafkaProducers in the pool. See FlinkKafkaProducer.Semantic.EXACTLY_ONCE.

另请参阅:

常量字段值

DEFAULT_KAFKA_TRANSACTION_TIMEOUT

public static final org.apache.flink.api.common.time.Time DEFAULT_KAFKA_TRANSACTION_TIMEOUT

已过时。

Default value for kafka transaction timeout.

KEY_DISABLE_METRICS

public static final String KEY_DISABLE_METRICS

已过时。

Configuration key for disabling the metrics reporting.

另请参阅:

常量字段值

producerConfig

protected final Properties producerConfig

已过时。

User defined properties for the Producer.

defaultTopicId

protected final String defaultTopicId

已过时。

The name of the default topic this producer is writing data to.

topicPartitionsMap

protected final Map<String,int[]> topicPartitionsMap

已过时。

Partitions of each topic.

writeTimestampToKafka

protected boolean writeTimestampToKafka

已过时。

Flag controlling whether we are writing the Flink record's timestamp into Kafka.

semantic

protected FlinkKafkaProducer.Semantic semantic

已过时。

Semantic chosen for this instance.

callback

@Nullable
protected transient org.apache.kafka.clients.producer.Callback callback

已过时。

The callback than handles error propagation or logging callbacks.

asyncException

@Nullable
protected transient volatile Exception asyncException

已过时。

Errors encountered in the async producer are stored here.

pendingRecords

protected final AtomicLong pendingRecords

已过时。

Number of unacknowledged records.

构造器详细资料

FlinkKafkaProducer

public FlinkKafkaProducer(String brokerList,
                          String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema)

已过时。

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

参数:

brokerList - Comma separated addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined (keyless) serialization schema.

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema,
                          Properties producerConfig)

已过时。

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, SerializationSchema, Properties, Optional) instead.

参数:

topicId - ID of the Kafka topic.

serializationSchema - User defined key-less serialization schema.

producerConfig - Properties with the producer configuration.

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          Optional<FlinkKafkaPartitioner<IN>> customPartitioner)

已过时。

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a key-less SerializationSchema and possibly a custom FlinkKafkaPartitioner.

Since a key-less SerializationSchema is used, all records sent to Kafka will not have an attached key. Therefore, if a partitioner is also not provided, records will be distributed to Kafka partitions in a round-robin fashion.

参数:

topicId - The topic to write data to

serializationSchema - A key-less serializable serialization schema for turning user objects into a kafka-consumable byte[]

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be distributed to Kafka partitions in a round-robin fashion.

FlinkKafkaProducer

public FlinkKafkaProducer(String topicId,
                          org.apache.flink.api.common.serialization.SerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          @Nullable
                          FlinkKafkaPartitioner<IN> customPartitioner,
                          FlinkKafkaProducer.Semantic semantic,
                          int kafkaProducersPoolSize)

已过时。

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a key-less SerializationSchema and possibly a custom FlinkKafkaPartitioner.

Since a key-less SerializationSchema is used, all records sent to Kafka will not have an attached key. Therefore, if a partitioner is also not provided, records will be distributed to Kafka partitions in a round-robin fashion.

参数:

topicId - The topic to write data to

serializationSchema - A key-less serializable serialization schema for turning user objects into a kafka-consumable byte[]

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be distributed to Kafka partitions in a round-robin fashion.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

kafkaProducersPoolSize - Overwrite default KafkaProducers pool size (see FlinkKafkaProducer.Semantic.EXACTLY_ONCE).

FlinkKafkaProducer

@Deprecated
public FlinkKafkaProducer(String brokerList,
                                      String topicId,
                                      KeyedSerializationSchema<IN> serializationSchema)

已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, KeyedSerializationSchema, Properties, Optional) instead.

参数:

brokerList - Comma separated addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

FlinkKafkaProducer

@Deprecated
public FlinkKafkaProducer(String topicId,
                                      KeyedSerializationSchema<IN> serializationSchema,
                                      Properties producerConfig)

已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

To use a custom partitioner, please use FlinkKafkaProducer(String, KeyedSerializationSchema, Properties, Optional) instead.

参数:

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

producerConfig - Properties with the producer configuration.

FlinkKafkaProducer

@Deprecated
public FlinkKafkaProducer(String topicId,
                                      KeyedSerializationSchema<IN> serializationSchema,
                                      Properties producerConfig,
                                      FlinkKafkaProducer.Semantic semantic)

已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)

Creates a FlinkKafkaProducer for a given topic. The sink produces a DataStream to the topic.

Using this constructor, the default FlinkFixedPartitioner will be used as the partitioner. This default partitioner maps each sink subtask to a single Kafka partition (i.e. all records received by a sink subtask will end up in the same Kafka partition).

参数:

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema supporting key/value messages

producerConfig - Properties with the producer configuration.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

FlinkKafkaProducer

@Deprecated
public FlinkKafkaProducer(String defaultTopicId,
                                      KeyedSerializationSchema<IN> serializationSchema,
                                      Properties producerConfig,
                                      Optional<FlinkKafkaPartitioner<IN>> customPartitioner)

已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a keyed KeyedSerializationSchema and possibly a custom FlinkKafkaPartitioner.

If a partitioner is not provided, written records will be partitioned by the attached key of each record (as determined by KeyedSerializationSchema.serializeKey(Object)). If written records do not have a key (i.e., KeyedSerializationSchema.serializeKey(Object) returns null), they will be distributed to Kafka partitions in a round-robin fashion.

参数:

defaultTopicId - The default topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be partitioned by the key of each record (determined by KeyedSerializationSchema.serializeKey(Object)). If the keys are null, then records will be distributed to Kafka partitions in a round-robin fashion.

FlinkKafkaProducer

@Deprecated
public FlinkKafkaProducer(String defaultTopicId,
                                      KeyedSerializationSchema<IN> serializationSchema,
                                      Properties producerConfig,
                                      Optional<FlinkKafkaPartitioner<IN>> customPartitioner,
                                      FlinkKafkaProducer.Semantic semantic,
                                      int kafkaProducersPoolSize)

已过时。 use FlinkKafkaProducer(String, KafkaSerializationSchema, Properties, FlinkKafkaProducer.Semantic)

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a keyed KeyedSerializationSchema and possibly a custom FlinkKafkaPartitioner.

If a partitioner is not provided, written records will be partitioned by the attached key of each record (as determined by KeyedSerializationSchema.serializeKey(Object)). If written records do not have a key (i.e., KeyedSerializationSchema.serializeKey(Object) returns null), they will be distributed to Kafka partitions in a round-robin fashion.

参数:

defaultTopicId - The default topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

customPartitioner - A serializable partitioner for assigning messages to Kafka partitions. If a partitioner is not provided, records will be partitioned by the key of each record (determined by KeyedSerializationSchema.serializeKey(Object)). If the keys are null, then records will be distributed to Kafka partitions in a round-robin fashion.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

kafkaProducersPoolSize - Overwrite default KafkaProducers pool size (see FlinkKafkaProducer.Semantic.EXACTLY_ONCE).

FlinkKafkaProducer

public FlinkKafkaProducer(String defaultTopic,
                          KafkaSerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          FlinkKafkaProducer.Semantic semantic)

已过时。

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a KafkaSerializationSchema for serializing records to a ProducerRecord, including partitioning information.

参数:

defaultTopic - The default topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

FlinkKafkaProducer

public FlinkKafkaProducer(String defaultTopic,
                          KafkaSerializationSchema<IN> serializationSchema,
                          Properties producerConfig,
                          FlinkKafkaProducer.Semantic semantic,
                          int kafkaProducersPoolSize)

已过时。

Creates a FlinkKafkaProducer for a given topic. The sink produces its input to the topic. It accepts a KafkaSerializationSchema and possibly a custom FlinkKafkaPartitioner.

参数:

defaultTopic - The default topic to write data to

serializationSchema - A serializable serialization schema for turning user objects into a kafka-consumable byte[] supporting key/value messages

producerConfig - Configuration properties for the KafkaProducer. 'bootstrap.servers.' is the only required argument.

semantic - Defines semantic that will be used by this producer (see FlinkKafkaProducer.Semantic).

kafkaProducersPoolSize - Overwrite default KafkaProducers pool size (see FlinkKafkaProducer.Semantic.EXACTLY_ONCE).

方法详细资料

setWriteTimestampToKafka

public void setWriteTimestampToKafka(boolean writeTimestampToKafka)

已过时。

If set to true, Flink will write the (event time) timestamp attached to each record into Kafka. Timestamps must be positive for Kafka to accept them.

参数:

writeTimestampToKafka - Flag indicating if Flink's internal timestamps are written to Kafka.

setLogFailuresOnly

public void setLogFailuresOnly(boolean logFailuresOnly)

已过时。

Defines whether the producer should fail on errors, or only log them. If this is set to true, then exceptions will be only logged, if set to false, exceptions will be eventually thrown and cause the streaming program to fail (and enter recovery).

参数:

logFailuresOnly - The flag to indicate logging-only on exceptions.

setTransactionalIdPrefix

public void setTransactionalIdPrefix(String transactionalIdPrefix)

已过时。

Specifies the prefix of the transactional.id property to be used by the producers when communicating with Kafka. If not set, the transactional.id will be prefixed with taskName + "-" + operatorUid.

Note that, if we change the prefix when the Flink application previously failed before first checkpoint completed or we are starting new batch of FlinkKafkaProducer from scratch without clean shutdown of the previous one, since we don't know what was the previously used transactional.id prefix, there will be some lingering transactions left.

参数:

transactionalIdPrefix - the transactional.id prefix

抛出:

NullPointerException - Thrown, if the transactionalIdPrefix was null.

ignoreFailuresAfterTransactionTimeout

public FlinkKafkaProducer<IN> ignoreFailuresAfterTransactionTimeout()

已过时。

Disables the propagation of exceptions thrown when committing presumably timed out Kafka transactions during recovery of the job. If a Kafka transaction is timed out, a commit will never be successful. Hence, use this feature to avoid recovery loops of the Job. Exceptions will still be logged to inform the user that data loss might have occurred.

Note that we use System.currentTimeMillis() to track the age of a transaction. Moreover, only exceptions thrown during the recovery are caught, i.e., the producer will attempt at least one commit of the transaction before giving up.

覆盖:

ignoreFailuresAfterTransactionTimeout 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

open

public void open(org.apache.flink.configuration.Configuration configuration)
          throws Exception

已过时。

Initializes the connection to Kafka.

指定者:

open 在接口中 org.apache.flink.api.common.functions.RichFunction

覆盖:

open 在类中 org.apache.flink.api.common.functions.AbstractRichFunction

抛出:

Exception

invoke

public void invoke(FlinkKafkaProducer.KafkaTransactionState transaction,
                   IN next,
                   org.apache.flink.streaming.api.functions.sink.SinkFunction.Context context)
            throws FlinkKafkaException

已过时。

指定者:

invoke 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

抛出:

FlinkKafkaException

close

public void close()
           throws FlinkKafkaException

已过时。

指定者:

close 在接口中 org.apache.flink.api.common.functions.RichFunction

覆盖:

close 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

抛出:

FlinkKafkaException

beginTransaction

protected FlinkKafkaProducer.KafkaTransactionState beginTransaction()
                                                             throws FlinkKafkaException

已过时。

指定者:

beginTransaction 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

抛出:

FlinkKafkaException

preCommit

protected void preCommit(FlinkKafkaProducer.KafkaTransactionState transaction)
                  throws FlinkKafkaException

已过时。

指定者:

preCommit 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

抛出:

FlinkKafkaException

commit

protected void commit(FlinkKafkaProducer.KafkaTransactionState transaction)

已过时。

指定者:

commit 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

recoverAndCommit

protected void recoverAndCommit(FlinkKafkaProducer.KafkaTransactionState transaction)

已过时。

覆盖:

recoverAndCommit 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

abort

protected void abort(FlinkKafkaProducer.KafkaTransactionState transaction)

已过时。

指定者:

abort 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

recoverAndAbort

protected void recoverAndAbort(FlinkKafkaProducer.KafkaTransactionState transaction)

已过时。

覆盖:

recoverAndAbort 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

acknowledgeMessage

protected void acknowledgeMessage()

已过时。

ATTENTION to subclass implementors: When overriding this method, please always call super.acknowledgeMessage() to keep the invariants of the internal bookkeeping of the producer. If not, be sure to know what you are doing.

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

已过时。

指定者:

snapshotState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

覆盖:

snapshotState 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

抛出:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

已过时。

指定者:

initializeState 在接口中 org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

覆盖:

initializeState 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

抛出:

Exception

initializeUserContext

protected Optional<FlinkKafkaProducer.KafkaTransactionContext> initializeUserContext()

已过时。

覆盖:

initializeUserContext 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

finishRecoveringContext

protected void finishRecoveringContext(Collection<FlinkKafkaProducer.KafkaTransactionState> handledTransactions)

已过时。

覆盖:

finishRecoveringContext 在类中 org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,FlinkKafkaProducer.KafkaTransactionState,FlinkKafkaProducer.KafkaTransactionContext>

createProducer

protected FlinkKafkaInternalProducer<byte[],byte[]> createProducer()

已过时。

checkErroneous

protected void checkErroneous()
                       throws FlinkKafkaException

已过时。

抛出:

FlinkKafkaException

getPartitionsByTopic

protected static int[] getPartitionsByTopic(String topic,
                                            org.apache.kafka.clients.producer.Producer<byte[],byte[]> producer)

已过时。

getTransactionTimeout

public static long getTransactionTimeout(Properties producerConfig)

已过时。