例子
Flink集群
mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--runner=FlinkRunner --flinkMaster=<flink master> --filesToStage=target/word-count-beam-bundled-0.1.jar --inputFile=/path/to/quickstart/pom.xml --output=/tmp/counts" -Pflink-runner
管道参数说明
| Field | Description | Default Value |
|---|---|---|
runner |
The pipeline runner to use. This option allows you to determine the pipeline runner at runtime. | Set to FlinkRunnerto run using Flink. |
streaming |
Whether streaming mode is enabled or disabled; true if enabled. Set to true if running pipelines with unbounded PCollections. |
false |
flinkMaster |
The url of the Flink JobManager on which to execute pipelines. This can either be the address of a cluster JobManager, in the form "host:port" or one of the special Strings "[local]" or "[auto]". "[local]" will start a local Flink Cluster in the JVM while "[auto]" will let the system decide where to execute the pipeline based on the environment. |
[auto] |
filesToStage |
Jar Files to send to all workers and put on the classpath. Here you have to put the fat jar that contains your program along with all dependencies. | empty |
parallelism |
The degree of parallelism to be used when distributing operations onto workers. | 1 |
checkpointingInterval |
The interval between consecutive checkpoints (i.e. snapshots of the current pipeline state used for fault tolerance). | -1L, i.e. disabled |
numberOfExecutionRetries |
Sets the number of times that failed tasks are re-executed. A value of 0 effectively disables fault tolerance. A value of -1 indicates that the system default value (as defined in the configuration) should be used. |
-1 |
executionRetryDelay |
Sets the delay between executions. A value of -1 indicates that the default value should be used. |
-1 |
stateBackend |
Sets the state backend to use in streaming mode. The default is to read this setting from the Flink config. | empty, i.e. read from Flink config
|
原创文章，作者：Maggie-Hunter，如若转载，请注明出处：https://blog.ytso.com/196333.html