Trigger

Trigger is used to define how often a streaming query should be executed to produce results.

Note
Trigger is a sealed trait so all available implementations are in the same file Trigger.scala.
Note
A trigger can also be considered a batch (as in Spark Streaming).

Import org.apache.spark.sql to work with Trigger and the only implementation ProcessingTime.

import org.apache.spark.sql._

ProcessingTime

ProcessingTime is the only available implementation of Trigger sealed trait. It assumes that milliseconds is the minimum time unit.

You can create an instance of ProcessingTime using the following constructors:

  • ProcessingTime(Long) that accepts non-negative values that represent milliseconds.

    ProcessingTime(10)
  • ProcessingTime(interval: String) or ProcessingTime.create(interval: String) that accept CalendarInterval instances with or without leading interval string.

    ProcessingTime("10 milliseconds")
    ProcessingTime("interval 10 milliseconds")
  • ProcessingTime(Duration) that accepts scala.concurrent.duration.Duration instances.

    ProcessingTime(10.seconds)
  • ProcessingTime.create(interval: Long, unit: TimeUnit) for Long and java.util.concurrent.TimeUnit instances.

    ProcessingTime.create(10, TimeUnit.SECONDS)

results matching ""

    No results matching ""