protected def handle(opt: String, value: String): Boolean
SparkSubmitArguments
— spark-submit’s Command-Line Argument Parser
SparkSubmitArguments
is a custom SparkSubmitArgumentsParser
to handle the command-line arguments of spark-submit
script that the actions (i.e. submit, kill and status) use for their execution (possibly with the explicit env
environment).
Note
|
SparkSubmitArguments is created when launching spark-submit script with only args passed in and later used for printing the arguments in verbose mode.
|
handle
Method
handle
parses the input opt
argument and returns true
or throws an IllegalArgumentException
when it finds an unknown opt
.
handle
sets the internal properties in the table Command-Line Options, Spark Properties and Environment Variables.
mergeDefaultSparkProperties
Internal Method
mergeDefaultSparkProperties(): Unit
mergeDefaultSparkProperties
merges Spark properties from the default Spark properties file, i.e. spark-defaults.conf
with those specified through --conf
command-line option.