pipeline

Value Members

1. def addStage(stage: PipelineStage)(mlpipe: Pipeline): Pipeline

   

   Add a stage (an Estimator or a Transformer) to an existing spark.ml Pipeline

   Add a stage (an Estimator or a Transformer) to an existing spark.ml Pipeline

   stage

   a spark.ml PipelineStage (an Estimator or Transformer)

   returns

   the input Pipeline, with the added stage

2. def applyModel(args: (PipelineModel, DataFrame)): DataFrame

   

   Apply a spark.ml PipelineModel to a DataFrame to make a prediction.

   Apply a spark.ml PipelineModel to a DataFrame to make a prediction. This op is intended to be used after merging two Pipes, one providng the spark.ml PipelineModel, and the other providing the DataFrame. For example, Pipe(mlModelPipe, dataPipe).to(ops.core.ml.applyModel)

   args

   a Tuple2 including a spark.ml PipelineModel and a DataFrame

   returns

   the resultant DataFrame

3. object docs

   

   Stub object necessary due to https://issues.scala-lang.org/browse/SI-8124

   Stub object necessary due to https://issues.scala-lang.org/browse/SI-8124

   Documentation for ops.core.ml.pipeline can be found at software.uncharted.sparkpipe.ops.core.ml.pipeline

   Attributes

   protected[this]

   See also

   software.uncharted.sparkpipe.ops.core.ml.pipeline

4. def fit(args: (Pipeline, DataFrame)): PipelineModel

   

   Fit a spark.ml pipeline to a DataFrame to produce a PipelineModel.

   Fit a spark.ml pipeline to a DataFrame to produce a PipelineModel. This op is intended to be used after merging two Pipes, one providng the spark.ml Pipeline, and the other providing the DataFrame. For example, Pipe(mlPipelinePipe, dataPipe).to(ops.core.ml.fit)

   args

   a Tuple2 including a spark.ml Pipeline and a DataFrame

   returns

   the fitted spark.ml PipelineModel

5. def load(sc: SparkContext, path: String): Pipeline

   

   Load a spark.ml Pipeline from a file

   Load a spark.ml Pipeline from a file

   sc

   the SparkContext

   path

   the path to the persisted Pipeline

   returns

   a spark.ml Pipeline constructed from the given file

   Exceptions thrown

   java.lang.UnsupportedOperationException on spark version < 1.6.0

6. def save(sc: SparkContext, path: String)(mlpipe: Pipeline): Pipeline

   

   Persist a spark.ml Pipeline to a file

   Persist a spark.ml Pipeline to a file

   sc

   the SparkContext

   path

   the path for the persisted Pipeline file

   returns

   the input spark.ml Pipeline, unchanged

   Exceptions thrown

   java.lang.UnsupportedOperationException on spark version < 1.6.x