temporal

Value Members

1. def dateField(timeCol: String, fieldCol: String, timeField: Int)(input: DataFrame): DataFrame

   

   Pipeline op to get a single field out of a date, and create a new column with that field

   Pipeline op to get a single field out of a date, and create a new column with that field

   For instance, this can take a date, and transform it to a week of the year, or a day of the month.

   timeCol

   Column spec denoting the name of a time column in the input DataFrame. In this case, the column is expected to store a Date.

   fieldCol

   The name of the column to create with the time field value

   timeField

   The field of the date to retrieve

   input

   Input pipeline data to transform

   returns

   Transformed pipeline data with the new time field column.

2. def dateFilter(minDate: Date, maxDate: Date, timeCol: String)(input: DataFrame): DataFrame

   

   Pipeline op to filter records to a specific date range.

   Pipeline op to filter records to a specific date range.

   minDate

   Start date for the range.

   maxDate

   End date for the range.

   timeCol

   Column spec denoting name of time column in input DataFrame. In this case time column is expected to store a Date.

   input

   Input pipeline data to filter.

   returns

   Transformed pipeline data, where records outside the specified time range have been removed.

3. def dateFilter(minDate: String, maxDate: String, format: String, timeCol: String)(input: DataFrame): DataFrame

   

   Pipeline op to filter records to a specific date range.

   Pipeline op to filter records to a specific date range.

   minDate

   Start date for the range, expressed in a format parsable by java.text.SimpleDateFormat.

   maxDate

   End date for the range, expressed in a format parsable by java.text.SimpleDateFormat.

   format

   Date parsing string, expressed according to java.text.SimpleDateFormat.

   timeCol

   Column spec denoting name of time column in input DataFrame.

   input

   Input pipeline data to filter.

   returns

   Transformed pipeline data, where records outside the specified time range have been removed.

4. def dateFilter(minDate: Date, maxDate: Date, format: String, timeCol: String)(input: DataFrame): DataFrame

   

   Pipeline op to filter records to a specific date range.

   Pipeline op to filter records to a specific date range.

   minDate

   Start date for the range.

   maxDate

   End date for the range.

   format

   Date parsing string, expressed according to java.text.SimpleDateFormat.

   timeCol

   Column spec denoting name of time column in input DataFrame. Column is expected to be a string.

   input

   Input pipeline data to filter.

   returns

   Transformed pipeline data, where records outside the specified time range have been removed.

5. object docs

   

   Stub object necessary due to https://issues.scala-lang.org/browse/SI-8124

   Stub object necessary due to https://issues.scala-lang.org/browse/SI-8124

   Documentation for ops.core.dataframe.temporal can be found at software.uncharted.sparkpipe.ops.core.dataframe.temporal

   Attributes

   protected[this]

   See also

   software.uncharted.sparkpipe.ops.core.dataframe.temporal

6. def parseDate(stringDateCol: String, dateCol: String, format: String)(input: DataFrame): DataFrame

   

   Pipeline op to parse a string date column into a timestamp column

   Pipeline op to parse a string date column into a timestamp column

   stringDateCol

   The column from which to get the date (as a string)

   dateCol

   The column into which to put the date (as a timestamp)

   format

   The expected format of the date

   input

   Input pipeline data to transform

   returns

   Transformed pipeline data with the new time field column.