pyspark.sql.Column

class pyspark.sql.Column(jc)

A column in a DataFrame.

Column instances can be created by:

# 1. Select a column out of a DataFrame

df.colName
df["colName"]

# 2. Create from an expression
df.colName + 1
1 / df.colName

New in version 1.3.0.

Methods

alias(*alias, **kwargs)	Returns this column aliased with a new name or names (in the case of expressions that return more than one column, such as explode).
asc()	Returns a sort expression based on ascending order of the column.
asc_nulls_first()	Returns a sort expression based on ascending order of the column, and null values return before non-null values.
asc_nulls_last()	Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
astype(dataType)	astype() is an alias for cast().
between(lowerBound, upperBound)	A boolean expression that is evaluated to true if the value of this expression is between the given columns.
bitwiseAND(other)	Compute bitwise AND of this expression with another expression.
bitwiseOR(other)	Compute bitwise OR of this expression with another expression.
bitwiseXOR(other)	Compute bitwise XOR of this expression with another expression.
cast(dataType)	Convert the column into type dataType.
contains(other)	Contains the other element.
desc()	Returns a sort expression based on the descending order of the column.
desc_nulls_first()	Returns a sort expression based on the descending order of the column, and null values appear before non-null values.
desc_nulls_last()	Returns a sort expression based on the descending order of the column, and null values appear after non-null values.
dropFields(*fieldNames)	An expression that drops fields in StructType by name.
endswith(other)	String ends with.
eqNullSafe(other)	Equality test that is safe for null values.
getField(name)	An expression that gets a field by name in a StructField.
getItem(key)	An expression that gets an item at position ordinal out of a list, or gets an item by key out of a dict.
isNotNull()	True if the current expression is NOT null.
isNull()	True if the current expression is null.
isin(*cols)	A boolean expression that is evaluated to true if the value of this expression is contained by the evaluated values of the arguments.
like(other)	SQL like expression.
name(*alias, **kwargs)	name() is an alias for alias().
otherwise(value)	Evaluates a list of conditions and returns one of multiple possible result expressions.
over(window)	Define a windowing column.
rlike(other)	SQL RLIKE expression (LIKE with Regex).
startswith(other)	String starts with.
substr(startPos, length)	Return a Column which is a substring of the column.
when(condition, value)	Evaluates a list of conditions and returns one of multiple possible result expressions.
withField(fieldName, col)	An expression that adds/replaces a field in StructType by name.

pyspark.sql.DataFrame pyspark.sql.Row