SparkMonitor JavaScript Documentation

Class: SparkMonitor

SparkMonitor~ SparkMonitor

new SparkMonitor()

SparkMonitor is the main singleton class that is responsible for managing CellMonitor instances for cells that run spark jobs. It also delegates spark lifecycle events from the backend to corresponding CellMonitor.

Members

cellmonitors

Dictionary of CellMonitor objects with cell_id as keys.

comm

Communication object with the kernel.

data

Data mapping jobs to cells for delegating further lifecycle events of a job.

Methods

onSparkApplicationEnd(data)

Called when a Spark Application is ended.
Parameters:
Name Type Description
data Object The data from the spark listener event.

getCellMonitor(cell_id)

Returns the CellMonitor given a cell_id
Parameters:
Name Type Description
cell_id string The Jupyter cell_id
Returns:
The CellMonitor object for the cell
Type
CellMonitor

startCellMonitor(cell)

Start a CellMonitor for a cell.
Parameters:
Name Type Description
cell CodeCell The Jupyter CodeCell instance
Returns:
The CellMonitor object for the cell
Type
CellMonitor

cellExecutedAgain(cell)

Callback called when a cell is executed again.
Parameters:
Name Type Description
cell CodeCell The Jupyter CodeCell instance.

stopCellMonitor(cell)

Stop the CellMonitor for a cell.
Parameters:
Name Type Description
cell CodeCell The Jupyter CodeCell instance

createButtons()

Adds a button to the toolbar for toggling all monitoring dispalys.

toggleAll()

Toggle all displays.

showAll()

Show all displays.

hideAll()

Hide all displays.

on_comm_msg(msg)

Called when a comm message is received from the kernel
Parameters:
Name Type Description
msg Object The JSON parsed message object.

on_comm_close(msg)

Called when comm to kernel is closed.
Parameters:
Name Type Description
msg Object The JSON parsed close message object.

startComm()

Starts communication with the kernel. Closes any existing communication.

send(msg)

Send a message to the kernel.
Parameters:
Name Type Description
msg Object The message object.

onSparkJobStart(data)

Called when a Spark job is started.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkJobEnd(data)

Called when a Spark job is ended.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkStageSubmitted(data)

Called when a Spark stage is submitted.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkStageCompleted(data)

Called when a Spark stage is completed.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkTaskStart(data)

Called when a Spark task is started.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkTaskEnd(data)

Called when a Spark task is ended.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkApplicationStart(data)

Called when a Spark Application is started.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkExecutorAdded(data)

Called when an executor is added.
Parameters:
Name Type Description
data Object The data from the spark listener event.

onSparkExecutorRemoved(data)

Called when a Spark executor is removed.
Parameters:
Name Type Description
data Object The data from the spark listener event.

handleMessage(msg)

Delegates a received message to corresponding function.
Parameters:
Name Type Description
msg Object The JSON parsed message object.