Grid Queueing

Interface for configuring and submitting jobs on a computing cluster. Do not use this class directly. Instead use one of the implementations that inherit from this class.

class iceprod.server.grid.BaseGrid(gridspec, queue_cfg, cfg, modules, io_loop, executor, statsd)

Interface for a generic job distribution system. Do not use this class directly. Use one of the plugins.

GRID_STATES = ('queued', 'processing', 'completed', 'error', 'unknown')

check_and_clean(*args, **kwargs)

Check and Clean the Grid

queue(*args, **kwargs)

Queue tasks to the grid

check_iceprod(*args, **kwargs)

check if any task is in a state for too long

check_grid(*args, **kwargs)

check the queueing system for problems

add_tasks_to_pilot_lookup(*args, **kwargs)

setup_pilots(*args, **kwargs)

Setup pilots for the task reqs

setup_submit_directory(*args, **kwargs)

Set up submit directory

write_cfg(*args, **kwargs)

Write the config file for a task

get_submit_args(task, cfg=None, passkey=None)

Get the submit arguments to start the loader script.

get_grid_status(*args, **kwargs)

Get all tasks running on the queue system. Returns {grid_queue_id:{status,submit_dir}}

generate_submit_file(*args, **kwargs)

Generate queueing system submit file for task in dir.

submit(*args, **kwargs)

Submit task to queueing system.

remove(*args, **kwargs)

Remove tasks from queueing system.