hypernets_processor.scheduler module

Contains class for scheduling and automating hypernets data processing jobs

class hypernets_processor.scheduler.Scheduler(logger=None)

Bases: object

Class to schedule recurring jobs (i.e. like cron)

Parameters:

logger (logging.logger) – logger (optional)

get_scheduled_jobs()

Return scheduled jobs

Returns:

scheduled jobs

Return type:

list

static job_wrapper(job, parallel=False, logger=None, name=None, *args, **kwargs)

Wraps job function to provide logging, error handling and parallel processing when scheduled

Parameters:

job (func) – function

run(start_time=None)

Run scheduled jobs

Parameters:

start_time (datetime.datetime) – time to delay starting running pending jobs too

schedule(job, *args, **kwargs)

Schedule job

Parameters:

  • job (func) – job function

  • scheduler_job_config (dict) – configuration parameters for job, entries may be:

  • seconds (int) - seconds between job repeats

  • minutes (int) - minutes between job repeats

  • parallel (bool) - switch for running jobs on different threads (False if omitted)

  • name (str) - job name for logging (optional)

Parameters:

  • args – args for job

  • kwargs – kwargs for job