The JobQueen class (think of a queen bee) has quite a few responsibilities: More...

#include <JobQueen.hh>

Inheritance diagram for protocols::jd3::JobQueen:

Public Member Functions
	JobQueen ()

	~JobQueen () override

JobDigraphOP	create_and_set_initial_job_dag ()
	Creates the initial job dag using create_initial_job_dag function and sets it to the JQ. More...

void	note_job_completed_and_track (LarvalJobCOP job, JobStatus status, core::Size nresults)
	Call note_job_completed in derived classes and track the job using the JobTracker. More...

LarvalJobs	determine_job_list_and_track (JobDAGNodeID job_dag_node_index, core::Size max_njobs)
	Call determine_job_list in derived classes and track the jobs using the JobTracker. More...

virtual std::string	job_definition_xsd () const =0
	All JobQueens must describe their job input XML format in the form of an XSD (XML Schema Definition), and they must validate their job input files against their XSDs. If the JobDistributor is awakened with the flag "jd3::output_job_xsd <output file>" on the command line, then the JobDistributor will write out the XSD to the output file. More...

virtual std::string	resource_definition_xsd () const =0
	JobQueens may optionally define an XSD for their resource definition file which is fed to their resource manager (if they control one). If the JobDistributor is awakened with the flag "jd3::output_resource_xsd <output file>" on the command line, and if the derived JobQueen defines an XSD for resources she uses, then the JobDistributor will write out the resource definition XSD to the output file. The derived queen may return an empty string to indicate that no resources are definable for the resource manager. More...

virtual void	update_job_dag (JobDigraphUpdater &updater)=0
	The JobQueen is allowed to update the JobDigraph over the course of execution but only in a particular way: by adding new nodes, and then by adding edges that land on those new nodes. The JobQueen is not allowed to add edges that land on an old node. More...

virtual LarvalJobs	determine_job_list (JobDAGNodeID job_dag_node_index, core::Size max_njobs)=0
	The JobDistributor asks the JobQueen for a list of jobs that should be performed. This function neither knows nor cares what jobs are already complete on disk/memory. More...

virtual bool	has_job_previously_been_output (LarvalJobCOP job)=0

virtual JobOP	mature_larval_job (LarvalJobCOP job, utility::vector1< JobResultCOP > const &input_job_results)=0
	Mature the input larval job into a full fledged job that will be run within this process (i.e. on this CPU), and thus can hold pointers to data that may persist inside the ResourceManager or may even be used by another process running in this thread. To pull that off, the Job object must share no non-bitwise-const data with any other Job object. That means if the Job were to contain a Pose and a Mover (as the MoverJob does), then so each Mover should be freshly constructed, and if the Pose had been copied from an existing Pose, must use the Poses "deep_copy" method (since Pose's sometimes share non-constant data between them, e.g. the AtomTree observer system, and sometimes Constraints ). The JobResults vector, which is supplied by the Jobdistributor, has entries corresponding to the JobResults from JobIDs in its input_job_result_indices vector. More...

virtual void	note_job_completed (LarvalJobCOP job, JobStatus status, core::Size nresults)=0
	The JobDistributor will call this function to inform the JobQueen that a job has "completed" – in the sense that it will not be run in the future. It does not guarantee that the job was run on this CPU or that it successfully completed anywhere. The JobStatus indicates whether the job completed or whether it failed. The nresults count lists how many JobSummary/JobResult pairs were produced by the completed job. More...

virtual void	completed_job_summary (LarvalJobCOP job, ResultIndex result_index, JobSummaryOP summary)=0
	The JobDistributor guarnatees that exactly one JobQueen will see every JobSummary generated within a Job batch. This guarantee allows the JobQueen to aggregate data across all of the Jobs so that Rosetta is able to compute data from structures instead of forcing that computation into accessory scripts. More...

virtual std::list < output::OutputSpecificationOP >	jobs_that_should_be_output ()=0
	The JobDistributor asks the JobQueen which JobResults should be queued for output. The JobQueen should reply by returning a list of OutputSpecification objects. Each OutputSpecification indicates a particular JobResult using the JobResultID object that the Specification holds. It also indicates how the JobResult should be output by whatever object will be doing to job outputting. The JobQueen should ask for a given JobResult to be output only once. After a job result is output, the JobDistributor will discard the JobResult, so the JobQueen should not tell the JobDistributor to output a job if it will be used as an input to another job in the future. Note that this method will only be called on the master-node JobQueen (aka JQ0), but that she will not necessarily be the JobQueen to actually return the ResultOutputter for the jobs that will be output. The JobDistributor holds the option to distribute output to other nodes. More...

virtual std::list< JobResultID >	job_results_that_should_be_discarded ()=0
	The JobDistributor, to manage memory use, asks the JobQueen which JobResults may be discarded because they will not be used in the future. The JobDistributor will exit with an error message if the JobQueen gives it a LarvalJob that lists one of these discarded JobResults as a required input for that LarvalJob. More...

virtual output::ResultOutputterOP	result_outputter (output::OutputSpecification const &output_specification)=0
	The JobDistributor will ask the JobQueen to provide an outputter for a result giving it an annotated-with-job-distributor-specific-prefix OutputSpecification. The JobQueen must guarantee that a distinct ResultOutputter is given for each distinct JD prefix and that no two ResultOutputters share non-(bitwise-)constant data as the JobDistributor may use multiple threads to perform output. Note that the JobQueen that creates the outputter may not have been the JobQueen to request that output be performed. This function is thus similar to mature_larval_job: it requires that the JobQueen make few assumptions about the state of her data in order for this behavior to translate from the VanillaJobDistributor to the MPIWorkPoolJobDistributor. More...

virtual void	flush ()=0
	Send all buffered output to disk – called by the JobDistributor right before it shuts down if it hits an error or catches an exception that it cannot ignore. More...

virtual std::list < deallocation::DeallocationMessageOP >	deallocation_messages ()=0
	Are there any deallocation messages that the JobQueen would like to have delivered to the JobQueens living on remote hosts? If there are none, the derived JobQueen may return an empty list. The JobDistributor guarantees to send the deallocation message to all remote hosts. The JobDistributor will ask the JobQueen on the head node for a set of deallocation messages after each set of jobs from determine_job_list have been distributed. (i.e., before the next call to determine_job_list is made). More...

virtual void	process_deallocation_message (deallocation::DeallocationMessageOP message)=0
	A deallocation message first sent to the JobDistributor on this host originating from a remote JobQueen. More...

Protected Member Functions
virtual JobDigraphOP	create_initial_job_dag ()=0
	The JobQueen creates a directed acyclic graph (DAG) describing the sets of all jobs and the interdependencies between them to the JobDistributor. The JobDistributor will allow the JobQueen to update the DAG by adding new nodes and adding edges to new nodes if the JobQueen does not know up front how many nodes will be in the graph. More...

JobDigraph const &	get_job_graph () const
	Get read access to the current job graph. More...

JobTracker &	get_job_tracker ()
	Get write access to the current job tracker. More...

JobTracker const &	get_job_tracker () const
	Get read access to the current job tracker. More...

Private Attributes
JobDigraphCOP	job_graph_ = nullptr

JobTrackerOP	job_tracker_ = nullptr

Detailed Description

The JobQueen class (think of a queen bee) has quite a few responsibilities:

determining what jobs should be run and representing them as larval job objects
taking a larval job object and maturing it into a fully functional Job that can be run
processing the JobResult produced by a Job when it runs
determining the number of rounds of larval-job creation and job execution that should be performed.

The JobQueen is responsible for creating all of the LarvalJobs, grouped into "batches" which are dispatched by the JobDistributor; the queen then matures the larval jobs into the mature form (a Job, e.g. a Mover/Pose pair) and gives them up to the JobDistributor to actually execute the work. The JobDistributor then actually calls the Job's "run" method, which produces a JobResult. It then hands this JobResult (e.g. a Pose or several Poses) back to the JobQueen for output. In fact, in MPI contexts, the JobDistributor guarantees that it will hand all of the completed JobResults for a single batch to a single JobQueen, so that she can compute aggregate statistics (e.g. averaging energies) or can look at the results from that round of jobs and decide which to carry forward into the next round of execution.

Constructor & Destructor Documentation

protocols::jd3::JobQueen::JobQueen ( )

References job_tracker_.

protocols::jd3::JobQueen::~JobQueen ( )

overridedefault

Member Function Documentation

virtual void protocols::jd3::JobQueen::completed_job_summary	(	LarvalJobCOP	job,
		ResultIndex	result_index,
		JobSummaryOP	summary
	)

pure virtual

The JobDistributor guarnatees that exactly one JobQueen will see every JobSummary generated within a Job batch. This guarantee allows the JobQueen to aggregate data across all of the Jobs so that Rosetta is able to compute data from structures instead of forcing that computation into accessory scripts.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

JobDigraphOP protocols::jd3::JobQueen::create_and_set_initial_job_dag ( )

Creates the initial job dag using create_initial_job_dag function and sets it to the JQ.

Decorator to create_initial_job_dag

References create_initial_job_dag(), and job_graph_.

Referenced by protocols::multistage_rosetta_scripts::MRSJobQueen::complete_larval_job_maturation().

virtual JobDigraphOP protocols::jd3::JobQueen::create_initial_job_dag ( )

protectedpure virtual

The JobQueen creates a directed acyclic graph (DAG) describing the sets of all jobs and the interdependencies between them to the JobDistributor. The JobDistributor will allow the JobQueen to update the DAG by adding new nodes and adding edges to new nodes if the JobQueen does not know up front how many nodes will be in the graph.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, protocols::jd3::full_model::FullModelJobQueen, protocols::rosetta_scripts::RosettaScriptsJobQueen, and protocols::multistage_rosetta_scripts::MRSJobQueenChecker.

Referenced by create_and_set_initial_job_dag().

virtual std::list< deallocation::DeallocationMessageOP > protocols::jd3::JobQueen::deallocation_messages ( )

pure virtual

Are there any deallocation messages that the JobQueen would like to have delivered to the JobQueens living on remote hosts? If there are none, the derived JobQueen may return an empty list. The JobDistributor guarantees to send the deallocation message to all remote hosts. The JobDistributor will ask the JobQueen on the head node for a set of deallocation messages after each set of jobs from determine_job_list have been distributed. (i.e., before the next call to determine_job_list is made).

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, protocols::jd3::full_model::FullModelJobQueen, and protocols::rosetta_scripts::RosettaScriptsJobQueen.

virtual LarvalJobs protocols::jd3::JobQueen::determine_job_list	(	JobDAGNodeID	job_dag_node_index,
		core::Size	max_njobs
	)

pure virtual

The JobDistributor asks the JobQueen for a list of jobs that should be performed. This function neither knows nor cares what jobs are already complete on disk/memory.

it just figures out what ones should exist given the input. The JobDistributor tells the JobQueen which node in the JobDAG it is requesting jobs for, and puts a limit on the number of LarvalJobs that the JobQueen should return. The JobDistributor will call this function repeatedly for a single node until the JobQueen returns an empty list, at which point, the JobDistributor will consider the node's jobs exhausted.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, protocols::jd3::full_model::FullModelJobQueen, and protocols::multistage_rosetta_scripts::MRSJobQueenChecker.

Referenced by determine_job_list_and_track().

LarvalJobs protocols::jd3::JobQueen::determine_job_list_and_track	(	JobDAGNodeID	job_dag_node_index,
		core::Size	max_njobs
	)

Call determine_job_list in derived classes and track the jobs using the JobTracker.

Decorator to determine_job_list

References determine_job_list(), and job_tracker_.

virtual void protocols::jd3::JobQueen::flush ( )

pure virtual

Send all buffered output to disk – called by the JobDistributor right before it shuts down if it hits an error or catches an exception that it cannot ignore.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

JobDigraph const & protocols::jd3::JobQueen::get_job_graph ( ) const

protected

Get read access to the current job graph.

References job_graph_.

JobTracker & protocols::jd3::JobQueen::get_job_tracker ( )

protected

Get write access to the current job tracker.

References job_tracker_.

Referenced by protocols::jd3::standard::StandardJobQueen::current_job_index(), protocols::multistage_rosetta_scripts::MRSJobQueen::determine_job_list(), and protocols::jd3::standard::StandardJobQueen::expand_job_list().

JobTracker const & protocols::jd3::JobQueen::get_job_tracker ( ) const

protected

Get read access to the current job tracker.

References job_tracker_.

virtual bool protocols::jd3::JobQueen::has_job_previously_been_output ( LarvalJobCOP job )

pure virtual

The JobQueen must be able to determine if a particular job has already been output. The JobQueen returns "true" if the job has already been written, and "false" otherwise. Used for overwrite behavior.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual std::string protocols::jd3::JobQueen::job_definition_xsd ( ) const

pure virtual

All JobQueens must describe their job input XML format in the form of an XSD (XML Schema Definition), and they must validate their job input files against their XSDs. If the JobDistributor is awakened with the flag "jd3::output_job_xsd <output file>" on the command line, then the JobDistributor will write out the XSD to the output file.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual std::list< JobResultID > protocols::jd3::JobQueen::job_results_that_should_be_discarded ( )

pure virtual

The JobDistributor, to manage memory use, asks the JobQueen which JobResults may be discarded because they will not be used in the future. The JobDistributor will exit with an error message if the JobQueen gives it a LarvalJob that lists one of these discarded JobResults as a required input for that LarvalJob.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual std::list< output::OutputSpecificationOP > protocols::jd3::JobQueen::jobs_that_should_be_output ( )

pure virtual

The JobDistributor asks the JobQueen which JobResults should be queued for output. The JobQueen should reply by returning a list of OutputSpecification objects. Each OutputSpecification indicates a particular JobResult using the JobResultID object that the Specification holds. It also indicates how the JobResult should be output by whatever object will be doing to job outputting. The JobQueen should ask for a given JobResult to be output only once. After a job result is output, the JobDistributor will discard the JobResult, so the JobQueen should not tell the JobDistributor to output a job if it will be used as an input to another job in the future. Note that this method will only be called on the master-node JobQueen (aka JQ0), but that she will not necessarily be the JobQueen to actually return the ResultOutputter for the jobs that will be output. The JobDistributor holds the option to distribute output to other nodes.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual JobOP protocols::jd3::JobQueen::mature_larval_job	(	LarvalJobCOP	job,
		utility::vector1< JobResultCOP > const &	input_job_results
	)

pure virtual

Mature the input larval job into a full fledged job that will be run within this process (i.e. on this CPU), and thus can hold pointers to data that may persist inside the ResourceManager or may even be used by another process running in this thread. To pull that off, the Job object must share no non-bitwise-const data with any other Job object. That means if the Job were to contain a Pose and a Mover (as the MoverJob does), then so each Mover should be freshly constructed, and if the Pose had been copied from an existing Pose, must use the Poses "deep_copy" method (since Pose's sometimes share non-constant data between them, e.g. the AtomTree observer system, and sometimes Constraints ). The JobResults vector, which is supplied by the Jobdistributor, has entries corresponding to the JobResults from JobIDs in its input_job_result_indices vector.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual void protocols::jd3::JobQueen::note_job_completed	(	LarvalJobCOP	job,
		JobStatus	status,
		core::Size	nresults
	)

pure virtual

The JobDistributor will call this function to inform the JobQueen that a job has "completed" – in the sense that it will not be run in the future. It does not guarantee that the job was run on this CPU or that it successfully completed anywhere. The JobStatus indicates whether the job completed or whether it failed. The nresults count lists how many JobSummary/JobResult pairs were produced by the completed job.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::multistage_rosetta_scripts::MRSJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

Referenced by note_job_completed_and_track().

void protocols::jd3::JobQueen::note_job_completed_and_track	(	LarvalJobCOP	job,
		JobStatus	status,
		core::Size	nresults
	)

Call note_job_completed in derived classes and track the job using the JobTracker.

Decorator to note_job_completed

References job_tracker_, and note_job_completed().

virtual void protocols::jd3::JobQueen::process_deallocation_message ( deallocation::DeallocationMessageOP message )

pure virtual

A deallocation message first sent to the JobDistributor on this host originating from a remote JobQueen.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual std::string protocols::jd3::JobQueen::resource_definition_xsd ( ) const

pure virtual

JobQueens may optionally define an XSD for their resource definition file which is fed to their resource manager (if they control one). If the JobDistributor is awakened with the flag "jd3::output_resource_xsd <output file>" on the command line, and if the derived JobQueen defines an XSD for resources she uses, then the JobDistributor will write out the resource definition XSD to the output file. The derived queen may return an empty string to indicate that no resources are definable for the resource manager.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual output::ResultOutputterOP protocols::jd3::JobQueen::result_outputter ( output::OutputSpecification const & output_specification )

pure virtual

The JobDistributor will ask the JobQueen to provide an outputter for a result giving it an annotated-with-job-distributor-specific-prefix OutputSpecification. The JobQueen must guarantee that a distinct ResultOutputter is given for each distinct JD prefix and that no two ResultOutputters share non-(bitwise-)constant data as the JobDistributor may use multiple threads to perform output. Note that the JobQueen that creates the outputter may not have been the JobQueen to request that output be performed. This function is thus similar to mature_larval_job: it requires that the JobQueen make few assumptions about the state of her data in order for this behavior to translate from the VanillaJobDistributor to the MPIWorkPoolJobDistributor.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

virtual void protocols::jd3::JobQueen::update_job_dag ( JobDigraphUpdater & updater )

pure virtual

The JobQueen is allowed to update the JobDigraph over the course of execution but only in a particular way: by adding new nodes, and then by adding edges that land on those new nodes. The JobQueen is not allowed to add edges that land on an old node.

Implemented in protocols::jd3::standard::StandardJobQueen, protocols::jd3::chunk_library::ChunkLibraryJobQueen, and protocols::jd3::full_model::FullModelJobQueen.

Member Data Documentation

JobDigraphCOP protocols::jd3::JobQueen::job_graph_ = nullptr

private

Referenced by create_and_set_initial_job_dag(), and get_job_graph().

JobTrackerOP protocols::jd3::JobQueen::job_tracker_ = nullptr

private

Referenced by determine_job_list_and_track(), get_job_tracker(), JobQueen(), and note_job_completed_and_track().

The documentation for this class was generated from the following files:

src/protocols/jd3/JobQueen.hh
src/protocols/jd3/JobQueen.cc

Public Member Functions

Protected Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation