 |
Rosetta
2021.16
|
All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
|
Rosetta
2021.16
|
The ChunkLibraryJobQueen is meant to handle the most common form of Rosetta jobs where a protocol is applied to a single input structure to generate a single output structure. Most JobQueens in Rosetta will to derive from this JobQueen. To make the process of deriving new JobQueens easy, this class's API has been carefully designed to be easy to work with and the class itself has been designed to do as much heavy lifting as possible and to leave only the responsibilities that the ChunkLibraryJobQueen could not perform to the derived classes. More...
#include <ChunkLibraryJobQueen.hh>
Public Member Functions | |
| ChunkLibraryJobQueen () | |
| The ChunkLibraryJobQueen constructor asks the ChunkLibraryInputterFactory for a ChunkLibraryInputter and creates a ResourceManager. More... | |
| ~ChunkLibraryJobQueen () override | |
| std::string | job_definition_xsd () const override |
| The ChunkLibraryJobQueen assembles the XSD from virtual functions she invokes on the derived JobQueen: append_job_tag_subelements, append_common_tag_subelements, and add_option/add_options. More... | |
| std::string | resource_definition_xsd () const override |
| JobQueens may optionally define an XSD for their resource definition file which is fed to their resource manager (if they control one). If the JobDistributor is awakened with the flag "jd3::output_resource_xsd <output file>" on the command line, and if the derived JobQueen defines an XSD for resources she uses, then the JobDistributor will write out the resource definition XSD to the output file. The derived queen may return an empty string to indicate that no resources are definable for the resource manager. More... | |
| JobDigraphOP | create_initial_job_dag () override |
| The ChunkLibraryJobQueen provides an implementation of this function which returns the most straight-forward DAG representing a set of jobs that have no interdependencies: a DAG with a single node and no edges. More... | |
| void | update_job_dag (JobDigraphUpdater &updater) override |
| The ChunkLibraryJobQueen's implementation is to not update the JobDAG at all: the most basic protocol defines a job DAG with only a single node. More... | |
| LarvalJobs | determine_job_list (JobDAGNodeID job_dag_node_index, core::Size max_njobs) override |
| The ChunkLibraryJobQueen manages the process of creating the list of LarvalJobs that will be later matured into actual jobs and run by the JobDistributor. Classes derived from the ChunkLibraryJobQueen need answer a few virtual functions that the ChunkLibraryJobQueen will invoke during this process. More... | |
| bool | has_job_previously_been_output (protocols::jd3::LarvalJobCOP job) override |
| protocols::jd3::JobOP | mature_larval_job (protocols::jd3::LarvalJobCOP job, utility::vector1< JobResultCOP > const &input_job_results) override |
| Mature the input larval job into a full fledged job that will be run within this process (i.e. on this CPU), and thus can hold pointers to data that may persist inside the ResourceManager or may even be used by another process running in this thread. To pull that off, the Job object must share no non-bitwise-const data with any other Job object. That means if the Job were to contain a Pose and a Mover (as the MoverJob does), then so each Mover should be freshly constructed, and if the Pose had been copied from an existing Pose, must use the Poses "deep_copy" method (since Pose's sometimes share non-constant data between them, e.g. the AtomTree observer system, and sometimes Constraints ). The JobResults vector, which is supplied by the Jobdistributor, has entries corresponding to the JobResults from JobIDs in its input_job_result_indices vector. More... | |
| void | note_job_completed (LarvalJobCOP job, JobStatus status, core::Size nresults) override |
| void | completed_job_summary (LarvalJobCOP job, ResultIndex result_index, JobSummaryOP summary) override |
| If the job result that is summarized here should be output, then the SJQ needs to have access to the LarvalJob in order to do that. More... | |
| std::list < output::OutputSpecificationOP > | jobs_that_should_be_output () override |
| The JobDistributor asks the JobQueen which JobResults should be queued for output. The JobQueen should reply by returning a list of OutputSpecification objects. Each OutputSpecification indicates a particular JobResult using the JobResultID object that the Specification holds. It also indicates how the JobResult should be output by whatever object will be doing to job outputting. The JobQueen should ask for a given JobResult to be output only once. After a job result is output, the JobDistributor will discard the JobResult, so the JobQueen should not tell the JobDistributor to output a job if it will be used as an input to another job in the future. Note that this method will only be called on the master-node JobQueen (aka JQ0), but that she will not necessarily be the JobQueen to actually return the ResultOutputter for the jobs that will be output. The JobDistributor holds the option to distribute output to other nodes. More... | |
| std::list< JobResultID > | job_results_that_should_be_discarded () override |
| output::ResultOutputterOP | result_outputter (output::OutputSpecification const &spec) override |
| Return the bag of of PoseOutputters (in the form of a MultipleOutputter) for the Pose that has been requested and specified by a particular OutputSpecification; this function guarantees that for each individual outputter-name (respecting the JD-provided filename suffix) that a separate set of PoseOutputters are returned so that multiple threads can potentially output at the same time. More... | |
| void | determine_preliminary_job_list_from_xml_file (std::string const &job_def_string) |
| Read from an input string representing the contents of the job-definiton XML file and construct a set of LarvalJobs; this function is primarily useful for testing, but could be used to organize jobs by an enterprising job distributor or by another JobQueen. More... | |
| void | flush () override |
| Send all buffered output to disk – called by the JobDistributor right before it shuts down if it hits an error or catches an exception that it cannot ignore. More... | |
| std::list < deallocation::DeallocationMessageOP > | deallocation_messages () override |
| Are there any deallocation messages that the JobQueen would like to have delivered to the JobQueens living on remote hosts? If there are none, the derived JobQueen may return an empty list. The JobDistributor guarantees to send the deallocation message to all remote hosts. The JobDistributor will ask the JobQueen on the head node for a set of deallocation messages after each set of jobs from determine_job_list have been distributed. (i.e., before the next call to determine_job_list is made). More... | |
| void | process_deallocation_message (deallocation::DeallocationMessageOP message) override |
| A deallocation message first sent to the JobDistributor on this host originating from a remote JobQueen. More... | |
 Public Member Functions inherited from protocols::jd3::JobQueen | |
| JobQueen () | |
| ~JobQueen () override | |
| JobDigraphOP | create_and_set_initial_job_dag () |
| Creates the initial job dag using create_initial_job_dag function and sets it to the JQ. More... | |
| void | note_job_completed_and_track (LarvalJobCOP job, JobStatus status, core::Size nresults) |
| Call note_job_completed in derived classes and track the job using the JobTracker. More... | |
| LarvalJobs | determine_job_list_and_track (JobDAGNodeID job_dag_node_index, core::Size max_njobs) |
| Call determine_job_list in derived classes and track the jobs using the JobTracker. More... | |
Static Public Member Functions | |
| static std::string | job_def_complex_type_name (std::string const &type) |
| Naming function for the complexTypes in the job-definition XSD. More... | |
Protected Member Functions | |
| void | do_not_accept_all_chunk_library_inputters_from_factory () |
| The derived job queen (DJQ) may tell the ChunkLibraryJobQueen to only accept a subset of ChunkLibraryInputters. First the DJQ says "do_not_accept_all_chunk_library_inputters_from_factory" and then she can call "allow_chunk_library_inputter." This function must only be called the DJQ's constructor. More... | |
| void | allow_chunk_library_inputter (chunk_library_inputters::ChunkLibraryInputterCreatorOP creator) |
| If the derived job queen (DJQ) wants to allow a ChunkLibraryInputter to be used either because it is not registered with the ChunkLibraryInputterFactory, or because she wants to allow only a subset of the inputters that are registered with the Factory, then she may indicate that to the ChunkLibraryJobQueen with this function. If the derived job queen does not want to use all of the pose inputters provided by the factory, then she must call "do_not_accept_all_chunk_library_inputters_from_factory()" before she calls this function. This function must only be called in the DJQ's constructor. All DJQs must invoke this method in their constructors and not simply the DJQ that lives on the "head node" in order for distributed output to workk correctly. More... | |
| void | do_not_accept_all_pose_outputters_from_factory () |
| The derived job queen (DJQ) may tell the ChunkLibraryJobQueen to only accept a subset of PoseOutputters. First the DJQ says "do_not_accept_all_pose_outputters_from_factory" and then she can call "allow_pose_outputter." This function must only be called the DJQ's constructor. More... | |
| void | allow_pose_outputter (pose_outputters::PoseOutputterCreatorOP creator) |
| If the derived job queen (DJQ) wants to allow a PoseOutputter to be used either because it is not registered with the PoseOutputterFactory, or because she wants to allow only a subset of the outputters that are registered with the Factory, then she may indicate that to the ChunkLibraryJobQueen with this function. If the derived job queen does not want to use all of the pose outputters provided by the factory, then she must call "do_not_accept_all_pose_outputters_from_factory()" before she calls this function. If the DJQ has called the "do_not_accept_all_pose_outputters_from_factory" function, then she specifies the default outputter on the first time she calls "allow_pose_outputter." This function must only be called in the DJQ's constructor. More... | |
| void | set_default_outputter (pose_outputters::PoseOutputterCreatorOP creator) |
| If the derived job queen wants the user to get a particular PoseOutputter by default (i.e. when the user hasn't specified which outputter to use in a job definition file and in the absence of a command-line flag that says which outputter to use) instead of the default PoseOutputter chosen by the PoseOutputterFactory, then the DJQ should call this function. If the DJQ has previously called "do_not_accept_all_pose_outputters_from_factory", then the first call to "allow_pose_outputter" specified the default outputter, and a separate call to this function is not absolutely necessary. This function must only be called in the DJQ's constructor. More... | |
| virtual void | append_job_tag_subelements (utility::tag::XMLSchemaDefinition &job_definition_xsd, utility::tag::XMLSchemaComplexTypeGenerator &ct_gen) const |
| The derived JobQueen must inform the ChunkLibraryJobQueen of any additional tags that belong as sub elements of the <Job> tag. The ChunkLibraryJobQueen will have already appended the <Input>, <Output>, and <Options> subtags before this function executes, so the only functions that the derived class should invoke of the XMLSchemaComplexTypeGenerator are its add_ordered_subelement_set_* functions, if you have additional subelements to add. Adding additional subelements to the complexType generator may require writing additional complexTypes to the XML Schema for the job definition file, and so that job definition xsd is passed in as well. More... | |
| virtual void | append_common_tag_subelements (utility::tag::XMLSchemaDefinition &xsd, utility::tag::XMLSchemaComplexTypeGenerator &ct_gen) const |
| The derived JobQueen must inform the ChunkLibraryJobQueen of any additional tags that belong as sub elements of the <Common> tag. The ChunkLibraryJobQueen will have already appended the <Options> subtag before this function executes, so the only functions that the derived class should invoke of the XMLSchemaComplexTypeGenerator are its add_ordered_subelement_set_* functions, if you have additional subelements to add. Adding additional subelements to the complexType generator may require writing additional complexTypes to the XML Schema for the job definition file, and so that job definition xsd is passed in as well. The Tags in the <Common> block should be parsed for each Job in the mature_larval_job step; these tags are available to the derived class through the common_block_tags method. This is where data that is not constant over the course of the job can be declared (e.g. ScoreFunctions, the weights of which are often modified during the course of protocol execution), but data that is constant and that can be shared between multiple jobs ought to be initialized and held by the ResourceManager. Options provided in the <Common> block will supercede the command line, but will be overriden by any options provided for a particular job. (This is logic the ChunkLibraryJobQueen manages). More... | |
| virtual void | determine_preliminary_job_list () |
| Allow the derived JobQueen to tell the ChunkLibraryJobQueen to initialize the preliminary job list; this is perhaps necessary in the context of multi-round protocols when job-definition file specifies the JobDAG. More... | |
| utility::vector1< core::Size > const & | preliminary_job_nodes () const |
| Read access for the subset of nodes in the job DAG which the ChunkLibraryJobQueen is responsible for producing the larval_jobs. They are called "preliminary" jobs because they do not depend on outputs from any previous node in the graph. (The set of job nodes that contain no incoming edges, though, could perhaps be different from the set of preliminary job nodes, so the ChunkLibraryJobQueen requires that the deried job queen inform her of which nodes are the preliminary job nodes.) More... | |
| bool | all_jobs_assigned_for_preliminary_job_node (core::Size node_id) const |
| Returns true if all of the jobs for the given input preliminary-job node have been created and returned in a call to determine_job_list. Even if all jobs have been created, not all jobs have necessarily been completed. More... | |
| utility::vector1 < ChunkLibraryPreliminaryLarvalJob > const & | get_preliminary_larval_jobs () const |
| Read access to derived JobQueens to the preliminary job list. This will return an empty list if determine_preliminary_jobs has not yet been called. More... | |
| virtual ChunkLibraryInnerLarvalJobs | refine_preliminary_job (ChunkLibraryPreliminaryLarvalJob const &prelim_job) |
| Ask the derived JobQueen to expand / refine a preliminary larval job, by possibly reading per-job data out of the Tag associated with each job. If there is nothing that needs to be done by the derived class, it may elect to use the base-class implementation of this function which simply returns a list of the inner-job pointers in the input list. Why all the back and forth? So that the derived job queens can take one input structure and multiply it into several sepearate jobs; e.g. in a ddG protocol, you need to run a separate set of jobs for the WT sequence and also for the mutant sequence. The preliminary job will reflect the structure of the WT, and then that will get expanded out. This funciton is non-const so that the derived class can even track which WT sequences it has seen so that after the first WT sequence for a particular input structure, it only expands out the mutant sequences if many mutants on that input. More... | |
| virtual LarvalJobs | expand_job_list (ChunkLibraryInnerLarvalJobOP inner_job, core::Size max_larval_jobs_to_create) |
| Expand a ChunkLibraryInnerLarvalJob into a full set of LarvalJobs, creating nstruct LarvalJob objects. More... | |
| virtual JobOP | complete_larval_job_maturation (protocols::jd3::LarvalJobCOP larval_job, utility::options::OptionCollectionCOP job_options, utility::vector1< JobResultCOP > const &input_job_results)=0 |
| The derived JobQueen must define the method that takes a larval job and the job-specific options and matures the larval job into a full job. More... | |
| void | create_and_store_output_specification_for_job_result (LarvalJobCOP job, ResultIndex result_index, core::Size nresults) |
| Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request. This function invokes the factory method create_output_specification_for_job_result which derive job queens may override. This version of the function will create an OptionCollection and then invoke the version of the function that expects one. More... | |
| void | create_and_store_output_specification_for_job_result (LarvalJobCOP job, utility::options::OptionCollection const &job_options, ResultIndex result_index, core::Size nresults) |
| Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request. This function invokes the factory method create_output_specification_for_job_result which derive job queens may override. This version of the function accepts an OptionCollection if there are multiple results for a particular job, and you can reuse the OptionCollection between calls. More... | |
| virtual output::OutputSpecificationOP | create_output_specification_for_job_result (LarvalJobCOP job, utility::options::OptionCollection const &job_options, ResultIndex result_index, core::Size nresults) |
| Construct the OutputSpecification for a completed successful job. Factory method that may be overriden by the derived job queen. Invoked by create_and_store_output_specification_for_job_result which is invoked via note_job_completed by default, and may be invoked by the derived job queen as needed. More... | |
| JobOutputIndex | build_output_index (protocols::jd3::LarvalJobCOP larval_job, ResultIndex result_index_for_job, core::Size n_results_for_job) |
| Construct a JobOutputIndex for a given job based on "the obvious", but giving derived classes the chance to assign their own indices via the assign_output_index function. More... | |
| virtual void | assign_output_index (protocols::jd3::LarvalJobCOP larval_job, ResultIndex result_index_for_job, core::Size n_results_for_job, JobOutputIndex &output_index) |
| The derived job queen may assign her own numbering to output Poses if she chooses. More... | |
| virtual void | derived_process_deallocation_message (deallocation::DeallocationMessageOP message) |
| If the derived class has deallocation messages that it needs to broadcast to remote nodes and then to process on those remote nodes, then the StanardJobQueen will pass them along to the derived job queen using this function. More... | |
| void | add_options (utility::options::OptionKeyList const &opts) |
| void | add_option (utility::options::OptionKey const &key) |
| void | remove_default_input_element () |
| Override the basic specification of input elements as either PDBs, Silent Structures, or ResourceManager defined structures. This should be called during the derived class's constructor if it is to be used. More... | |
| utility::tag::TagCOP | common_block_tags () const |
| Return the set of subtags that are common to the whole set of jobs in the Job definition file, if any are given. This set of tags is read from disk at most once per execution. More... | |
| core::pose::PoseOP | pose_for_job (LarvalJobCOP job, utility::options::OptionCollection const &options) |
| Return a copy of the Pose to be used with the given job. More... | |
| chunk_library_inputters::ChunkLibraryInputterOP | chunk_library_inputter_for_job (ChunkLibraryInnerLarvalJob const &inner_job) const |
| Access the pose inputter. More... | |
| pose_outputters::PoseOutputterOP | pose_outputter_for_job (ChunkLibraryInnerLarvalJob const &innerJob) |
| Access the pose outputter for a particular job; perhaps this outputter has already been created, or perhaps it needs to be created and stored. More... | |
| pose_outputters::PoseOutputterOP | pose_outputter_for_job (ChunkLibraryInnerLarvalJob const &innerJob, utility::options::OptionCollection const &job_options) |
| pose_outputters::PoseOutputterOP | pose_outputter_for_job (pose_outputters::PoseOutputSpecification const &spec) |
| Find the PoseOutputter for a job specification, respecting the Job-distributor provided output file suffix. The ChunkLibraryJobQueen guarantees that a different Outputter is used for each unique name as long as that name is not the empty string. This allows multiple threads to write to separate files concurrently. More... | |
| utility::vector1 < pose_outputters::SecondaryPoseOutputterOP > | secondary_outputters_for_job (ChunkLibraryInnerLarvalJob const &innerJob, utility::options::OptionCollection const &job_options) |
| core::Size | nstruct_for_job (utility::tag::TagCOP job_tag) const |
| utility::options::OptionCollectionOP | options_for_job (ChunkLibraryInnerLarvalJob const &inner_job) const |
| utility::options::OptionCollectionOP | options_from_tag (utility::tag::TagCOP job_options_tags) const |
| Protected Member Functions inherited from protocols::jd3::JobQueen | |
| JobDigraph const & | get_job_graph () const |
| Get read access to the current job graph. More... | |
| JobTracker & | get_job_tracker () |
| Get write access to the current job tracker. More... | |
| JobTracker const & | get_job_tracker () const |
| Get read access to the current job tracker. More... | |
Private Types | |
| typedef std::map< std::string, pose_outputters::PoseOutputterOP > | RepresentativeOutputterMap |
| typedef std::map< std::string, pose_outputters::SecondaryPoseOutputterOP > | SecondaryRepresentativeOutputterMap |
| typedef std::map< std::pair < std::string, std::string > , pose_outputters::PoseOutputterOP > | PoseOutputterMap |
| typedef std::map< std::pair < std::string, std::string > , pose_outputters::SecondaryPoseOutputterOP > | SecondaryOutputterMap |
Private Member Functions | |
| void | determine_preliminary_job_list_from_xml_file (std::string const &job_def_string, std::string const &job_def_schema) |
| After generating the job-definition XSD, construct the preliminary job list. This is invoked both from determine_job_list_from_xml_file and determine_job_list – the latter always constructs an XSD to ensure that the derived JobQueen has properly constructed an XSD, even if a job definition file has not been provided on the command line. More... | |
| void | load_job_definition_file (std::string const &job_def_string, std::string const &job_def_schema) |
| void | determine_preliminary_job_list_from_command_line () |
| Instead of reading a JobDefinition file, construct the set of ChunkLibraryPreliminaryLarvalJobs reading from the command line. Invoked by determine_preliminary_job_list. More... | |
| LarvalJobs | next_batch_of_larval_jobs_from_prelim (core::Size job_node_index, core::Size max_njobs) |
| pose_outputters::SecondaryPoseOutputterOP | secondary_outputter_for_job (ChunkLibraryInnerLarvalJob const &inner_job, utility::options::OptionCollection const &job_options, std::string const &secondary_outputter_type) |
| pose_outputters::SecondaryPoseOutputterOP | secondary_outputter_for_job (pose_outputters::PoseOutputSpecification const &) |
| void | note_job_result_discarded (LarvalJobCOP job, ResultIndex result_index) |
| The SJQ will keep track of all discarded jobs in the discarded_jobs_ diet. More... | |
| utility::options::OptionKeyList | concatenate_all_options () const |
| pose_outputters::PoseOutputterOP | get_outputter_from_job_tag (utility::tag::TagCOP tag) const |
The ChunkLibraryJobQueen is meant to handle the most common form of Rosetta jobs where a protocol is applied to a single input structure to generate a single output structure. Most JobQueens in Rosetta will to derive from this JobQueen. To make the process of deriving new JobQueens easy, this class's API has been carefully designed to be easy to work with and the class itself has been designed to do as much heavy lifting as possible and to leave only the responsibilities that the ChunkLibraryJobQueen could not perform to the derived classes.
|
private |
|
private |
|
private |
|
private |
| protocols::jd3::chunk_library::ChunkLibraryJobQueen::ChunkLibraryJobQueen | ( | ) |
The ChunkLibraryJobQueen constructor asks the ChunkLibraryInputterFactory for a ChunkLibraryInputter and creates a ResourceManager.
References allow_chunk_library_inputter(), allow_pose_outputter(), do_not_accept_all_chunk_library_inputters_from_factory(), do_not_accept_all_pose_outputters_from_factory(), inputter_options_, outputter_options_, and secondary_outputter_options_.
|
overridedefault |
|
protected |
|
protected |
|
protected |
Returns true if all of the jobs for the given input preliminary-job node have been created and returned in a call to determine_job_list. Even if all jobs have been created, not all jobs have necessarily been completed.
|
protected |
If the derived job queen (DJQ) wants to allow a ChunkLibraryInputter to be used either because it is not registered with the ChunkLibraryInputterFactory, or because she wants to allow only a subset of the inputters that are registered with the Factory, then she may indicate that to the ChunkLibraryJobQueen with this function. If the derived job queen does not want to use all of the pose inputters provided by the factory, then she must call "do_not_accept_all_chunk_library_inputters_from_factory()" before she calls this function. This function must only be called in the DJQ's constructor. All DJQs must invoke this method in their constructors and not simply the DJQ that lives on the "head node" in order for distributed output to workk correctly.
Referenced by ChunkLibraryJobQueen().
|
protected |
If the derived job queen (DJQ) wants to allow a PoseOutputter to be used either because it is not registered with the PoseOutputterFactory, or because she wants to allow only a subset of the outputters that are registered with the Factory, then she may indicate that to the ChunkLibraryJobQueen with this function. If the derived job queen does not want to use all of the pose outputters provided by the factory, then she must call "do_not_accept_all_pose_outputters_from_factory()" before she calls this function. If the DJQ has called the "do_not_accept_all_pose_outputters_from_factory" function, then she specifies the default outputter on the first time she calls "allow_pose_outputter." This function must only be called in the DJQ's constructor.
Referenced by ChunkLibraryJobQueen().
|
protectedvirtual |
The derived JobQueen must inform the ChunkLibraryJobQueen of any additional tags that belong as sub elements of the <Common> tag. The ChunkLibraryJobQueen will have already appended the <Options> subtag before this function executes, so the only functions that the derived class should invoke of the XMLSchemaComplexTypeGenerator are its add_ordered_subelement_set_* functions, if you have additional subelements to add. Adding additional subelements to the complexType generator may require writing additional complexTypes to the XML Schema for the job definition file, and so that job definition xsd is passed in as well. The Tags in the <Common> block should be parsed for each Job in the mature_larval_job step; these tags are available to the derived class through the common_block_tags method. This is where data that is not constant over the course of the job can be declared (e.g. ScoreFunctions, the weights of which are often modified during the course of protocol execution), but data that is constant and that can be shared between multiple jobs ought to be initialized and held by the ResourceManager. Options provided in the <Common> block will supercede the command line, but will be overriden by any options provided for a particular job. (This is logic the ChunkLibraryJobQueen manages).
Derived classes may choose to not override this method as a way to indicate that they have no common-block data that needs to be defined.
|
protectedvirtual |
The derived JobQueen must inform the ChunkLibraryJobQueen of any additional tags that belong as sub elements of the <Job> tag. The ChunkLibraryJobQueen will have already appended the <Input>, <Output>, and <Options> subtags before this function executes, so the only functions that the derived class should invoke of the XMLSchemaComplexTypeGenerator are its add_ordered_subelement_set_* functions, if you have additional subelements to add. Adding additional subelements to the complexType generator may require writing additional complexTypes to the XML Schema for the job definition file, and so that job definition xsd is passed in as well.
Derived classes may choose to not override this method as a way to indicate that they have no additional subtags of the <Job> tag they wish to add. This is a no-op implementation.
|
protectedvirtual |
The derived job queen may assign her own numbering to output Poses if she chooses.
No-op implementation – leave the indices the way they were initialized.
Just before a job result is written to disk, the ChunkLibraryJobQueen asks the derived job queen what indices should be used to identify the soon-to-be-output Pose. The default behavior is to use the nstruct index for the primary output index, and if there are multiple result Poses from a job, to use the result index (as is) for the secondary output index. These values will already have been loaded into the output_index before this function is called. This function will possibly be invoked more than a single time for a single job result, so it is important that the derived JobQueen not assume that it only happens once. This function will only be called on JQ0; it will not be called on any other JQs, so it is welcome to use information that would only be known on the "head node"
|
protected |
Construct a JobOutputIndex for a given job based on "the obvious", but giving derived classes the chance to assign their own indices via the assign_output_index function.
References protocols::mean_field::max(), protocols::jd3::JobOutputIndex::n_primary_outputs, protocols::jd3::JobOutputIndex::n_secondary_outputs, protocols::jd3::JobOutputIndex::primary_output_index, and protocols::jd3::JobOutputIndex::secondary_output_index.
|
protected |
Access the pose inputter.
References protocols::jd3::InnerLarvalJob::input_source(), and protocols::jd3::InnerLarvalJob::job_node().
|
protected |
Return the set of subtags that are common to the whole set of jobs in the Job definition file, if any are given. This set of tags is read from disk at most once per execution.
|
protectedpure virtual |
The derived JobQueen must define the method that takes a larval job and the job-specific options and matures the larval job into a full job.
|
overridevirtual |
If the job result that is summarized here should be output, then the SJQ needs to have access to the LarvalJob in order to do that.
Implements protocols::jd3::JobQueen.
|
private |
|
protected |
Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request. This function invokes the factory method create_output_specification_for_job_result which derive job queens may override. This version of the function will create an OptionCollection and then invoke the version of the function that expects one.
References protocols::jd3::chunk_library::bad_inner_job_exception(), and protocols::jd3::options_for_job().
|
protected |
Create an output specification for a given job + result index and store it in the recent_successes_ list, which will be given to the JobDistributor upon request. This function invokes the factory method create_output_specification_for_job_result which derive job queens may override. This version of the function accepts an OptionCollection if there are multiple results for a particular job, and you can reuse the OptionCollection between calls.
|
overridevirtual |
The ChunkLibraryJobQueen provides an implementation of this function which returns the most straight-forward DAG representing a set of jobs that have no interdependencies: a DAG with a single node and no edges.
The base class provides a digraph with a single node – that is, all the jobs are independent of each other. This is equivalent to the kind of jobs that could be run in JD2.
Implements protocols::jd3::JobQueen.
|
protectedvirtual |
Construct the OutputSpecification for a completed successful job. Factory method that may be overriden by the derived job queen. Invoked by create_and_store_output_specification_for_job_result which is invoked via note_job_completed by default, and may be invoked by the derived job queen as needed.
Creates a MultipleOutputSpecification and packs it with one OutputSpecification per PoseOutputter / SecondaryPoseOutputter. The order of the OutputSpecifications that are given here needs to be recapitulated for the PoseMultipleOutputter that will be created by the call to result_outputter, as the OutputSpecifiations are iterated across in the same order as the PoseOutputters in the PoseMultipleOutputter.
References protocols::jd3::chunk_library::bad_inner_job_exception().
|
overridevirtual |
Are there any deallocation messages that the JobQueen would like to have delivered to the JobQueens living on remote hosts? If there are none, the derived JobQueen may return an empty list. The JobDistributor guarantees to send the deallocation message to all remote hosts. The JobDistributor will ask the JobQueen on the head node for a set of deallocation messages after each set of jobs from determine_job_list have been distributed. (i.e., before the next call to determine_job_list is made).
Implements protocols::jd3::JobQueen.
|
protectedvirtual |
If the derived class has deallocation messages that it needs to broadcast to remote nodes and then to process on those remote nodes, then the StanardJobQueen will pass them along to the derived job queen using this function.
|
overridevirtual |
The ChunkLibraryJobQueen manages the process of creating the list of LarvalJobs that will be later matured into actual jobs and run by the JobDistributor. Classes derived from the ChunkLibraryJobQueen need answer a few virtual functions that the ChunkLibraryJobQueen will invoke during this process.
The process begins by first constructing the job definition and resource definition XSDs. With these schemas, the ChunkLibraryJobQueen validates the input XML files (if present). The ChunkLibraryJobQueen then populates preliminary versions of LarvalJob objects./ If the XSD includes "command line options" (which may be specified either from the command line or in the <options> sections of the Job XML file), the ChunkLibraryJobQueen loads the preliminary LarvalJob objects with the options. These preliminary LarvalJob objects will not have been nstruct expanded (i.e. if there are 100 nstruct for each of 5 different jobs, then there will only be 5 preliminary larval jobs created). It then passes the preliminary LarvalJob list and the TagOP objects for each preliminary LarvalJob to the derived class through the refine_job_list method.
Implements protocols::jd3::JobQueen.
|
protectedvirtual |
Allow the derived JobQueen to tell the ChunkLibraryJobQueen to initialize the preliminary job list; this is perhaps necessary in the context of multi-round protocols when job-definition file specifies the JobDAG.
|
private |
Instead of reading a JobDefinition file, construct the set of ChunkLibraryPreliminaryLarvalJobs reading from the command line. Invoked by determine_preliminary_job_list.
References protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::chunk_library_inputter, protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::inner_job, protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::job_options, protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::job_tag, protocols::jd3::nstruct_for_job(), and protocols::jd3::options_from_tag().
| void protocols::jd3::chunk_library::ChunkLibraryJobQueen::determine_preliminary_job_list_from_xml_file | ( | std::string const & | job_def_string | ) |
Read from an input string representing the contents of the job-definiton XML file and construct a set of LarvalJobs; this function is primarily useful for testing, but could be used to organize jobs by an enterprising job distributor or by another JobQueen.
Construct the XSD and then invoke the (private) determine_preliminary_job_list_from_xml_file method, which is also invoked by determine_preliminary_job_list.
|
private |
After generating the job-definition XSD, construct the preliminary job list. This is invoked both from determine_job_list_from_xml_file and determine_job_list – the latter always constructs an XSD to ensure that the derived JobQueen has properly constructed an XSD, even if a job definition file has not been provided on the command line.
References protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::chunk_library_inputter, protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::inner_job, protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::job_options, protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::job_tag, protocols::jd3::nstruct_for_job(), and protocols::jd3::options_from_tag().
|
protected |
The derived job queen (DJQ) may tell the ChunkLibraryJobQueen to only accept a subset of ChunkLibraryInputters. First the DJQ says "do_not_accept_all_chunk_library_inputters_from_factory" and then she can call "allow_chunk_library_inputter." This function must only be called the DJQ's constructor.
Referenced by ChunkLibraryJobQueen().
|
protected |
The derived job queen (DJQ) may tell the ChunkLibraryJobQueen to only accept a subset of PoseOutputters. First the DJQ says "do_not_accept_all_pose_outputters_from_factory" and then she can call "allow_pose_outputter." This function must only be called the DJQ's constructor.
Referenced by ChunkLibraryJobQueen().
|
protectedvirtual |
Expand a ChunkLibraryInnerLarvalJob into a full set of LarvalJobs, creating nstruct LarvalJob objects.
References protocols::mean_field::min().
|
overridevirtual |
Send all buffered output to disk – called by the JobDistributor right before it shuts down if it hits an error or catches an exception that it cannot ignore.
Implements protocols::jd3::JobQueen.
|
private |
|
protected |
Read access to derived JobQueens to the preliminary job list. This will return an empty list if determine_preliminary_jobs has not yet been called.
|
overridevirtual |
The JobQueen must be able to determine if a particular job has already been output. The JobQueen returns "true" if the job has already been written, and "false" otherwise. Used for overwrite behavior.
Implements protocols::jd3::JobQueen.
References protocols::jd3::chunk_library::bad_inner_job_exception(), and protocols::jd3::options_for_job().
|
static |
Naming function for the complexTypes in the job-definition XSD.
|
overridevirtual |
The ChunkLibraryJobQueen assembles the XSD from virtual functions she invokes on the derived JobQueen: append_job_tag_subelements, append_common_tag_subelements, and add_option/add_options.
Implements protocols::jd3::JobQueen.
References protocols::jd3::chunk_library_inputters::ChunkLibraryInputterFactory::chunk_library_inputter_xml_schema_group_name(), protocols::jd3::chunk_library_inputters::ChunkLibraryInputterFactory::complex_type_name_for_chunk_library_inputter(), protocols::jd3::pose_outputters::PoseOutputterFactory::complex_type_name_for_pose_outputter(), protocols::jd3::chunk_library::option_type_from_key(), protocols::jd3::pose_outputters::PoseOutputterFactory::pose_outputter_xml_schema_group_name(), protocols::jd3::pose_outputters::PoseOutputterFactory::secondary_pose_outputter_xml_schema_group_name(), and protocols::jd3::chunk_library::value_attribute_type_for_option().
|
overridevirtual |
Default implementation does not discard any job results.
Implements protocols::jd3::JobQueen.
|
overridevirtual |
The JobDistributor asks the JobQueen which JobResults should be queued for output. The JobQueen should reply by returning a list of OutputSpecification objects. Each OutputSpecification indicates a particular JobResult using the JobResultID object that the Specification holds. It also indicates how the JobResult should be output by whatever object will be doing to job outputting. The JobQueen should ask for a given JobResult to be output only once. After a job result is output, the JobDistributor will discard the JobResult, so the JobQueen should not tell the JobDistributor to output a job if it will be used as an input to another job in the future. Note that this method will only be called on the master-node JobQueen (aka JQ0), but that she will not necessarily be the JobQueen to actually return the ResultOutputter for the jobs that will be output. The JobDistributor holds the option to distribute output to other nodes.
Implements protocols::jd3::JobQueen.
|
private |
References protocols::jd3::TR().
|
overridevirtual |
Mature the input larval job into a full fledged job that will be run within this process (i.e. on this CPU), and thus can hold pointers to data that may persist inside the ResourceManager or may even be used by another process running in this thread. To pull that off, the Job object must share no non-bitwise-const data with any other Job object. That means if the Job were to contain a Pose and a Mover (as the MoverJob does), then so each Mover should be freshly constructed, and if the Pose had been copied from an existing Pose, must use the Poses "deep_copy" method (since Pose's sometimes share non-constant data between them, e.g. the AtomTree observer system, and sometimes Constraints ). The JobResults vector, which is supplied by the Jobdistributor, has entries corresponding to the JobResults from JobIDs in its input_job_result_indices vector.
Implements protocols::jd3::JobQueen.
References protocols::jd3::chunk_library::bad_inner_job_exception(), and protocols::jd3::options_for_job().
|
private |
|
overridevirtual |
Prepare this job for output by building an OutputSpecification for it and storing this specification in the list of recent successes.
Implements protocols::jd3::JobQueen.
References protocols::jd3::chunk_library::bad_inner_job_exception(), protocols::jd3::jd3_job_status_success, and protocols::jd3::options_for_job().
|
private |
The SJQ will keep track of all discarded jobs in the discarded_jobs_ diet.
|
protected |
the nstruct count is taken from either the Job tag, or the command line. "nstruct" is not a valid option to be provided in the <Options> element.
|
protected |
|
protected |
Note that jobs specified on the command line but which the ChunkLibraryJobQueen does not know about do not get set or added to the per-job options. Functions trying to read per-job options that the ChunkLibraryJobQueen does not know about will die. This is intentional.
References protocols::jd3::chunk_library::option_type_from_key().
|
protected |
Return a copy of the Pose to be used with the given job.
References protocols::jd3::chunk_library::bad_inner_job_exception().
|
protected |
Access the pose outputter for a particular job; perhaps this outputter has already been created, or perhaps it needs to be created and stored.
References protocols::jd3::options_for_job().
|
protected |
|
protected |
Find the PoseOutputter for a job specification, respecting the Job-distributor provided output file suffix. The ChunkLibraryJobQueen guarantees that a different Outputter is used for each unique name as long as that name is not the empty string. This allows multiple threads to write to separate files concurrently.
|
protected |
Read access for the subset of nodes in the job DAG which the ChunkLibraryJobQueen is responsible for producing the larval_jobs. They are called "preliminary" jobs because they do not depend on outputs from any previous node in the graph. (The set of job nodes that contain no incoming edges, though, could perhaps be different from the set of preliminary job nodes, so the ChunkLibraryJobQueen requires that the deried job queen inform her of which nodes are the preliminary job nodes.)
Read access for the subset of nodes in the job DAG which the ChunkLibraryJobQueen is responsible for producing the larval_jobs. They are called "preliminary" jobs because they do not depend on outputs from any previous node in the graph. (The set of job nodes that contain no incoming edges, though, could perhaps be different from the set of preliminary job nodes, so the ChunkLibraryJobQueen requires that the derived job queen inform her of which nodes are the preliminary job nodes.)
|
overridevirtual |
A deallocation message first sent to the JobDistributor on this host originating from a remote JobQueen.
Implements protocols::jd3::JobQueen.
References protocols::jd3::deallocation::input_pose_deallocation_msg, and protocols::jd3::TR().
|
protectedvirtual |
Ask the derived JobQueen to expand / refine a preliminary larval job, by possibly reading per-job data out of the Tag associated with each job. If there is nothing that needs to be done by the derived class, it may elect to use the base-class implementation of this function which simply returns a list of the inner-job pointers in the input list. Why all the back and forth? So that the derived job queens can take one input structure and multiply it into several sepearate jobs; e.g. in a ddG protocol, you need to run a separate set of jobs for the WT sequence and also for the mutant sequence. The preliminary job will reflect the structure of the WT, and then that will get expanded out. This funciton is non-const so that the derived class can even track which WT sequences it has seen so that after the first WT sequence for a particular input structure, it only expands out the mutant sequences if many mutants on that input.
This base class implementation merely returns a one-element list containing the input inner_job. Derived classes have the flexibility to create several preliminary jobs from this input job
References protocols::jd3::chunk_library::ChunkLibraryPreliminaryLarvalJob::inner_job.
|
protected |
Override the basic specification of input elements as either PDBs, Silent Structures, or ResourceManager defined structures. This should be called during the derived class's constructor if it is to be used.
TO DO
|
overridevirtual |
JobQueens may optionally define an XSD for their resource definition file which is fed to their resource manager (if they control one). If the JobDistributor is awakened with the flag "jd3::output_resource_xsd <output file>" on the command line, and if the derived JobQueen defines an XSD for resources she uses, then the JobDistributor will write out the resource definition XSD to the output file. The derived queen may return an empty string to indicate that no resources are definable for the resource manager.
Implements protocols::jd3::JobQueen.
|
overridevirtual |
Return the bag of of PoseOutputters (in the form of a MultipleOutputter) for the Pose that has been requested and specified by a particular OutputSpecification; this function guarantees that for each individual outputter-name (respecting the JD-provided filename suffix) that a separate set of PoseOutputters are returned so that multiple threads can potentially output at the same time.
Implements protocols::jd3::JobQueen.
|
private |
helper function that should only be called by the above secondary_outputter_for_job function because of its assumption that the representative_secondary_outputter_map_ map contains an entry for the requested secondary_outputter_type
References protocols::jd3::InnerLarvalJob::jobdef_tag().
|
private |
|
protected |
References protocols::jd3::InnerLarvalJob::jobdef_tag().
|
protected |
If the derived job queen wants the user to get a particular PoseOutputter by default (i.e. when the user hasn't specified which outputter to use in a job definition file and in the absence of a command-line flag that says which outputter to use) instead of the default PoseOutputter chosen by the PoseOutputterFactory, then the DJQ should call this function. If the DJQ has previously called "do_not_accept_all_pose_outputters_from_factory", then the first call to "allow_pose_outputter" specified the default outputter, and a separate call to this function is not absolutely necessary. This function must only be called in the DJQ's constructor.
|
overridevirtual |
The ChunkLibraryJobQueen's implementation is to not update the JobDAG at all: the most basic protocol defines a job DAG with only a single node.
Implements protocols::jd3::JobQueen.
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
Referenced by ChunkLibraryJobQueen().
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
Referenced by ChunkLibraryJobQueen().
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
Referenced by ChunkLibraryJobQueen().
|
private |
|
private |
1.8.7