org.apache.manifoldcf.crawler.interfaces
Interface IJobManager

All Known Implementing Classes:

JobManager

public interface IJobManager

This manager deals with jobs. Each job is associated with a repository connection, and has a number of scheduling options: starting every n hours/days/weeks/months, on specific dates, or "continuous" (which basically establishes a priority queue based on modification frequency). The job itself also specifies "seeds" (or starting points), which are the places that scanning begins. NOTE WELL: Every job is incremental. This means that the job will check for deletions among all the documents that it has scanned in the past, as part of the process of ingesting.

Field Summary

static java.lang.String	_rcsid
static int	ACTION_REMOVE
static int	ACTION_RESCAN
static int	DOCSTATE_NEVERPROCESSED
static int	DOCSTATE_PREVIOUSLYPROCESSED
static int	DOCSTATUS_DELETING
static int	DOCSTATUS_EXPIRING
static int	DOCSTATUS_INACTIVE
static int	DOCSTATUS_PROCESSING
static int	DOCSTATUS_READYFOREXPIRATION
static int	DOCSTATUS_READYFORPROCESSING
static int	DOCSTATUS_WAITINGFOREVER
static int	DOCSTATUS_WAITINGFOREXPIRATION
static int	DOCSTATUS_WAITINGFORPROCESSING

Method Summary

boolean	addDocument(java.lang.Long jobID, java.lang.String[] legalLinkTypes, java.lang.String docIDHash, java.lang.String docID, java.lang.String parentIdentifierHash, java.lang.String relationshipType, int hopcountMethod, java.lang.String[] dataNames, java.lang.Object[][] dataValues, long currentTime, double priority, java.lang.String[] prereqEventNames) Add a document to the queue.
boolean[]	addDocuments(java.lang.Long jobID, java.lang.String[] legalLinkTypes, java.lang.String[] docIDHashes, java.lang.String[] docIDs, java.lang.String parentIdentifierHash, java.lang.String relationshipType, int hopcountMethod, java.lang.String[][] dataNames, java.lang.Object[][][] dataValues, long currentTime, double[] priorities, java.lang.String[][] prereqEventNames) Add documents to the queue in bulk.
boolean[]	addDocumentsInitial(java.lang.Long jobID, java.lang.String[] legalLinkTypes, java.lang.String[] docIDHashes, java.lang.String[] docIDs, boolean overrideSchedule, int hopcountMethod, long currentTime, double[] documentPriorities, java.lang.String[][] prereqEventNames) Add an initial set of documents to the queue.
void	addRemainingDocumentsInitial(java.lang.Long jobID, java.lang.String[] legalLinkTypes, java.lang.String[] docIDHashes, int hopcountMethod) Add an initial set of remaining documents to the queue.
boolean	beginEventSequence(java.lang.String eventName) Begin an event sequence.
boolean	carrydownChangeDocument(DocumentDescription documentDescription, long currentTime, double docPriority) Requeue a document because of carrydown changes.
boolean[]	carrydownChangeDocumentMultiple(DocumentDescription[] documentDescriptions, long currentTime, double[] docPriorities) Requeue a document set because of carrydown changes.
boolean	checkIfOutputReference(java.lang.String connectionName) See if there's a reference to an output connection name.
boolean	checkIfReference(java.lang.String connectionName) See if there's a reference to a connection name.
boolean	checkJobActive(java.lang.Long jobID) Verify that a specific job is indeed still active.
boolean	checkJobBusy(java.lang.Long jobID) Verify if a job is still processing documents, or no longer has any outstanding active documents
void	completeEventSequence(java.lang.String eventName) Complete an event sequence.
IJobDescription	createJob() Create a new job.
void	deinstall() Uninstall the job manager's tables.
void	deleteIngestedDocumentIdentifiers(DocumentDescription[] identifiers) Delete ingested document identifiers (as part of deleting the owning job).
void	deleteJob(java.lang.Long id) Delete a job.
void	deleteJobsReadyForDelete() Delete jobs in need of being deleted (which are marked "ready for delete").
void	doneDocumentsInitial(java.lang.Long jobID, java.lang.String[] legalLinkTypes, boolean isPartial, int hopcountMethod) Signal that a seeding pass has been done.
boolean	errorAbort(java.lang.Long jobID, java.lang.String errorText) Abort a running job due to a fatal error condition.
void	exportConfiguration(java.io.OutputStream os) Export configuration
boolean[]	findHopCounts(java.lang.Long jobID, java.lang.String[] legalLinkTypes, java.lang.String[] docIDHashes, java.lang.String linkType, int limit, int hopcountMethod) Get the specified hop counts, with the limit as described.
IJobDescription[]	findJobsForConnection(java.lang.String connectionName) Get the job IDs associated with a given connection name.
DocumentDescription[]	finishDocuments(java.lang.Long jobID, java.lang.String[] legalLinkTypes, java.lang.String[] parentIdentifierHashes, int hopcountMethod) Complete adding child documents to the queue, for a set of documents.
void	finishJobAborts(long timestamp, java.util.ArrayList abortJobs) Complete the sequence that aborts jobs and makes them runnable again.
void	finishJobs() Put all eligible jobs in the "shutting down" state.
IResultSet	genDocumentStatus(java.lang.String connectionName, StatusFilterCriteria filterCriteria, SortOrder sortOrder, int startRow, int rowCount) Run a 'document status' report.
IResultSet	genQueueStatus(java.lang.String connectionName, StatusFilterCriteria filterCriteria, SortOrder sortOrder, BucketDescription idBucketDescription, int startRow, int rowCount) Run a 'queue status' report.
IJobDescription[]	getAllJobs() Load a sorted list of job descriptions.
java.lang.String[]	getAllSeeds(java.lang.Long jobID) Get all the current seeds.
JobStatus[]	getAllStatus() Get a list of all jobs, and their status information.
DocumentSetAndFlags	getExpiredDocuments(int n, long currentTime) Get up to the next n documents to be expired.
JobStatus[]	getFinishedJobs() Get a list of completed jobs, and their statistics.
JobStartRecord[]	getJobsReadyForDelete() Get the list of jobs that are ready for deletion.
JobStartRecord[]	getJobsReadyForInactivity() Find the list of jobs that need to have their connectors notified of job completion.
JobStartRecord[]	getJobsReadyForSeeding(long currentTime) Get the list of jobs that are ready for seeding.
JobStartRecord[]	getJobsReadyForStartup() Get the list of jobs that are ready for startup.
DocumentDescription[]	getNextAlreadyProcessedReprioritizationDocuments(long currentTime, int n) Get a list of already-processed documents to reprioritize.
DocumentSetAndFlags	getNextCleanableDocuments(int n, long currentTime) Get list of cleanable document descriptions.
DocumentDescription[]	getNextDeletableDocuments(int n, long currentTime) Get list of deletable document descriptions.
DocumentDescription[]	getNextDocuments(int n, long currentTime, long interval, BlockingDocuments blockingDocuments, PerformanceStatistics statistics, DepthStatistics scanRecord) Get up to the next n document(s) to be fetched and processed.
DocumentDescription[]	getNextNotYetProcessedReprioritizationDocuments(long currentTime, int n) Get a list of not-yet-processed documents to reprioritize.
JobStatus[]	getRunningJobs() Get a list of running jobs.
JobStatus	getStatus(java.lang.Long jobID) Get the status of a job.
void	importConfiguration(java.io.InputStream is) Import configuration
void	inactivateJob(java.lang.Long jobID) Inactivate a job, from the notification state.
void	install() Install the job manager's tables.
IJobDescription	load(java.lang.Long id) Load a job for editing.
IJobDescription	load(java.lang.Long id, boolean readOnly) Load a job.
void	manualAbort(java.lang.Long jobID) Manually abort a running job.
void	manualAbortRestart(java.lang.Long jobID) Manually restart a running job.
void	manualStart(java.lang.Long jobID) Manually start a job.
void	markDocumentCompleted(DocumentDescription documentDescription) Note completion of document processing by a job thread of a document.
void	markDocumentCompletedMultiple(DocumentDescription[] documentDescriptions) Note completion of document processing by a job thread of a document.
DocumentDescription[]	markDocumentDeleted(java.lang.Long jobID, java.lang.String[] legalLinkTypes, DocumentDescription documentDescription, int hopcountMethod) Note deletion as result of document processing by a job thread of a document.
DocumentDescription[]	markDocumentDeletedMultiple(java.lang.Long jobID, java.lang.String[] legalLinkTypes, DocumentDescription[] documentDescriptions, int hopcountMethod) Note deletion as result of document processing by a job thread of a document.
void	noteConnectionChange(java.lang.String connectionName) Note a change in connection configuration.
void	noteConnectorDeregistration(java.lang.String[] connectionNames) Note the deregistration of a connector used by the specified connections.
void	noteConnectorRegistration(java.lang.String[] connectionNames) Note the registration of a connector used by the specified connections.
void	noteJobDeleteStarted(java.lang.Long jobID, long startTime) Note job delete started.
void	noteJobSeeded(java.lang.Long jobID, long startTime) Note job seeded.
void	noteJobStarted(java.lang.Long jobID, long startTime) Note job started.
void	noteOutputConnectionChange(java.lang.String connectionName) Note a change in output connection configuration.
void	noteOutputConnectorDeregistration(java.lang.String[] connectionNames) Note the deregistration of an output connector used by the specified connections.
void	noteOutputConnectorRegistration(java.lang.String[] connectionNames) Note the registration of an output connector used by the specified connections.
void	pauseJob(java.lang.Long jobID) Pause a job.
void	prepareDeleteScan(java.lang.Long jobID) Prepare for a delete scan.
void	prepareForStart() Reset the job queue immediately before starting up.
void	prepareFullScan(java.lang.Long jobID, java.lang.String[] legalLinkTypes, int hopcountMethod) Prepare for a full scan.
void	prepareIncrementalScan(java.lang.Long jobID, java.lang.String[] legalLinkTypes, int hopcountMethod) Prepare for an incremental scan.
void	requeueDocument(DocumentDescription documentDescription, java.lang.Long executeTime, int action) Requeue a document for further processing in the future.
void	requeueDocumentMultiple(DocumentDescription[] documentDescriptions, java.lang.Long[] executeTimes, int[] actions) Requeue a document for further processing in the future.
void	resetCleaningDocument(DocumentDescription documentDescription, long checkTime) Reset a cleaning document back to its former state.
void	resetCleaningDocumentMultiple(DocumentDescription[] documentDescriptions, long checkTime) Reset a set of cleaning documents for further processing in the future.
void	resetDeleteStartupWorkerStatus() Reset as part of restoring delete startup threads.
void	resetDeletingDocument(DocumentDescription documentDescription, long checkTime) Reset a deleting document back to its former state.
void	resetDeletingDocumentMultiple(DocumentDescription[] documentDescriptions, long checkTime) Reset a set of deleting documents for further processing in the future.
void	resetDocCleanupWorkerStatus() Reset as part of restoring doc cleanup threads.
void	resetDocDeleteWorkerStatus() Reset as part of restoring doc delete threads.
void	resetDocument(DocumentDescription documentDescription, long executeTime, int action, long failTime, int failCount) Reset an active document back to its former state.
void	resetDocumentMultiple(DocumentDescription[] documentDescriptions, long executeTime, int action, long failTime, int failCount) Reset documents for further processing in the future.
void	resetDocumentWorkerStatus() Reset as part of restoring document worker threads.
void	resetJobs(long currentTime, java.util.ArrayList resetJobs) Reset eligible jobs either back to the "inactive" state, or make them active again.
void	resetJobSchedule(java.lang.Long jobID) Reset job schedule.
void	resetNotificationWorkerStatus() Reset as part of restoring notification threads.
void	resetNotifyJob(java.lang.Long jobID) Reset a job that is notifying back to "ready for notify" state.
void	resetSeedingWorkerStatus() Reset as part of restoring seeding threads.
void	resetSeedJob(java.lang.Long jobID) Reset a seeding job back to "active" state.
void	resetStartDeleteJob(java.lang.Long jobID) Reset a job starting for delete back to "ready for delete" state.
void	resetStartupJob(java.lang.Long jobID) Reset a starting job back to "ready for startup" state.
void	resetStartupWorkerStatus() Reset as part of restoring startup threads.
void	restartJob(java.lang.Long jobID) Restart a paused job.
java.lang.String[]	retrieveParentData(java.lang.Long jobID, java.lang.String docIDHash, java.lang.String dataName) Retrieve specific parent data for a given document.
CharacterInput[]	retrieveParentDataAsFiles(java.lang.Long jobID, java.lang.String docIDHash, java.lang.String dataName) Retrieve specific parent data for a given document.
void	save(IJobDescription jobDescription) Save a job.
void	startJobs(long currentTime, java.util.ArrayList unwaitList) Start jobs based on schedule.
void	waitJobs(long currentTime, java.util.ArrayList waitList) Put active or paused jobs in wait state, if they've exceeded their window.
void	writeDocumentPriorities(long currentTime, DocumentDescription[] descriptions, double[] priorities) Save a set of document priorities.

Field Detail

_rcsid

static final java.lang.String _rcsid

See Also:

Constant Field Values

ACTION_RESCAN

static final int ACTION_RESCAN

See Also:

Constant Field Values

ACTION_REMOVE

static final int ACTION_REMOVE

See Also:

Constant Field Values

DOCSTATE_NEVERPROCESSED

static final int DOCSTATE_NEVERPROCESSED

See Also:

Constant Field Values

DOCSTATE_PREVIOUSLYPROCESSED

static final int DOCSTATE_PREVIOUSLYPROCESSED

See Also:

Constant Field Values

DOCSTATUS_INACTIVE

static final int DOCSTATUS_INACTIVE

See Also:

Constant Field Values

DOCSTATUS_PROCESSING

static final int DOCSTATUS_PROCESSING

See Also:

Constant Field Values

DOCSTATUS_EXPIRING

static final int DOCSTATUS_EXPIRING

See Also:

Constant Field Values

DOCSTATUS_DELETING

static final int DOCSTATUS_DELETING

See Also:

Constant Field Values

DOCSTATUS_READYFORPROCESSING

static final int DOCSTATUS_READYFORPROCESSING

See Also:

Constant Field Values

DOCSTATUS_READYFOREXPIRATION

static final int DOCSTATUS_READYFOREXPIRATION

See Also:

Constant Field Values

DOCSTATUS_WAITINGFORPROCESSING

static final int DOCSTATUS_WAITINGFORPROCESSING

See Also:

Constant Field Values

DOCSTATUS_WAITINGFOREXPIRATION

static final int DOCSTATUS_WAITINGFOREXPIRATION

See Also:

Constant Field Values

DOCSTATUS_WAITINGFOREVER

static final int DOCSTATUS_WAITINGFOREVER

See Also:

Constant Field Values

Method Detail

install

void install()
             throws ManifoldCFException

Install the job manager's tables.

Throws:

ManifoldCFException

deinstall

void deinstall()
               throws ManifoldCFException

Uninstall the job manager's tables.

Throws:

ManifoldCFException

exportConfiguration

void exportConfiguration(java.io.OutputStream os)
                         throws java.io.IOException,
                                ManifoldCFException

Export configuration

Throws:

java.io.IOException

ManifoldCFException

importConfiguration

void importConfiguration(java.io.InputStream is)
                         throws java.io.IOException,
                                ManifoldCFException

Import configuration

Throws:

java.io.IOException

ManifoldCFException

getAllJobs

IJobDescription[] getAllJobs()
                             throws ManifoldCFException

Load a sorted list of job descriptions.

Returns:

the list, sorted by description.

Throws:

ManifoldCFException

createJob

IJobDescription createJob()
                          throws ManifoldCFException

Create a new job.

Returns:

the new job.

Throws:

ManifoldCFException

deleteJob

void deleteJob(java.lang.Long id)
               throws ManifoldCFException

Delete a job.

Parameters:

id - is the job's identifier. This method will purge all the records belonging to the job from the database, as well as remove all documents indexed by the job from the index.

Throws:

ManifoldCFException

load

IJobDescription load(java.lang.Long id)
                     throws ManifoldCFException

Load a job for editing.

Parameters:

id - is the job's identifier.

Returns:

null if the job doesn't exist.

Throws:

ManifoldCFException

load

IJobDescription load(java.lang.Long id,
                     boolean readOnly)
                     throws ManifoldCFException

Load a job.

Parameters:

id - is the job's identifier.

readOnly - is true if a read-only object is desired.

Returns:

null if the job doesn't exist.

Throws:

ManifoldCFException

save

void save(IJobDescription jobDescription)
          throws ManifoldCFException

Save a job.

Parameters:

jobDescription - is the job description.

Throws:

ManifoldCFException

checkIfReference

boolean checkIfReference(java.lang.String connectionName)
                         throws ManifoldCFException

See if there's a reference to a connection name.

Parameters:

connectionName - is the name of the connection.

Returns:

true if there is a reference, false otherwise.

Throws:

ManifoldCFException

checkIfOutputReference

boolean checkIfOutputReference(java.lang.String connectionName)
                               throws ManifoldCFException

See if there's a reference to an output connection name.

Parameters:

connectionName - is the name of the connection.

Returns:

true if there is a reference, false otherwise.

Throws:

ManifoldCFException

findJobsForConnection

IJobDescription[] findJobsForConnection(java.lang.String connectionName)
                                        throws ManifoldCFException

Get the job IDs associated with a given connection name.

Parameters:

connectionName - is the name of the connection.

Returns:

the set of job id's associated with that connection.

Throws:

ManifoldCFException

prepareForStart

void prepareForStart()
                     throws ManifoldCFException

Reset the job queue immediately before starting up. If the system was shut down in the middle of a job, sufficient information should be around in the database to allow it to restart. However, BEFORE all the job threads are spun up, there needs to be a pass over the queue to bring things back to a "normal" state.

Throws:

ManifoldCFException

resetDocumentWorkerStatus

void resetDocumentWorkerStatus()
                               throws ManifoldCFException

Reset as part of restoring document worker threads.

Throws:

ManifoldCFException

resetSeedingWorkerStatus

void resetSeedingWorkerStatus()
                              throws ManifoldCFException

Reset as part of restoring seeding threads.

Throws:

ManifoldCFException

resetDocDeleteWorkerStatus

void resetDocDeleteWorkerStatus()
                                throws ManifoldCFException

Reset as part of restoring doc delete threads.

Throws:

ManifoldCFException

resetDocCleanupWorkerStatus

void resetDocCleanupWorkerStatus()
                                 throws ManifoldCFException

Reset as part of restoring doc cleanup threads.

Throws:

ManifoldCFException

resetDeleteStartupWorkerStatus

void resetDeleteStartupWorkerStatus()
                                    throws ManifoldCFException

Reset as part of restoring delete startup threads.

Throws:

ManifoldCFException

resetNotificationWorkerStatus

void resetNotificationWorkerStatus()
                                   throws ManifoldCFException

Reset as part of restoring notification threads.

Throws:

ManifoldCFException

resetStartupWorkerStatus

void resetStartupWorkerStatus()
                              throws ManifoldCFException

Reset as part of restoring startup threads.

Throws:

ManifoldCFException

getNextAlreadyProcessedReprioritizationDocuments

DocumentDescription[] getNextAlreadyProcessedReprioritizationDocuments(long currentTime,
                                                                       int n)
                                                                       throws ManifoldCFException

Get a list of already-processed documents to reprioritize. Documents in all jobs will be returned by this method. Up to n document descriptions will be returned.

Parameters:

currentTime - is the current time stamp for this prioritization pass. Avoid picking up any documents that are labeled with this timestamp or after.

n - is the maximum number of document descriptions desired.

Returns:

the document descriptions.

Throws:

ManifoldCFException

getNextNotYetProcessedReprioritizationDocuments

DocumentDescription[] getNextNotYetProcessedReprioritizationDocuments(long currentTime,
                                                                      int n)
                                                                      throws ManifoldCFException

Get a list of not-yet-processed documents to reprioritize. Documents in all jobs will be returned by this method. Up to n document descriptions will be returned.

Parameters:

currentTime - is the current time stamp for this prioritization pass. Avoid picking up any documents that are labeled with this timestamp or after.

n - is the maximum number of document descriptions desired.

Returns:

the document descriptions.

Throws:

ManifoldCFException

writeDocumentPriorities

void writeDocumentPriorities(long currentTime,
                             DocumentDescription[] descriptions,
                             double[] priorities)
                             throws ManifoldCFException

Save a set of document priorities. In the case where a document was eligible to have its priority set, but it no longer is eligible, then the provided priority will not be written.

Parameters:

currentTime - is the time in milliseconds since epoch.

descriptions - are the document descriptions.

priorities - are the desired priorities.

Throws:

ManifoldCFException

getExpiredDocuments

DocumentSetAndFlags getExpiredDocuments(int n,
                                        long currentTime)
                                        throws ManifoldCFException

Get up to the next n documents to be expired. This method marks the documents whose descriptions have been returned as "being processed", or active. The same marking is used as is used for documents that have been queued for worker threads. The model is thus identical.

Parameters:

n - is the maximum number of records desired.

currentTime - is the current time.

Returns:

the array of document descriptions to expire.

Throws:

ManifoldCFException

getNextDocuments

DocumentDescription[] getNextDocuments(int n,
                                       long currentTime,
                                       long interval,
                                       BlockingDocuments blockingDocuments,
                                       PerformanceStatistics statistics,
                                       DepthStatistics scanRecord)
                                       throws ManifoldCFException

Get up to the next n document(s) to be fetched and processed. This fetch returns records that contain the document identifier, plus all instructions pertaining to the document's handling (e.g. whether it should be refetched if the version has not changed). This method also marks the documents whose descriptions have be returned as "being processed".

Parameters:

n - is the number of documents desired.

currentTime - is the current time; some fetches do not occur until a specific time.

interval - is the number of milliseconds that this set of documents should represent (for throttling).

blockingDocuments - is the place to record documents that were encountered, are eligible for reprioritization, but could not be queued due to throttling considerations.

statistics - are the current performance statistics per connection, which are used to balance the queue stuffing so that individual connections are not overwhelmed.

scanRecord - retains the bins from all documents encountered from the query, even those that were skipped due to being overcommitted.

Returns:

the array of document descriptions to fetch and process.

Throws:

ManifoldCFException

checkJobActive

boolean checkJobActive(java.lang.Long jobID)
                       throws ManifoldCFException

Verify that a specific job is indeed still active. This is used to permit abort or pause to be relatively speedy. The query done within MUST be cached in order to not cause undue performance degradation.

Parameters:

jobID - is the job identifier.

Returns:

true if the job is in one of the "active" states.

Throws:

ManifoldCFException

checkJobBusy

boolean checkJobBusy(java.lang.Long jobID)
                     throws ManifoldCFException

Verify if a job is still processing documents, or no longer has any outstanding active documents

Throws:

ManifoldCFException

markDocumentCompletedMultiple

void markDocumentCompletedMultiple(DocumentDescription[] documentDescriptions)
                                   throws ManifoldCFException

Note completion of document processing by a job thread of a document. This method causes the state of the document to be marked as "completed".

Parameters:

documentDescriptions - are the description objects for the documents that were processed.

Throws:

ManifoldCFException

markDocumentCompleted

void markDocumentCompleted(DocumentDescription documentDescription)
                           throws ManifoldCFException

Note completion of document processing by a job thread of a document. This method causes the state of the document to be marked as "completed".

Parameters:

documentDescription - is the description object for the document that was processed.

Throws:

ManifoldCFException

markDocumentDeletedMultiple

DocumentDescription[] markDocumentDeletedMultiple(java.lang.Long jobID,
                                                  java.lang.String[] legalLinkTypes,
                                                  DocumentDescription[] documentDescriptions,
                                                  int hopcountMethod)
                                                  throws ManifoldCFException

Note deletion as result of document processing by a job thread of a document.

Parameters:

documentDescriptions - are the set of description objects for the documents that were processed.

hopcountMethod - is one of complete, partial, or nevercomplete.

Returns:

the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.

Throws:

ManifoldCFException

markDocumentDeleted

DocumentDescription[] markDocumentDeleted(java.lang.Long jobID,
                                          java.lang.String[] legalLinkTypes,
                                          DocumentDescription documentDescription,
                                          int hopcountMethod)
                                          throws ManifoldCFException

Note deletion as result of document processing by a job thread of a document.

Parameters:

documentDescription - is the description object for the document that was processed.

hopcountMethod - is one of complete, partial, or nevercomplete.

Returns:

the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.

Throws:

ManifoldCFException

carrydownChangeDocumentMultiple

boolean[] carrydownChangeDocumentMultiple(DocumentDescription[] documentDescriptions,
                                          long currentTime,
                                          double[] docPriorities)
                                          throws ManifoldCFException

Requeue a document set because of carrydown changes. This method is called when carrydown data is modified for a set of documents. The documents must be requeued for immediate reprocessing, even to the extent that if one is *already* being processed, it will need to be done over again.

Parameters:

documentDescriptions - is the set of description objects for the documents that have had their parent carrydown information changed.

docPriorities - are the document priorities to assign to the documents, if needed.

Returns:

a flag for each document priority, true if it was used, false otherwise.

Throws:

ManifoldCFException

carrydownChangeDocument

boolean carrydownChangeDocument(DocumentDescription documentDescription,
                                long currentTime,
                                double docPriority)
                                throws ManifoldCFException

Requeue a document because of carrydown changes. This method is called when carrydown data is modified for a document. The document must be requeued for immediate reprocessing, even to the extent that if it is *already* being processed, it will need to be done over again.

Parameters:

documentDescription - is the description object for the document that has had its parent carrydown information changed.

docPriority - is the document priority to assign to the document, if needed.

Returns:

a flag for the document priority, true if it was used, false otherwise.

Throws:

ManifoldCFException

requeueDocumentMultiple

void requeueDocumentMultiple(DocumentDescription[] documentDescriptions,
                             java.lang.Long[] executeTimes,
                             int[] actions)
                             throws ManifoldCFException

Requeue a document for further processing in the future. This method is called after a document is processed, when the job is a "continuous" one. It is essentially equivalent to noting that the document processing is complete, except the document remains on the queue.

Parameters:

documentDescriptions - is the set of description objects for the document that was processed.

executeTimes - are the times that the documents should be rescanned. Null indicates "never".

actions - are what should be done when the time arrives. Choices are ACTION_RESCAN or ACTION_REMOVE.

Throws:

ManifoldCFException

requeueDocument

void requeueDocument(DocumentDescription documentDescription,
                     java.lang.Long executeTime,
                     int action)
                     throws ManifoldCFException

Requeue a document for further processing in the future. This method is called after a document is processed, when the job is a "continuous" one. It is essentially equivalent to noting that the document processing is complete, except the document remains on the queue.

Parameters:

documentDescription - is the description object for the document that was processed.

executeTime - is the time that the document should be rescanned. Null indicates "never".

action - is what should be done when the time arrives. Choices include ACTION_RESCAN or ACTION_REMOVE.

Throws:

ManifoldCFException

resetDocumentMultiple

void resetDocumentMultiple(DocumentDescription[] documentDescriptions,
                           long executeTime,
                           int action,
                           long failTime,
                           int failCount)
                           throws ManifoldCFException

Reset documents for further processing in the future. This method is called after a service interruption is thrown. It is essentially equivalent to resetting the time for documents to be reprocessed.

Parameters:

documentDescriptions - is the set of description objects for the document that was processed.

executeTime - is the time that the documents should be rescanned.

failTime - is the time beyond which hard failure should occur.

failCount - is the number of permitted failures before a hard error is signalled.

Throws:

ManifoldCFException

resetDocument

void resetDocument(DocumentDescription documentDescription,
                   long executeTime,
                   int action,
                   long failTime,
                   int failCount)
                   throws ManifoldCFException

Reset an active document back to its former state. This gets done when there's a service interruption and the document cannot be processed yet.

Parameters:

documentDescription - is the description object for the document that was processed.

executeTime - is the time that the document should be rescanned.

failTime - is the time that the document should be considered to have failed, if it has not been successfully read until then.

failCount - is the number of permitted failures before a hard error is signalled.

Throws:

ManifoldCFException

resetDeletingDocumentMultiple

void resetDeletingDocumentMultiple(DocumentDescription[] documentDescriptions,
                                   long checkTime)
                                   throws ManifoldCFException

Reset a set of deleting documents for further processing in the future. This method is called after some unknown number of the documents were deleted, but then an ingestion service interruption occurred. Note well: The logic here basically presumes that we cannot know whether the documents were indeed processed or not. If we knew for a fact that none of the documents had been handled, it would be possible to look at the document's current status and decide what the new status ought to be, based on a true rollback scenario. Such cases, however, are rare enough so that special logic is probably not worth it.

Parameters:

documentDescriptions - is the set of description objects for the document that was processed.

checkTime - is the minimum time for the next cleaning attempt.

Throws:

ManifoldCFException

resetDeletingDocument

void resetDeletingDocument(DocumentDescription documentDescription,
                           long checkTime)
                           throws ManifoldCFException

Reset a deleting document back to its former state. This gets done when a deleting thread sees a service interruption, etc., from the ingestion system.

Parameters:

documentDescription - is the description object for the document that was cleaned.

checkTime - is the minimum time for the next cleaning attempt.

Throws:

ManifoldCFException

resetCleaningDocument

void resetCleaningDocument(DocumentDescription documentDescription,
                           long checkTime)
                           throws ManifoldCFException

Reset a cleaning document back to its former state. This gets done when a cleaning thread sees a service interruption, etc., from the ingestion system.

Parameters:

documentDescription - is the description object for the document that was cleaned.

checkTime - is the minimum time for the next cleaning attempt.

Throws:

ManifoldCFException

resetCleaningDocumentMultiple

void resetCleaningDocumentMultiple(DocumentDescription[] documentDescriptions,
                                   long checkTime)
                                   throws ManifoldCFException

Reset a set of cleaning documents for further processing in the future. This method is called after some unknown number of the documents were cleaned, but then an ingestion service interruption occurred. Note well: The logic here basically presumes that we cannot know whether the documents were indeed cleaned or not. If we knew for a fact that none of the documents had been handled, it would be possible to look at the document's current status and decide what the new status ought to be, based on a true rollback scenario. Such cases, however, are rare enough so that special logic is probably not worth it.

Parameters:

documentDescriptions - is the set of description objects for the document that was cleaned.

checkTime - is the minimum time for the next cleaning attempt.

Throws:

ManifoldCFException

addDocumentsInitial

boolean[] addDocumentsInitial(java.lang.Long jobID,
                              java.lang.String[] legalLinkTypes,
                              java.lang.String[] docIDHashes,
                              java.lang.String[] docIDs,
                              boolean overrideSchedule,
                              int hopcountMethod,
                              long currentTime,
                              double[] documentPriorities,
                              java.lang.String[][] prereqEventNames)
                              throws ManifoldCFException

Add an initial set of documents to the queue. This method is called during job startup, when the queue is being loaded. A set of document references is passed to this method, which updates the status of the document in the specified job's queue, according to specific state rules.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

docIDHashes - are the hashes of the local document identifiers (primary key).

docIDs - are the local document identifiers.

overrideSchedule - is true if any existing document schedule should be overridden.

hopcountMethod - is either accurate, nodelete, or neverdelete.

currentTime - is the current time in milliseconds since epoch.

documentPriorities - are the document priorities corresponding to the document identifiers.

prereqEventNames - are the events that must be completed before each document can be processed.

Returns:

true if the priority value(s) were used, false otherwise.

Throws:

ManifoldCFException

addRemainingDocumentsInitial

void addRemainingDocumentsInitial(java.lang.Long jobID,
                                  java.lang.String[] legalLinkTypes,
                                  java.lang.String[] docIDHashes,
                                  int hopcountMethod)
                                  throws ManifoldCFException

Add an initial set of remaining documents to the queue. This method is called during job startup, when the queue is being loaded, to list documents that were NOT included by calling addDocumentsInitial(). Documents listed here are simply designed to enable the framework to get rid of old, invalid seeds. They are not queued for processing.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

docIDHashes - are the hash values of the local document identifiers.

hopcountMethod - is either accurate, nodelete, or neverdelete.

Throws:

ManifoldCFException

doneDocumentsInitial

void doneDocumentsInitial(java.lang.Long jobID,
                          java.lang.String[] legalLinkTypes,
                          boolean isPartial,
                          int hopcountMethod)
                          throws ManifoldCFException

Signal that a seeding pass has been done. Call this method at the end of a seeding pass. It is used to perform the bookkeeping necessary to maintain the hopcount table.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

isPartial - is set if the seeds provided are only a partial list. Some connectors cannot supply a full list of seeds on every seeding iteration; this acknowledges that limitation.

hopcountMethod - describes how to handle deletions for hopcount purposes.

Throws:

ManifoldCFException

beginEventSequence

boolean beginEventSequence(java.lang.String eventName)
                           throws ManifoldCFException

Begin an event sequence.

Parameters:

eventName - is the name of the event.

Returns:

true if the event could be created, or false if it's already there.

Throws:

ManifoldCFException

completeEventSequence

void completeEventSequence(java.lang.String eventName)
                           throws ManifoldCFException

Complete an event sequence.

Parameters:

eventName - is the name of the event.

Throws:

ManifoldCFException

findHopCounts

boolean[] findHopCounts(java.lang.Long jobID,
                        java.lang.String[] legalLinkTypes,
                        java.lang.String[] docIDHashes,
                        java.lang.String linkType,
                        int limit,
                        int hopcountMethod)
                        throws ManifoldCFException

Get the specified hop counts, with the limit as described.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

docIDHashes - is the set of document hashes to find the hopcount for.

linkType - is the kind of link to find the hopcount for.

limit - is the limit, beyond which a negative distance may be returned.

hopcountMethod - describes how to handle deletions for hopcount purposes.

Returns:

a vector of booleans corresponding to the documents requested. A true value is returned if the document is within the specified limit, false otherwise.

Throws:

ManifoldCFException

getAllSeeds

java.lang.String[] getAllSeeds(java.lang.Long jobID)
                               throws ManifoldCFException

Get all the current seeds. Returns the seed document identifiers for a job.

Parameters:

jobID - is the job identifier.

Returns:

the document identifier hashes that are currently considered to be seeds.

Throws:

ManifoldCFException

addDocument

boolean addDocument(java.lang.Long jobID,
                    java.lang.String[] legalLinkTypes,
                    java.lang.String docIDHash,
                    java.lang.String docID,
                    java.lang.String parentIdentifierHash,
                    java.lang.String relationshipType,
                    int hopcountMethod,
                    java.lang.String[] dataNames,
                    java.lang.Object[][] dataValues,
                    long currentTime,
                    double priority,
                    java.lang.String[] prereqEventNames)
                    throws ManifoldCFException

Add a document to the queue. This method is called during document processing, when a document reference is discovered. The document reference is passed to this method, which updates the status of the document in the specified job's queue, according to specific state rules.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

docIDHash - is the local document identifier hash value.

parentIdentifierHash - is the optional parent identifier hash value for this document. Pass null if none.

relationshipType - is the optional link type between this document and its parent. Pass null if there is no relationship with a parent.

hopcountMethod - is either accurate, nodelete, or neverdelete.

dataNames - are the names of the data to carry down to the child from this parent.

dataValues - are the values to carry down to the child from this parent, corresponding to dataNames above. If CharacterInput objects are passed in here, it is the caller's responsibility to clean these up.

currentTime - is the time in milliseconds since epoch that will be recorded for this operation.

priority - is the desired document priority for the document.

prereqEventNames - are the events that must be completed before the document can be processed.

Returns:

true if the priority value was used, false otherwise.

Throws:

ManifoldCFException

addDocuments

boolean[] addDocuments(java.lang.Long jobID,
                       java.lang.String[] legalLinkTypes,
                       java.lang.String[] docIDHashes,
                       java.lang.String[] docIDs,
                       java.lang.String parentIdentifierHash,
                       java.lang.String relationshipType,
                       int hopcountMethod,
                       java.lang.String[][] dataNames,
                       java.lang.Object[][][] dataValues,
                       long currentTime,
                       double[] priorities,
                       java.lang.String[][] prereqEventNames)
                       throws ManifoldCFException

Add documents to the queue in bulk. This method is called during document processing, when a set of document references are discovered. The document references are passed to this method, which updates the status of the document(s) in the specified job's queue, according to specific state rules.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

docIDHashes - are the hashes of the local document identifiers.

docIDs - are the local document identifiers.

parentIdentifierHash - is the optional parent identifier hash of these documents. Pass null if none.

relationshipType - is the optional link type between this document and its parent. Pass null if there is no relationship with a parent.

hopcountMethod - is either accurate, nodelete, or neverdelete.

dataNames - are the names of the data to carry down to the child from this parent.

dataValues - are the values to carry down to the child from this parent, corresponding to dataNames above. If CharacterInput objects are passed in here, it is the caller's responsibility to clean these up.

currentTime - is the time in milliseconds since epoch that will be recorded for this operation.

priorities - are the desired document priorities for the documents.

prereqEventNames - are the events that must be completed before each document can be processed.

Returns:

an array of boolean values indicating whether or not the passed-in priority value was used or not for each doc id (true if used).

Throws:

ManifoldCFException

finishDocuments

DocumentDescription[] finishDocuments(java.lang.Long jobID,
                                      java.lang.String[] legalLinkTypes,
                                      java.lang.String[] parentIdentifierHashes,
                                      int hopcountMethod)
                                      throws ManifoldCFException

Complete adding child documents to the queue, for a set of documents. This method is called at the end of document processing, to help the hopcount tracking engine do its bookkeeping.

Parameters:

jobID - is the job identifier.

legalLinkTypes - is the set of legal link types that this connector generates.

parentIdentifierHashes - are the hashes of the document identifiers for whom child link extraction just took place.

hopcountMethod - describes how to handle deletions for hopcount purposes.

Returns:

the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.

Throws:

ManifoldCFException

retrieveParentData

java.lang.String[] retrieveParentData(java.lang.Long jobID,
                                      java.lang.String docIDHash,
                                      java.lang.String dataName)
                                      throws ManifoldCFException

Retrieve specific parent data for a given document.

Parameters:

jobID - is the job identifier.

docIDHash - is the hash of the document identifier.

dataName - is the kind of data to retrieve.

Returns:

the unique data values.

Throws:

ManifoldCFException

retrieveParentDataAsFiles

CharacterInput[] retrieveParentDataAsFiles(java.lang.Long jobID,
                                           java.lang.String docIDHash,
                                           java.lang.String dataName)
                                           throws ManifoldCFException

Retrieve specific parent data for a given document.

Parameters:

jobID - is the job identifier.

docIDHash - is the document identifier hash value.

dataName - is the kind of data to retrieve.

Returns:

the unique data values.

Throws:

ManifoldCFException

manualStart

void manualStart(java.lang.Long jobID)
                 throws ManifoldCFException

Manually start a job. The specified job will be run REGARDLESS of the timed windows, and will not cease until complete. If the job is already running, this operation will assure that the job does not pause when its window ends. The job can be manually paused, or manually aborted.

Parameters:

jobID - is the ID of the job to start.

Throws:

ManifoldCFException

manualAbort

void manualAbort(java.lang.Long jobID)
                 throws ManifoldCFException

Manually abort a running job. The job will be permanently stopped, and will not run again until automatically started based on schedule, or manually started.

Parameters:

jobID - is the job to abort.

Throws:

ManifoldCFException

manualAbortRestart

void manualAbortRestart(java.lang.Long jobID)
                        throws ManifoldCFException

Manually restart a running job. The job will be stopped and restarted. Any schedule affinity will be lost, until the job finishes on its own.

Parameters:

jobID - is the job to abort.

Throws:

ManifoldCFException

pauseJob

void pauseJob(java.lang.Long jobID)
              throws ManifoldCFException

Pause a job.

Parameters:

jobID - is the job identifier to pause.

Throws:

ManifoldCFException

restartJob

void restartJob(java.lang.Long jobID)
                throws ManifoldCFException

Restart a paused job.

Parameters:

jobID - is the job identifier to restart.

Throws:

ManifoldCFException

resetJobSchedule

void resetJobSchedule(java.lang.Long jobID)
                      throws ManifoldCFException

Reset job schedule. This re-evaluates whether the job should be started now. This method would typically be called after a job's scheduling window has been changed.

Parameters:

jobID - is the job identifier.

Throws:

ManifoldCFException

startJobs

void startJobs(long currentTime,
               java.util.ArrayList unwaitList)
               throws ManifoldCFException

Start jobs based on schedule. This method marks all the appropriate jobs as "in progress", which is all that should be needed to start them.

Parameters:

currentTime - is the current time in milliseconds since epoch.

unwaitList - is filled in with the set of job id's that were resumed (Long's).

Throws:

ManifoldCFException

waitJobs

void waitJobs(long currentTime,
              java.util.ArrayList waitList)
              throws ManifoldCFException

Put active or paused jobs in wait state, if they've exceeded their window.

Parameters:

currentTime - is the current time in milliseconds since epoch.

waitList - is filled in with the set of job id's that were put into a wait state (Long's).

Throws:

ManifoldCFException

getJobsReadyForSeeding

JobStartRecord[] getJobsReadyForSeeding(long currentTime)
                                        throws ManifoldCFException

Get the list of jobs that are ready for seeding.

Parameters:

currentTime - is the current time in milliseconds since epoch.

Returns:

jobs that are active and are running in adaptive mode. These will be seeded based on what the connector says should be added to the queue.

Throws:

ManifoldCFException

resetSeedJob

void resetSeedJob(java.lang.Long jobID)
                  throws ManifoldCFException

Reset a seeding job back to "active" state.

Parameters:

jobID - is the job id.

Throws:

ManifoldCFException

getJobsReadyForDelete

JobStartRecord[] getJobsReadyForDelete()
                                       throws ManifoldCFException

Get the list of jobs that are ready for deletion.

Returns:

jobs that were in the "readyfordelete" state.

Throws:

ManifoldCFException

getJobsReadyForStartup

JobStartRecord[] getJobsReadyForStartup()
                                        throws ManifoldCFException

Get the list of jobs that are ready for startup.

Returns:

jobs that were in the "readyforstartup" state. These will be marked as being in the "starting up" state.

Throws:

ManifoldCFException

getJobsReadyForInactivity

JobStartRecord[] getJobsReadyForInactivity()
                                           throws ManifoldCFException

Find the list of jobs that need to have their connectors notified of job completion.

Returns:

the ID's of jobs that need their output connectors notified in order to become inactive.

Throws:

ManifoldCFException

inactivateJob

void inactivateJob(java.lang.Long jobID)
                   throws ManifoldCFException

Inactivate a job, from the notification state.

Parameters:

jobID - is the ID of the job to inactivate.

Throws:

ManifoldCFException

resetStartDeleteJob

void resetStartDeleteJob(java.lang.Long jobID)
                         throws ManifoldCFException

Reset a job starting for delete back to "ready for delete" state.

Parameters:

jobID - is the job id.

Throws:

ManifoldCFException

resetNotifyJob

void resetNotifyJob(java.lang.Long jobID)
                    throws ManifoldCFException

Reset a job that is notifying back to "ready for notify" state.

Parameters:

jobID - is the job id.

Throws:

ManifoldCFException

resetStartupJob

void resetStartupJob(java.lang.Long jobID)
                     throws ManifoldCFException

Reset a starting job back to "ready for startup" state.

Parameters:

jobID - is the job id.

Throws:

ManifoldCFException

prepareDeleteScan

void prepareDeleteScan(java.lang.Long jobID)
                       throws ManifoldCFException

Prepare for a delete scan.

Parameters:

jobID - is the job id.

Throws:

ManifoldCFException

prepareFullScan

void prepareFullScan(java.lang.Long jobID,
                     java.lang.String[] legalLinkTypes,
                     int hopcountMethod)
                     throws ManifoldCFException

Prepare for a full scan.

Parameters:

jobID - is the job id.

legalLinkTypes - are the link types allowed for the job.

hopcountMethod - describes how to handle deletions for hopcount purposes.

Throws:

ManifoldCFException

prepareIncrementalScan

void prepareIncrementalScan(java.lang.Long jobID,
                            java.lang.String[] legalLinkTypes,
                            int hopcountMethod)
                            throws ManifoldCFException

Prepare for an incremental scan.

Parameters:

jobID - is the job id.

legalLinkTypes - are the link types allowed for the job.

hopcountMethod - describes how to handle deletions for hopcount purposes.

Throws:

ManifoldCFException

noteJobDeleteStarted

void noteJobDeleteStarted(java.lang.Long jobID,
                          long startTime)
                          throws ManifoldCFException

Note job delete started.

Parameters:

jobID - is the job id.

startTime - is the job start time.

Throws:

ManifoldCFException

noteJobStarted

void noteJobStarted(java.lang.Long jobID,
                    long startTime)
                    throws ManifoldCFException

Note job started.

Parameters:

jobID - is the job id.

startTime - is the job start time.

Throws:

ManifoldCFException

noteJobSeeded

void noteJobSeeded(java.lang.Long jobID,
                   long startTime)
                   throws ManifoldCFException

Note job seeded.

Parameters:

jobID - is the job id.

startTime - is the job seed time.

Throws:

ManifoldCFException

noteConnectorDeregistration

void noteConnectorDeregistration(java.lang.String[] connectionNames)
                                 throws ManifoldCFException

Note the deregistration of a connector used by the specified connections. This method will be called when the connector is deregistered. Jobs that use these connections must therefore enter appropriate states.

Parameters:

connectionNames - is the set of connection names.

Throws:

ManifoldCFException

noteConnectorRegistration

void noteConnectorRegistration(java.lang.String[] connectionNames)
                               throws ManifoldCFException

Note the registration of a connector used by the specified connections. This method will be called when a connector is registered, on which the specified connections depend.

Parameters:

connectionNames - is the set of connection names.

Throws:

ManifoldCFException

noteConnectionChange

void noteConnectionChange(java.lang.String connectionName)
                          throws ManifoldCFException

Note a change in connection configuration. This method will be called whenever a connection's configuration is modified, or when an external repository change is signalled.

Throws:

ManifoldCFException

noteOutputConnectorDeregistration

void noteOutputConnectorDeregistration(java.lang.String[] connectionNames)
                                       throws ManifoldCFException

Note the deregistration of an output connector used by the specified connections. This method will be called when the connector is deregistered. Jobs that use these connections must therefore enter appropriate states.

Parameters:

connectionNames - is the set of connection names.

Throws:

ManifoldCFException

noteOutputConnectorRegistration

void noteOutputConnectorRegistration(java.lang.String[] connectionNames)
                                     throws ManifoldCFException

Note the registration of an output connector used by the specified connections. This method will be called when a connector is registered, on which the specified connections depend.

Parameters:

connectionNames - is the set of connection names.

Throws:

ManifoldCFException

noteOutputConnectionChange

void noteOutputConnectionChange(java.lang.String connectionName)
                                throws ManifoldCFException

Note a change in output connection configuration. This method will be called whenever a connection's configuration is modified, or when an external output target change is signalled.

Throws:

ManifoldCFException

deleteJobsReadyForDelete

void deleteJobsReadyForDelete()
                              throws ManifoldCFException

Delete jobs in need of being deleted (which are marked "ready for delete"). This method is meant to be called periodically to perform delete processing on jobs.

Throws:

ManifoldCFException

getNextDeletableDocuments

DocumentDescription[] getNextDeletableDocuments(int n,
                                                long currentTime)
                                                throws ManifoldCFException

Get list of deletable document descriptions. This list will take into account multiple jobs that may own the same document.

Parameters:

n - is the maximum number of documents to return.

currentTime - is the current time; some fetches do not occur until a specific time.

Returns:

the document descriptions for these documents.

Throws:

ManifoldCFException

getNextCleanableDocuments

DocumentSetAndFlags getNextCleanableDocuments(int n,
                                              long currentTime)
                                              throws ManifoldCFException

Get list of cleanable document descriptions. This list will take into account multiple jobs that may own the same document.

Parameters:

n - is the maximum number of documents to return.

currentTime - is the current time; some fetches do not occur until a specific time.

Returns:

the document descriptions for these documents.

Throws:

ManifoldCFException

deleteIngestedDocumentIdentifiers

void deleteIngestedDocumentIdentifiers(DocumentDescription[] identifiers)
                                       throws ManifoldCFException

Delete ingested document identifiers (as part of deleting the owning job). The number of identifiers specified is guaranteed to be less than the maxInClauseCount for the database.

Parameters:

identifiers - is the set of document identifiers.

Throws:

ManifoldCFException

errorAbort

boolean errorAbort(java.lang.Long jobID,
                   java.lang.String errorText)
                   throws ManifoldCFException

Abort a running job due to a fatal error condition.

Parameters:

jobID - is the job to abort.

errorText - is the error text.

Returns:

true if this is the first abort for the job.

Throws:

ManifoldCFException

finishJobAborts

void finishJobAborts(long timestamp,
                     java.util.ArrayList abortJobs)
                     throws ManifoldCFException

Complete the sequence that aborts jobs and makes them runnable again.

Parameters:

timestamp - is the current time in milliseconds since epoch.

abortJobs - is filled in with the set of IJobDescription objects that were aborted.

Throws:

ManifoldCFException

finishJobs

void finishJobs()
                throws ManifoldCFException

Put all eligible jobs in the "shutting down" state.

Throws:

ManifoldCFException

resetJobs

void resetJobs(long currentTime,
               java.util.ArrayList resetJobs)
               throws ManifoldCFException

Reset eligible jobs either back to the "inactive" state, or make them active again. The latter will occur if the cleanup phase of the job generated more pending documents. This method is used to pick up all jobs in the shutting down state whose purgatory or being-cleaned records have been all processed.

Parameters:

currentTime - is the current time in milliseconds since epoch.

resetJobs - is filled in with the set of IJobDescription objects that were reset.

Throws:

ManifoldCFException

getStatus

JobStatus getStatus(java.lang.Long jobID)
                    throws ManifoldCFException

Get the status of a job.

Returns:

the status object for the specified job.

Throws:

ManifoldCFException

getAllStatus

JobStatus[] getAllStatus()
                         throws ManifoldCFException

Get a list of all jobs, and their status information.

Returns:

an ordered array of job status objects.

Throws:

ManifoldCFException

getRunningJobs

JobStatus[] getRunningJobs()
                           throws ManifoldCFException

Get a list of running jobs. This is for status reporting.

Returns:

an array of the job status objects.

Throws:

ManifoldCFException

getFinishedJobs

JobStatus[] getFinishedJobs()
                            throws ManifoldCFException

Get a list of completed jobs, and their statistics.

Returns:

an array of the job status objects.

Throws:

ManifoldCFException

genDocumentStatus

IResultSet genDocumentStatus(java.lang.String connectionName,
                             StatusFilterCriteria filterCriteria,
                             SortOrder sortOrder,
                             int startRow,
                             int rowCount)
                             throws ManifoldCFException

Run a 'document status' report.

Parameters:

connectionName - is the name of the connection.

filterCriteria - are the criteria used to limit the records considered for the report.

sortOrder - is the specified sort order of the final report.

startRow - is the first row to include.

rowCount - is the number of rows to include.

Returns:

the results, with the following columns: identifier, job, state, status, scheduled, action, retrycount, retrylimit. The "scheduled" column and the "retrylimit" column are long values representing a time; all other values will be user-friendly strings.

Throws:

ManifoldCFException

genQueueStatus

IResultSet genQueueStatus(java.lang.String connectionName,
                          StatusFilterCriteria filterCriteria,
                          SortOrder sortOrder,
                          BucketDescription idBucketDescription,
                          int startRow,
                          int rowCount)
                          throws ManifoldCFException

Run a 'queue status' report.

Parameters:

connectionName - is the name of the connection.

filterCriteria - are the criteria used to limit the records considered for the report.

sortOrder - is the specified sort order of the final report.

idBucketDescription - is the bucket description for generating the identifier class.

startRow - is the first row to include.

rowCount - is the number of rows to include.

Returns:

the results, with the following columns: idbucket, inactive, processing, expiring, deleting, processready, expireready, processwaiting, expirewaiting

Throws:

ManifoldCFException