org.apache.manifoldcf.crawler.jobs
Class JobManager.ThrottleLimit

java.lang.Object
  org.apache.manifoldcf.crawler.jobs.JobManager.ThrottleLimit

All Implemented Interfaces:

ILimitChecker

Enclosing class:

JobManager

protected static class JobManager.ThrottleLimit
extends java.lang.Object
implements ILimitChecker

This class provides the throttling limits for the job queueing query.

Field Summary

protected java.util.HashMap	activeConnections
protected java.util.ArrayList	blockingDocumentArray
protected java.util.HashMap	connectionMap
protected int	documentsProcessed
protected java.util.HashMap	jobConnection
protected java.util.HashMap	jobQueueHash
protected java.util.HashMap	maxConnectionCounts
protected int	maxSetSize
protected int	n
protected long	prioritizationTime
protected java.util.HashMap	setSizes

Fields inherited from interface org.apache.manifoldcf.core.interfaces.ILimitChecker

_rcsid

Constructor Summary

JobManager.ThrottleLimit(int n, long prioritizationTime)
Constructor.

Method Summary

void	addConnectionName(java.lang.String connectionName, IRepositoryConnector connectorInstance) Add an active connection.
void	addJob(java.lang.Long jobID, java.lang.String connectionName) Add a job/connection name map entry.
void	addLimit(java.lang.String connectionName, java.lang.String regexp, int upperLimit) Add a document limit for a specified connection.
boolean	checkContinue() See if we should examine another row.
boolean	checkInclude(IResultRow row) See if a result row should be included in the final result set.
boolean	doesCompareWork() See if this class can be legitimately compared against another of the same type.
ILimitChecker	duplicate() Create a duplicate of this class instance.
boolean	equals(java.lang.Object object) Compare two objects and see if equal.
int	getRemainingDocuments() Get the remaining documents we should query for.
int	hashCode() Find the hashcode for this class.
JobManager.ThrottleLimit	makeDeepCopy() Make a deep copy
void	setConnectionLimit(java.lang.String connectionName, int maxDocuments) Set a connection-based total document limit.
void	tallyBlockingDocuments(BlockingDocuments blockingDocuments) Transfer blocking documents discovered to BlockingDocuments object

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

connectionMap

protected java.util.HashMap connectionMap

n

protected int n

jobQueueHash

protected java.util.HashMap jobQueueHash

jobConnection

protected java.util.HashMap jobConnection

activeConnections

protected java.util.HashMap activeConnections

setSizes

protected java.util.HashMap setSizes

maxConnectionCounts

protected java.util.HashMap maxConnectionCounts

maxSetSize

protected int maxSetSize

documentsProcessed

protected int documentsProcessed

blockingDocumentArray

protected java.util.ArrayList blockingDocumentArray

prioritizationTime

protected long prioritizationTime

Constructor Detail

JobManager.ThrottleLimit

public JobManager.ThrottleLimit(int n,
                                long prioritizationTime)

Constructor. This class is built up piecemeal, so the constructor does nothing.

Parameters:

n - is the maximum number of full job descriptions we want at this time.

Method Detail

tallyBlockingDocuments

public void tallyBlockingDocuments(BlockingDocuments blockingDocuments)

Transfer blocking documents discovered to BlockingDocuments object

addJob

public void addJob(java.lang.Long jobID,
                   java.lang.String connectionName)

Add a job/connection name map entry.

Parameters:

jobID - is the job id.

connectionName - is the connection name.

addConnectionName

public void addConnectionName(java.lang.String connectionName,
                              IRepositoryConnector connectorInstance)
                       throws ManifoldCFException

Add an active connection. This is the pool of active connections that will be used for the lifetime of this operation.

Parameters:

connectionName - is the connection name.

Throws:

ManifoldCFException

addLimit

public void addLimit(java.lang.String connectionName,
                     java.lang.String regexp,
                     int upperLimit)

Add a document limit for a specified connection. This is the limit across all matching bins; if any individual matching bin exceeds that limit, then documents that belong to that bin will be excluded.

Parameters:

connectionName - is the connection name.

regexp - is the regular expression, which we will match against various bins.

upperLimit - is the maximum count associated with the specified job.

setConnectionLimit

public void setConnectionLimit(java.lang.String connectionName,
                               int maxDocuments)

Set a connection-based total document limit.

doesCompareWork

public boolean doesCompareWork()

See if this class can be legitimately compared against another of the same type.

Specified by:

doesCompareWork in interface ILimitChecker

Returns:

true if comparisons will ever return "true".

duplicate

public ILimitChecker duplicate()

Create a duplicate of this class instance. All current state should be preserved. NOTE: Since doesCompareWork() returns false, queries using this limit checker cannot be cached, and therefore duplicate() is never called from the query executor. But it can be called from other places.

Specified by:

duplicate in interface ILimitChecker

Returns:

the duplicate.

makeDeepCopy

public JobManager.ThrottleLimit makeDeepCopy()

Make a deep copy

hashCode

public int hashCode()

Find the hashcode for this class. This will only ever be used if doesCompareWork() returns true.

Specified by:

hashCode in interface ILimitChecker

Overrides:

hashCode in class java.lang.Object

Returns:

the hashcode.

equals

public boolean equals(java.lang.Object object)

Compare two objects and see if equal. This will only ever be used if doesCompareWork() returns true.

Specified by:

equals in interface ILimitChecker

Overrides:

equals in class java.lang.Object

Parameters:

object - is the object to compare against.

Returns:

true if equal.

getRemainingDocuments

public int getRemainingDocuments()

Get the remaining documents we should query for.

Returns:

the maximal remaining count.

checkInclude

public boolean checkInclude(IResultRow row)
                     throws ManifoldCFException

See if a result row should be included in the final result set.

Specified by:

checkInclude in interface ILimitChecker

Parameters:

row - is the result row to check.

Returns:

true if it should be included, false otherwise.

Throws:

ManifoldCFException

checkContinue

public boolean checkContinue()
                      throws ManifoldCFException

See if we should examine another row.

Specified by:

checkContinue in interface ILimitChecker

Returns:

true if we need to keep going, or false if we are done.

Throws:

ManifoldCFException