org.apache.manifoldcf.crawler.connectors.webcrawler
Class ThrottleDescription

java.lang.Object
  org.apache.manifoldcf.crawler.connectors.webcrawler.ThrottleDescription

public class ThrottleDescription
extends java.lang.Object

This class describes complex throttling criteria pulled from a configuration. The data contained is organized by regular expression performed on a bin. What we store for each regular expression is a Pattern, for efficiency. This structure deals with bandwidth limits, maximum connection limits, and maximum fetch rate limits. Average fetch rate limits are handled in the infrastructure. Generally it is a good thing to limit the number of regexps that need to be evaluated against any given bin value as much as possible. For that reason I've organized this structure accordingly.

Nested Class Summary

protected static class

ThrottleDescription.ThrottleItem Class representing an individual throttle item.

Field Summary

static java.lang.String	_rcsid
protected java.util.HashMap	patternHash This is the hash that contains everything.

Constructor Summary

ThrottleDescription(org.apache.manifoldcf.core.interfaces.ConfigParams configData)
Constructor.

Method Summary

int	getMaxOpenConnections(java.lang.String binName) Given a bin name, find the max open connections to use for that bin.
double	getMinimumMillisecondsPerByte(java.lang.String binName) Look up minimum milliseconds per byte for a bin.
long	getMinimumMillisecondsPerFetch(java.lang.String binName) Look up minimum milliseconds for a fetch for a bin.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_rcsid

public static final java.lang.String _rcsid

See Also:

Constant Field Values

patternHash

protected java.util.HashMap patternHash

This is the hash that contains everything. It's keyed by the regexp string itself. Values are ThrottleItem's.

Constructor Detail

ThrottleDescription

public ThrottleDescription(org.apache.manifoldcf.core.interfaces.ConfigParams configData)
                    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Constructor. Build the description from the ConfigParams.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

Method Detail

getMaxOpenConnections

public int getMaxOpenConnections(java.lang.String binName)

Given a bin name, find the max open connections to use for that bin.

Returns:

-1 if no limit found.

getMinimumMillisecondsPerByte

public double getMinimumMillisecondsPerByte(java.lang.String binName)

Look up minimum milliseconds per byte for a bin.

Returns:

0.0 if no limit found.

getMinimumMillisecondsPerFetch

public long getMinimumMillisecondsPerFetch(java.lang.String binName)

Look up minimum milliseconds for a fetch for a bin.

Returns:

0 if no limit found.