org.apache.manifoldcf.crawler.connectors.jdbc
Class JDBCConnector

java.lang.Object
  org.apache.manifoldcf.core.connector.BaseConnector
      org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
          org.apache.manifoldcf.crawler.connectors.jdbc.JDBCConnector

All Implemented Interfaces:

org.apache.manifoldcf.core.interfaces.IConnector, org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

public class JDBCConnector
extends org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

This interface describes an instance of a connection between a repository and ManifoldCF's standard "pull" ingestion agent. Each instance of this interface is used in only one thread at a time. Connection Pooling on these kinds of objects is performed by the factory which instantiates repository connectors from symbolic names and config parameters, and is pooled by these parameters. That is, a pooled connector handle is used only if all the connection parameters for the handle match. Implementers of this interface should provide a default constructor which has this signature: xxx(); Connectors are either configured or not. If configured, they will persist in a pool, and be reused multiple times. Certain methods of a connector may be called before the connector is configured. This includes basically all methods that permit inspection of the connector's capabilities. The complete list is: The purpose of the repository connector is to allow documents to be fetched from the repository. Each repository connector describes a set of documents that are known only to that connector. It therefore establishes a space of document identifiers. Each connector will only ever be asked to deal with identifiers that have in some way originated from the connector. Documents are fetched in three stages. First, the getDocuments() method is called in the connector implementation. This returns a set of document identifiers. The document identifiers are used to obtain the current document version strings in the second stage, using the getDocumentVersions() method. The last stage is processDocuments(), which queues up any additional documents needed, and also ingests. This method will not be called if the document version seems to indicate that no document change took place.

Nested Class Summary

protected static class	JDBCConnector.TableSpec This class represents data gleaned from a document specification, in a more usable form.
protected static class	JDBCConnector.VariableMap Variable map.
protected static class	JDBCConnector.VariableMapItem Variable map entry.

Field Summary

static java.lang.String	_rcsid
protected static java.lang.String[]	activitiesList
protected static java.lang.String	ACTIVITY_EXTERNAL_QUERY
protected JDBCConnection	connection
protected java.lang.String	databaseName
protected static java.util.HashMap	documentKnownColumns Special column names, as far as document queries are concerned
protected java.lang.String	host
protected java.lang.String	jdbcProvider
protected java.lang.String	password
protected java.lang.String	userName

Fields inherited from class org.apache.manifoldcf.core.connector.BaseConnector

currentContext, params

Fields inherited from interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

JOBMODE_CONTINUOUS, JOBMODE_ONCEONLY, MODEL_ADD, MODEL_ADD_CHANGE, MODEL_ADD_CHANGE_DELETE, MODEL_ALL, MODEL_PARTIAL

Constructor Summary

JDBCConnector()
Constructor.

Method Summary

protected static void	addConstant(JDBCConnector.VariableMap map, java.lang.String varName, java.lang.String value) Add string query constants
protected static boolean	addIDList(JDBCConnector.VariableMap map, java.lang.String varName, java.lang.String[] documentIdentifiers, boolean[] scanOnly) Build an idlist variable, and add it to the specified variable map.
void	addSeedDocuments(org.apache.manifoldcf.crawler.interfaces.ISeedingActivity activities, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec, long startTime, long endTime, int jobMode) Queue "seed" documents.
protected static void	addVariable(JDBCConnector.VariableMap map, java.lang.String varName, long variable) Add starttime and endtime query variables
protected static void	addVariable(JDBCConnector.VariableMap map, java.lang.String varName, java.lang.String variable) Add string query variables
protected void	applyAccessTokens(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd, java.lang.String version, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec) Apply access tokens to a repository document.
protected void	applyMetadata(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd, org.apache.manifoldcf.core.interfaces.IResultRow row) Apply metadata to a repository document.
java.lang.String	check() Check status of connection.
void	connect(org.apache.manifoldcf.core.interfaces.ConfigParams configParams) Connect.
protected static java.lang.String	createQueryString(java.lang.String queryText, java.util.ArrayList paramList) Create an entity identifier from a querystring and a parameter list.
void	disconnect() Close the connection.
protected static java.lang.String[]	getAcls(org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec) Grab forced acl out of document specification.
java.lang.String[]	getActivitiesList() Return the list of activities that this connector supports (i.e.
java.lang.String[]	getBinNames(java.lang.String documentIdentifier) Get the bin name string for a document identifier.
int	getConnectorModel() Model.
java.lang.String[]	getDocumentVersions(java.lang.String[] documentIdentifiers, java.lang.String[] oldVersions, org.apache.manifoldcf.crawler.interfaces.IVersionActivity activities, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec, int jobMode, boolean usesDefaultAuthority) Get document versions given an array of document identifiers.
java.lang.String	getJSPFolder() Return the path for the UI interface JSP elements.
int	getMaxDocumentRequest() Get the maximum number of documents to amalgamate together into one batch, for this connector.
protected void	getSession() Set up a session
void	outputConfigurationBody(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.lang.String tabName) Output the configuration body section.
void	outputConfigurationHeader(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.util.ArrayList tabsArray) Output the configuration header section.
void	outputSpecificationBody(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds, java.lang.String tabName) Output the specification body section.
void	outputSpecificationHeader(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds, java.util.ArrayList tabsArray) Output the specification header section.
protected static void	pack(java.lang.StringBuffer output, java.lang.String value, char delimiter) Stuffer for packing a single string with an end delimiter
protected static void	packFixedList(java.lang.StringBuffer output, java.lang.String[] values, char delimiter) Stuffer for packing lists of fixed length
protected static void	packList(java.lang.StringBuffer output, java.util.ArrayList values, char delimiter) Stuffer for packing lists of variable length
protected static void	packList(java.lang.StringBuffer output, java.lang.String[] values, char delimiter) Another stuffer for packing lists of variable length
java.lang.String	processConfigurationPost(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, org.apache.manifoldcf.core.interfaces.ConfigParams parameters) Process a configuration post.
void	processDocuments(java.lang.String[] documentIdentifiers, java.lang.String[] versions, org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec, boolean[] scanOnly) Process a set of documents.
java.lang.String	processSpecificationPost(org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds) Process a specification post.
protected static java.lang.String	quoteSQLString(java.lang.String input) Quote a sql string.
protected static java.lang.String	readAsString(java.lang.Object o) Make sure we read this field as a string
protected static void	substituteQuery(java.lang.String inputString, JDBCConnector.VariableMap inputMap, java.lang.StringBuffer outputQuery, java.util.ArrayList outputParams) Given a query, and a parameter map, substitute it.
protected static int	unpack(java.lang.StringBuffer sb, java.lang.String value, int startPosition, char delimiter) Unstuffer for the above.
protected static int	unpackFixedList(java.lang.String[] output, java.lang.String value, int startPosition, char delimiter) Unstuffer for unpacking lists of fixed length
protected static int	unpackList(java.util.ArrayList output, java.lang.String value, int startPosition, char delimiter) Unstuffer for unpacking lists of variable length.
void	viewConfiguration(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.ConfigParams parameters) View configuration.
void	viewSpecification(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds) View specification.

Methods inherited from class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

addSeedDocuments, getDocumentIdentifiers, getDocumentIdentifiers, getDocumentVersions, getDocumentVersions, getDocumentVersions, getDocumentVersions, getRelationshipTypes, getRemainingDocumentIdentifiers, processDocuments, releaseDocumentVersions, requestInfo

Methods inherited from class org.apache.manifoldcf.core.connector.BaseConnector

clearThreadContext, deinstall, getConfiguration, install, poll, setThreadContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.manifoldcf.core.interfaces.IConnector

clearThreadContext, deinstall, getConfiguration, install, poll, setThreadContext

Field Detail

_rcsid

public static final java.lang.String _rcsid

See Also:

Constant Field Values

ACTIVITY_EXTERNAL_QUERY

protected static final java.lang.String ACTIVITY_EXTERNAL_QUERY

See Also:

Constant Field Values

activitiesList

protected static final java.lang.String[] activitiesList

connection

protected JDBCConnection connection

jdbcProvider

protected java.lang.String jdbcProvider

host

protected java.lang.String host

databaseName

protected java.lang.String databaseName

userName

protected java.lang.String userName

password

protected java.lang.String password

documentKnownColumns

protected static java.util.HashMap documentKnownColumns

Special column names, as far as document queries are concerned

Constructor Detail

JDBCConnector

public JDBCConnector()

Constructor.

Method Detail

getSession

protected void getSession()
                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Set up a session

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getActivitiesList

public java.lang.String[] getActivitiesList()

Return the list of activities that this connector supports (i.e. writes into the log).

Specified by:

getActivitiesList in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getActivitiesList in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Returns:

the list.

getJSPFolder

public java.lang.String getJSPFolder()

Return the path for the UI interface JSP elements. These JSP's must be provided to allow the connector to be configured, and to permit it to present document filtering specification information in the UI. This method should return the name of the folder, under the /connectors/ area, where the appropriate JSP's can be found. The name should NOT have a slash in it.

Returns:

the folder part

getConnectorModel

public int getConnectorModel()

Model. Depending on what people enter for the seeding query, this could be either ALL or could be less than that. So, I've decided it will be at least the adds and changes, and won't include the deletes.

Specified by:

getConnectorModel in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getConnectorModel in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

connect

public void connect(org.apache.manifoldcf.core.interfaces.ConfigParams configParams)

Connect. The configuration parameters are included.

Specified by:

connect in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

connect in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

configParams - are the configuration parameters for this connection.

check

public java.lang.String check()
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Check status of connection.

Specified by:

check in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

check in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

disconnect

public void disconnect()
                throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Close the connection. Call this before discarding the repository connector.

Specified by:

disconnect in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

disconnect in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getBinNames

public java.lang.String[] getBinNames(java.lang.String documentIdentifier)

Get the bin name string for a document identifier. The bin name describes the queue to which the document will be assigned for throttling purposes. Throttling controls the rate at which items in a given queue are fetched; it does not say anything about the overall fetch rate, which may operate on multiple queues or bins. For example, if you implement a web crawler, a good choice of bin name would be the server name, since that is likely to correspond to a real resource that will need real throttle protection.

Specified by:

getBinNames in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getBinNames in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

documentIdentifier - is the document identifier.

Returns:

the bin name.

addSeedDocuments

public void addSeedDocuments(org.apache.manifoldcf.crawler.interfaces.ISeedingActivity activities,
                             org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec,
                             long startTime,
                             long endTime,
                             int jobMode)
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                             org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Queue "seed" documents. Seed documents are the starting places for crawling activity. Documents are seeded when this method calls appropriate methods in the passed in ISeedingActivity object. This method can choose to find repository changes that happen only during the specified time interval. The seeds recorded by this method will be viewed by the framework based on what the getConnectorModel() method returns. It is not a big problem if the connector chooses to create more seeds than are strictly necessary; it is merely a question of overall work required. The times passed to this method may be interpreted for greatest efficiency. The time ranges any given job uses with this connector will not overlap, but will proceed starting at 0 and going to the "current time", each time the job is run. For continuous crawling jobs, this method will be called once, when the job starts, and at various periodic intervals as the job executes. When a job's specification is changed, the framework automatically resets the seeding start time to 0. The seeding start time may also be set to 0 on each job run, depending on the connector model returned by getConnectorModel(). Note that it is always ok to send MORE documents rather than less to this method.

Specified by:

addSeedDocuments in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

addSeedDocuments in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

activities - is the interface this method should use to perform whatever framework actions are desired.

spec - is a document specification (that comes from the job).

startTime - is the beginning of the time range to consider, inclusive.

endTime - is the end of the time range to consider, exclusive.

jobMode - is an integer describing how the job is being run, whether continuous or once-only.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

getDocumentVersions

public java.lang.String[] getDocumentVersions(java.lang.String[] documentIdentifiers,
                                              java.lang.String[] oldVersions,
                                              org.apache.manifoldcf.crawler.interfaces.IVersionActivity activities,
                                              org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec,
                                              int jobMode,
                                              boolean usesDefaultAuthority)
                                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                              org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Get document versions given an array of document identifiers. This method is called for EVERY document that is considered. It is therefore important to perform as little work as possible here.

Specified by:

getDocumentVersions in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getDocumentVersions in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

documentIdentifiers - is the array of local document identifiers, as understood by this connector.

oldVersions - is the corresponding array of version strings that have been saved for the document identifiers. A null value indicates that this is a first-time fetch, while an empty string indicates that the previous document had an empty version string.

activities - is the interface this method should use to perform whatever framework actions are desired.

spec - is the current document specification for the current job. If there is a dependency on this specification, then the version string should include the pertinent data, so that reingestion will occur when the specification changes. This is primarily useful for metadata.

jobMode - is an integer describing how the job is being run, whether continuous or once-only.

usesDefaultAuthority - will be true only if the authority in use for these documents is the default one.

Returns:

the corresponding version strings, with null in the places where the document no longer exists. Empty version strings indicate that there is no versioning ability for the corresponding document, and the document will always be processed.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

processDocuments

public void processDocuments(java.lang.String[] documentIdentifiers,
                             java.lang.String[] versions,
                             org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities,
                             org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec,
                             boolean[] scanOnly)
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                             org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Process a set of documents. This is the method that should cause each document to be fetched, processed, and the results either added to the queue of documents for the current job, and/or entered into the incremental ingestion manager. The document specification allows this class to filter what is done based on the job.

Overrides:

processDocuments in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

documentIdentifiers - is the set of document identifiers to process.

versions - is the corresponding document versions to process, as returned by getDocumentVersions() above. The implementation may choose to ignore this parameter and always process the current version.

activities - is the interface this method should use to queue up new document references and ingest documents.

spec - is the document specification.

scanOnly - is an array corresponding to the document identifiers. It is set to true to indicate when the processing should only find other references, and should not actually call the ingestion methods.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

outputConfigurationHeader

public void outputConfigurationHeader(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                      org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                      java.util.ArrayList tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException

Output the configuration header section. This method is called in the head section of the connector's configuration page. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the configuration editing HTML.

Specified by:

outputConfigurationHeader in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

outputConfigurationHeader in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputConfigurationBody

public void outputConfigurationBody(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                    org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException

Output the configuration body section. This method is called in the body section of the connector's configuration page. Its purpose is to present the required form elements for editing. The coder can presume that the HTML that is output from this configuration will be within appropriate , , and

tags. The name of the form is "editconnection".

Specified by:

outputConfigurationBody in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

outputConfigurationBody in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

tabName - is the current tab name.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

processConfigurationPost

public java.lang.String processConfigurationPost(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                                 org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Process a configuration post. This method is called at the start of the connector's configuration page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the configuration parameters accordingly. The name of the posted form is "editconnection".

Specified by:

processConfigurationPost in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

processConfigurationPost in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

variableContext - is the set of variables available from the post, including binary file post information.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

Returns:

null if all is well, or a string error message if there is an error that should prevent saving of the connection (and cause a redirection to an error page).

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

viewConfiguration

public void viewConfiguration(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                              org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException

View configuration. This method is called in the body section of the connector's view configuration page. Its purpose is to present the connection information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate and tags.

Specified by:

viewConfiguration in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

viewConfiguration in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputSpecificationHeader

public void outputSpecificationHeader(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds,
                                      java.util.ArrayList tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException

Output the specification header section. This method is called in the head section of a job page which has selected a repository connection of the current type. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the job editing HTML.

Specified by:

outputSpecificationHeader in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

outputSpecificationHeader in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

out - is the output to which any HTML should be sent.

ds - is the current document specification for this job.

tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputSpecificationBody

public void outputSpecificationBody(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException

Output the specification body section. This method is called in the body section of a job page which has selected a repository connection of the current type. Its purpose is to present the required form elements for editing. The coder can presume that the HTML that is output from this configuration will be within appropriate , , and tags. The name of the form is "editjob".

Specified by:

outputSpecificationBody in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

outputSpecificationBody in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

out - is the output to which any HTML should be sent.

ds - is the current document specification for this job.

tabName - is the current tab name.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

processSpecificationPost

public java.lang.String processSpecificationPost(org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Process a specification post. This method is called at the start of job's edit or view page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the document specification accordingly. The name of the posted form is "editjob".

Specified by:

processSpecificationPost in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

processSpecificationPost in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

variableContext - contains the post data, including binary file-upload information.

ds - is the current document specification for this job.

Returns:

null if all is well, or a string error message if there is an error that should prevent saving of the job (and cause a redirection to an error page).

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

viewSpecification

public void viewSpecification(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              org.apache.manifoldcf.crawler.interfaces.DocumentSpecification ds)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException

View specification. This method is called in the body section of a job's view page. Its purpose is to present the document specification information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate and tags.

Specified by:

viewSpecification in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

viewSpecification in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

out - is the output to which any HTML should be sent.

ds - is the current document specification for this job.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

applyMetadata

protected void applyMetadata(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd,
                             org.apache.manifoldcf.core.interfaces.IResultRow row)
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Apply metadata to a repository document.

Parameters:

rd - is the repository document to apply the metadata to.

row - is the resultset row to use to get the metadata. All non-special columns from this row will be considered to be metadata.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

applyAccessTokens

protected void applyAccessTokens(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd,
                                 java.lang.String version,
                                 org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec)
                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Apply access tokens to a repository document.

Parameters:

rd - is the repository document to apply the access tokens to.

version - is the version string.

spec - is the document specification.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getMaxDocumentRequest

public int getMaxDocumentRequest()

Get the maximum number of documents to amalgamate together into one batch, for this connector.

Specified by:

getMaxDocumentRequest in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getMaxDocumentRequest in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Returns:

the maximum number. 0 indicates "unlimited".

addVariable

protected static void addVariable(JDBCConnector.VariableMap map,
                                  java.lang.String varName,
                                  long variable)

Add starttime and endtime query variables

addVariable

protected static void addVariable(JDBCConnector.VariableMap map,
                                  java.lang.String varName,
                                  java.lang.String variable)

Add string query variables

addConstant

protected static void addConstant(JDBCConnector.VariableMap map,
                                  java.lang.String varName,
                                  java.lang.String value)

Add string query constants

addIDList

protected static boolean addIDList(JDBCConnector.VariableMap map,
                                   java.lang.String varName,
                                   java.lang.String[] documentIdentifiers,
                                   boolean[] scanOnly)

Build an idlist variable, and add it to the specified variable map.

substituteQuery

protected static void substituteQuery(java.lang.String inputString,
                                      JDBCConnector.VariableMap inputMap,
                                      java.lang.StringBuffer outputQuery,
                                      java.util.ArrayList outputParams)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Given a query, and a parameter map, substitute it. Each variable substitutes the string, and it also substitutes zero or more query parameters.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getAcls

protected static java.lang.String[] getAcls(org.apache.manifoldcf.crawler.interfaces.DocumentSpecification spec)

Grab forced acl out of document specification.

Parameters:

spec - is the document specification.

Returns:

the acls.

pack

protected static void pack(java.lang.StringBuffer output,
                           java.lang.String value,
                           char delimiter)

Stuffer for packing a single string with an end delimiter

unpack

protected static int unpack(java.lang.StringBuffer sb,
                            java.lang.String value,
                            int startPosition,
                            char delimiter)

Unstuffer for the above.

packFixedList

protected static void packFixedList(java.lang.StringBuffer output,
                                    java.lang.String[] values,
                                    char delimiter)

Stuffer for packing lists of fixed length

unpackFixedList

protected static int unpackFixedList(java.lang.String[] output,
                                     java.lang.String value,
                                     int startPosition,
                                     char delimiter)

Unstuffer for unpacking lists of fixed length

packList

protected static void packList(java.lang.StringBuffer output,
                               java.util.ArrayList values,
                               char delimiter)

Stuffer for packing lists of variable length

packList

protected static void packList(java.lang.StringBuffer output,
                               java.lang.String[] values,
                               char delimiter)

Another stuffer for packing lists of variable length

unpackList

protected static int unpackList(java.util.ArrayList output,
                                java.lang.String value,
                                int startPosition,
                                char delimiter)

Unstuffer for unpacking lists of variable length.

Parameters:

output - is the array to write the unpacked result into.

value - is the value to unpack.

startPosition - is the place to start the unpack.

delimiter - is the character to use between values.

Returns:

the next position beyond the end of the list.

createQueryString

protected static java.lang.String createQueryString(java.lang.String queryText,
                                                    java.util.ArrayList paramList)

Create an entity identifier from a querystring and a parameter list.

quoteSQLString

protected static java.lang.String quoteSQLString(java.lang.String input)

Quote a sql string.

readAsString

protected static java.lang.String readAsString(java.lang.Object o)
                                        throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Make sure we read this field as a string

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException