public class FakeUserFetcherPolicy extends FetcherPolicy
FetcherPolicy.FetcherMode, FetcherPolicy.RedirectMode
_crawlDelay, DEFAULT_ACCEPT_LANGUAGE, DEFAULT_CRAWL_DELAY, DEFAULT_CRAWL_END_TIME, DEFAULT_MAX_CONNECTIONS_PER_HOST, DEFAULT_MAX_CONTENT_SIZE, DEFAULT_MAX_REDIRECTS, DEFAULT_MAX_REQUESTS_PER_CONNECTION, DEFAULT_MIN_PAGE_FETCH_INTERVAL, DEFAULT_MIN_RESPONSE_RATE, NO_CRAWL_END_TIME, NO_MIN_RESPONSE_RATE, NO_REDIRECTS
Constructor and Description |
---|
FakeUserFetcherPolicy() |
FakeUserFetcherPolicy(long crawlDelay) |
Modifier and Type | Method and Description |
---|---|
long |
getCrawlDelay()
The (default) crawl delay should be specified via the BaseFetchJobPolicy, not the FetcherPolicy
|
long |
getDefaultCrawlDelay() |
FetchRequest |
getFetchRequest(long now,
long crawlDelay,
int maxUrls) |
addValidMimeType, addValidMimeTypes, calcMaxUrls, equals, getAcceptLanguage, getCrawlEndTime, getFetcherMode, getMaxConnectionsPerHost, getMaxContentSize, getMaxRedirects, getMaxRequestsPerConnection, getMaxUrls, getMinPageFetchInterval, getMinResponseRate, getRedirectMode, getRequestTimeout, getValidMimeTypes, hashCode, isTerminateFetch, seMinPageFetchInterval, setAcceptLanguage, setCrawlDelay, setCrawlEndTime, setFetcherMode, setMaxConnectionsPerHost, setMaxContentSize, setMaxRedirects, setMaxRequestsPerConnection, setMinResponseRate, setRedirectMode, setRequestTimeout, setValidMimeTypes, toString
public FakeUserFetcherPolicy()
public FakeUserFetcherPolicy(long crawlDelay)
public long getDefaultCrawlDelay()
getDefaultCrawlDelay
in class FetcherPolicy
public long getCrawlDelay()
FetcherPolicy
getCrawlDelay
in class FetcherPolicy
public FetchRequest getFetchRequest(long now, long crawlDelay, int maxUrls)
Copyright © 2012 Bixo Labs