Package | Description |
---|---|
bixo.operations | |
bixo.pipes |
Modifier and Type | Class and Description |
---|---|
class |
FixedScoreGenerator |
Constructor and Description |
---|
FilterAndScoreByUrlAndRobots(BaseFetcher fetcher,
BaseRobotsParser parser,
BaseScoreGenerator scorer) |
FilterAndScoreByUrlAndRobots(UserAgent userAgent,
int maxThreads,
BaseRobotsParser parser,
BaseScoreGenerator scorer) |
ProcessRobotsTask(java.lang.String protocolAndDomain,
BaseScoreGenerator scorer,
java.util.Queue<GroupedUrlDatum> urls,
BaseFetcher fetcher,
BaseRobotsParser parser,
cascading.tuple.TupleEntryCollector collector,
com.scaleunlimited.cascading.LoggingFlowProcess flowProcess) |
Constructor and Description |
---|
FetchPipe(cascading.pipe.Pipe urlProvider,
BaseScoreGenerator scorer,
BaseFetcher fetcher,
BaseFetcher robotsFetcher,
BaseRobotsParser parser,
BaseFetchJobPolicy fetchJobPolicy,
int numReducers) |
FetchPipe(cascading.pipe.Pipe urlProvider,
BaseScoreGenerator scorer,
BaseFetcher fetcher,
int numReducers)
Generate an assembly that will fetch all of the UrlDatum tuples coming out of urlProvider.
|
Copyright © 2012 Bixo Labs