org.apache.nutch.segment

Class SegmentReader

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.nutch.segment.SegmentReader

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,NutchWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

public class SegmentReader
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,NutchWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Dump the content of a segment.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SegmentReader.InputCompatMapper
static class	SegmentReader.SegmentReaderStats
static class	SegmentReader.TextOutputFormat Implements a text output format

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
SegmentReader()
SegmentReader(org.apache.hadoop.conf.Configuration conf, boolean co, boolean fe, boolean ge, boolean pa, boolean pd, boolean pt)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	configure(org.apache.hadoop.mapred.JobConf job)
void	dump(org.apache.hadoop.fs.Path segment, org.apache.hadoop.fs.Path output)
void	get(org.apache.hadoop.fs.Path segment, org.apache.hadoop.io.Text key, Writer writer, Map<String,List<org.apache.hadoop.io.Writable>> results)
void	getStats(org.apache.hadoop.fs.Path segment, SegmentReader.SegmentReaderStats stats)
void	list(List<org.apache.hadoop.fs.Path> dirs, Writer writer)
static void	main(String[] args)
void	reduce(org.apache.hadoop.io.Text key, Iterator<NutchWritable> values, org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> output, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

SegmentReader

public SegmentReader()

SegmentReader

public SegmentReader(org.apache.hadoop.conf.Configuration conf,
             boolean co,
             boolean fe,
             boolean ge,
             boolean pa,
             boolean pd,
             boolean pt)

Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf job)

Specified by:

configure in interface org.apache.hadoop.mapred.JobConfigurable

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

reduce

public void reduce(org.apache.hadoop.io.Text key,
          Iterator<NutchWritable> values,
          org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> output,
          org.apache.hadoop.mapred.Reporter reporter)
            throws IOException

Specified by:

reduce in interface org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,NutchWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

dump

public void dump(org.apache.hadoop.fs.Path segment,
        org.apache.hadoop.fs.Path output)
          throws IOException

Throws:

IOException

get

public void get(org.apache.hadoop.fs.Path segment,
       org.apache.hadoop.io.Text key,
       Writer writer,
       Map<String,List<org.apache.hadoop.io.Writable>> results)
         throws Exception

Throws:

Exception

list

public void list(List<org.apache.hadoop.fs.Path> dirs,
        Writer writer)
          throws Exception

Throws:

Exception

getStats

public void getStats(org.apache.hadoop.fs.Path segment,
            SegmentReader.SegmentReaderStats stats)
              throws Exception

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception