|
||||||||||
PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES |
See:
Description
Interface Summary | |
FieldExtractor | Interface to all information extraction components. |
RankedExtractor | Interface for information extraction components that support merging of two Instances, using field-level confidence ratings. |
Class Summary | |
AbstractFieldExtractor | Abstract superclass for implementations of the FieldExtractor interface. |
Confidence | Meant to serve as a parallel structure to the KAON Instance . |
ExtractDemo | This is a simple GUI front end that demonstrates the basic functionality of the information extraction packages. |
ExtractorUtilities | Utility class for deserializing FieldExtractors from files. |
SingleFieldExtractor | Superclass of FieldExtractors that only return a single (literal) field. |
This package implements various subpackages for information extraction. Some examples of use appear later in this description. At the moment, three types of information extraction are supported (where some of these have internal variants):
There are some demonstrations of the stuff here which you can run (and several
other classes have main()
methods which exhibit their
functionality):
ExtractDemo
is a simple GUI front-end to the information
extraction components. It isn't intended for serious use, but
illustrates loading and using various extractors, and includes
an interface to web search. This can be done via Google (using the
Google API) or Altavista or other engines (using screen scraping).
The web search
functionality is provided by the edu.stanford.nlp.web
package. You should try this out first (see the examples below).edu.stanford.nlp.ie.test.ExtractorTest
is a simple
command line test of the
information extraction components. It extracts only from a file.
0. Setup: For all of these examples except 3., you need to be
connected to the Internet, and for the application's web search module
to be
able to connect to search engines. The web search
functionality is provided by the supplied edu.stanford.nlp.web
package. How web search works is controlled
by a websearch.init
file in your current directory (or if
none is present, you will get search results from AltaVista). If
you are registered to use the GoogleAPI, you should probably edit
this file so web queries can be done to Google using their SOAP
interface. Even if not, you can specify additional or different
search engines to access in websearch.init
.
A copy of this file is supplied in the distribution. The
DescExtractor
in 4. also requires another init file so that
it can use the include part-of-speech tagger.
1. Corporate Contact Information. This illustrates simple information
extraction from a web page.
Using the included
ExtractDemo.bat
or by hand run:
java edu.stanford.nlp.ie.ExtractDemo
serialized-extractors/companycontact
serialized-extractors/companycontact/Corporation-Information.kaon
Corporation
as the Concept to extract.Extract
, and look at the results:
http://www.ziatech.com/
http://www.cs.stanford.edu/
http://www.ananova.com/business/story/sm_635565.html
Audiovox Corporation
2. Corporate Contact Information merged. This illustrates the addition
of information merger across web pages. Using the included
MergeExtractDemo.bat
or similarly do:
java edu.stanford.nlp.ie.ExtractDemo -m
ExtractDemo
screen is similar, but adds a button to
Select a Merger.
serialized-extractors/mergers
and Select the file
unscoredmerger.obj
.Audiovox Corporation
3. Company names via direct use of an HMM information extractor.
One can also train, load, and use HMM information extractors directly,
without using any of the RDF-based KAON framework
(http://kaon.semanticweb.org/
) used by ExtractDemo.
edu.stanford.nlp.ie.hmm.Tester
illustrates the use
of a pretrained HMM on data via the command line interface:
cd serialized-extractors/companycontact/
java edu.stanford.nlp.ie.hmm.Tester cisco.txt company
company-name.hmm
java edu.stanford.nlp.ie.hmm.Tester EarningsReports.txt
company company-name.hmm
java edu.stanford.nlp.ie.hmm.Tester companytest.txt
company company-name.hmm
Corpus
of several
documents, separated with ENDOFDOC, used as a document delimiter
inside a Corpus. This second use of Tester
expects to
normally have an annotated corpus on which it can score its answers.
Here, the corpus is unannotated, and so some of the output is
inappropriate, but it shows what is selected as the company name
for each document (it's mostly correct...).
The final example shows it running on a corpus that does have answers
marked in it. It does the testing with the XML elements stripped, but
then uses them to evaluate correctness.
ENDOFDOC
on them. Then one can
train (and then test) as follows. Training an HMM
(optimizing all its probabilities) takes a long time
(it depends on the speed of the computer, but 10 minutes or
so to adjust probabilities for a fixed structure, and often
hours if one additionally attempts structure learning).
cd edu/stanford/nlp/ie/training/
java -server edu.stanford.nlp.ie.hmm.Trainer companydata.txt
company mycompany.hmm
java edu.stanford.nlp.ie.hmm.HMMSingleFieldExtractor Company
mycompany.hmm mycompany.obj
java edu.stanford.nlp.ie.hmm.Tester testdoc.txt company
mycompany.hmm
ExtractDemo
. Note that company
in the second line must match the element name in the
marked-up data that you will train on, while
Company
in the third line must match the
relation name in the ontology over which you will extract with
mycompany.obj
. These two names need not be the
same. The last step then runs the trained HMM on a file.
4. Extraction of descriptions (such as biographical information about
a person or a description of an animal).
This does extraction of such descriptions
from a web page. This component uses a POS tagger, and looks for where
to find a path to it in the file
descextractor.init
in the current directory. So,
you should be in the root directory of the current archive,
which has such a file. Double click on the included
MergeExtractDemo.bat
in that directory, or by hand
one can equivalently do:
java edu.stanford.nlp.ie.ExtractDemo -m
serialized-extractors/description
serialized-extractors/description/Entity-NameDescription.kaon
serialized-extractors/mergers
and Select the file
unscoredmerger.obj
.Entity
as the Concept to extract.
Gareth Evans
Tawny Frogmouth
Christopher Manning
Joshua Nkomo
|
||||||||||
PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES |